- galaxy-dev - lists.galaxyproject.org

error after pulling latest updates
by Branden Timm 05 Jun '13

05 Jun '13

Hi All, Just did an update to HEAD, and upon restarting the daemons received the following messages: galaxy.jobs.handler DEBUG 2013-06-05 10:55:31,478 recovering job 2083 in condor runner Traceback (most recent call last): File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/app.py", line 164, in __init__ self.job_manager = manager.JobManager( self ) File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/jobs/manager.py", line 36, in __init__ self.job_handler.start() File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/jobs/handler.py", line 34, in start self.job_queue.start() File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/jobs/handler.py", line 77, in start self.__check_jobs_at_startup() File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/jobs/handler.py", line 125, in __check_jobs_at_startup self.dispatcher.recover( job, job_wrapper ) File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/jobs/handler.py", line 620, in recover self.job_runners[runner_name].recover( job, job_wrapper ) File "/home/GLBRCORG/galaxy/galaxy-central/lib/galaxy/jobs/runners/condor.py", line 243, in recover cjs.user_log = os.path.join( self.app.config.cluster_files_directory, '%s.condor.log' % galaxy_id_tag ) NameError: global name 'galaxy_id_tag' is not defined Removing PID file main.pid -- Branden Timm btimm(a)energy.wisc.edu

2 2

Tool Visibility
by Michael E. Cotterell 05 Jun '13

05 Jun '13

Suppose a tool didn't have any kind of configuration and therefore didn't need to be displayed in the "Analyze Data" part of Galaxy. Is there a way to modify tool_conf.xml so that a particular tool is only displayed in the Workflow editor? Thanks! Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA & TA, University of Georgia mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) mepcott(a)uga.edu (mailto:mepcott@uga.edu) mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) http://michaelcotterell.com/

3 10

Re: [galaxy-dev] Creating multiple datasets in a libset
by Neil.Burdett＠csiro.au 05 Jun '13

05 Jun '13

Hi Guys, Did you manage to get multiple datasets working? I can't seem to upload multiple files. Only the last file appears in the history. I changed my code as mentioned in the thread below in "example_watch_folder.py" to add multiple files separated by a new line and increased the sleep time: for fname in os.listdir(in_folder): fullpath = os.path.join(in_folder, fname) print ' fullpath is [%s] ' % fullpath if os.path.isfile(fullpath): data = {} data['folder_id'] = library_folder_id data['file_type'] = 'auto' data['dbkey'] = '' data['upload_option'] = 'upload_paths' data['filesystem_paths'] = "/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz\n /home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz" print ' data is [%s] ' % str(data['filesystem_paths']) data['create_type'] = 'file' libset = submit(api_key, api_url + "libraries/%s/contents" % library_id, data, return_formatted = False) #TODO Handle this better, but the datatype isn't always # set for the followup workflow execution without this # pause. time.sleep(65) However, I get the following crash: ./example_watch_folder.py 64f3209856a3cf4f2d034a1ad5bf851c http://barium-rbh/csiro/api/ /home/galaxy/galaxy-drop/input /home/galaxy/galaxy-drop/output "This One" f2db41e1fa331b3e fullpath is [/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz] data is [/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz /home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz] url is : http://barium-rbh/csiro/api/libraries/33b43b4e7093c91f/contents?key=64f3209… data is : {'file_type': 'auto', 'dbkey': '', 'create_type': 'file', 'folder_id': 'F33b43b4e7093c91f', 'upload_option': 'upload_paths', 'filesystem_paths': '/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T1_Screening.nii.gz\n /home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz'} url is : http://barium-rbh/csiro/api/workflows?key=64f3209856a3cf4f2d034a1ad5bf851c data is : {'workflow_id': 'f2db41e1fa331b3e', 'ds_map': {'14': {'src': 'ld', 'id': 'ff5476bcf6c921fa'}}, 'history': '141_S_0851_MRI_T2_Screening.nii.gz - apiFullCTE'} {'outputs': ['daecbdd824e1c349', '358eb58cd5463e0d', 'c0279aab05812500'], 'history': '3cc0effd29705aa3'} url is : http://barium-rbh/csiro/api/workflows?key=64f3209856a3cf4f2d034a1ad5bf851c data is : {'workflow_id': 'f2db41e1fa331b3e', 'ds_map': {'14': {'src': 'ld', 'id': '79966582feb6c081'}}, 'history': '141_S_0851_MRI_T2_Screening.nii.gz - apiFullCTE'} {'outputs': ['19c51286b777bc04', '0f71f1fc170d4ab9', '256444f6e7017e58'], 'history': 'b701da857886499b'} Traceback (most recent call last): File "./example_watch_folder.py", line 89, in <module> main(api_key, api_url, in_folder, out_folder, data_library, workflow ) File "./example_watch_folder.py", line 75, in main shutil.move(fullpath, os.path.join(out_folder, fname)) File "/usr/lib/python2.7/shutil.py", line 299, in move copy2(src, real_dst) File "/usr/lib/python2.7/shutil.py", line 128, in copy2 copyfile(src, dst) File "/usr/lib/python2.7/shutil.py", line 82, in copyfile with open(src, 'rb') as fsrc: IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-drop/input/141_S_0851_MRI_T2_Screening.nii.gz' It says there is no such file, but this file has already been copied from the input to the output directory. Any help much appreciated Neil ------------------------------ Message: 2 Date: Mon, 29 Apr 2013 16:11:39 -0400 From: Rob Leclerc <robert.leclerc(a)gmail.com> To: Dannon Baker <dannon.baker(a)gmail.com> Cc: "galaxy-dev(a)bx.psu.edu" <galaxy-dev(a)bx.psu.edu> Subject: Re: [galaxy-dev] Creating multiple datasets in a libset Message-ID: <CAGkd85fHSgO2YC1T+Frctyso9G5rfQb=_mLyHGSdxPM+s3=8pg(a)mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hi Dannon, I've written some code to (i) query a dataset to ensure that it's been uploaded after a submit and (ii) to ensure a resulting dataset has been written to the file. *#Block until all datasets have been uploaded* libset = submit(api_key, api_url + "libraries/%s/contents" % library_id, data, return_formatted = False) for ds in libset: while True: uploaded_file = display(api_key, api_url + 'libraries/%s/contents/%s' %(library_id, ds['id']), return_formatted=False) if uploaded_file['misc_info'] == None: time.sleep(1) else: break *#Block until all result datasets have been saved to the filesystem* result_ds_url = api_url + 'histories/' + history_id + '/contents/' + dsh['id']; while True: result_ds = display(api_key, result_ds_url, return_formatted=False) if result_ds["state"] == 'ok': break else: time.sleep(1) Rob Leclerc, PhD <http://www.linkedin.com/in/robleclerc> <https://twitter.com/#!/robleclerc> P: (US) +1-(917)-873-3037 P: (Shanghai) +86-1-(861)-612-5469 Personal Email: rob.leclerc(a)aya.yale.edu On Mon, Apr 29, 2013 at 11:18 AM, Dannon Baker <dannon.baker(a)gmail.com>wrote: > Yep, that example filesystem_paths you suggest should work fine. The > sleep() bit was a complete hack from the start, for simplicity in > demonstrating a very basic pipeline, but what you probably want to do for a > real implementation is query the dataset in question via the API, verify > that the datatype/etc have been set, and only after that execute the > workflow; instead of relying on sleep. > > > On Mon, Apr 29, 2013 at 9:24 AM, Rob Leclerc <robert.leclerc(a)gmail.com>wrote: > >> Hi Dannon, >> >> Thanks for the response. Sorry to be pedantic, but just to make sure that >> I understand the interpretation of this field on the other side of the API, >> I would need to have something like the following: >> >> data['filesystem_paths'] = "/home/me/file1.vcf \n /home/me/file2.vcf /n >> /home/me/file3.vcf" >> >> I assume I should also increase the time.sleep() to reflect the uploading >> of extra files? >> >> Cheers, >> >> Rob >> >> Rob Leclerc, PhD >> <http://www.linkedin.com/in/robleclerc><https://twitter.com/#!/robleclerc> >> P: (US) +1-(917)-873-3037 >> P: (Shanghai) +86-1-(861)-612-5469 >> Personal Email: rob.leclerc(a)aya.yale.edu >> >> >> On Mon, Apr 29, 2013 at 9:15 AM, Dannon Baker <dannon.baker(a)gmail.com>wrote: >> >>> Hey Rob, >>> >>> That example_watch_folder.py does just submit exactly one at a time, >>> executes the workflow, and then does the next all in separate transactions. >>> If you wanted to upload multiple filepaths at once, you'd just append more >>> to the ''filesystem_paths' field (newline separated paths). >>> >>> -Dannon >>> >>> >>> On Fri, Apr 26, 2013 at 11:54 PM, Rob Leclerc <robert.leclerc(a)gmail.com>wrote: >>> >>>> I'm looking at example_watch_folder.py and it's not clear from the >>>> example how you submit multiple datasets to a library. In the example, the >>>> first submit returns a libset [] with only a single entry and then proceeds >>>> to iterate through each dataset in the libset in the following section: >>>> >>>> data = {} >>>> >>>> data['folder_id'] = library_folder_id >>>> >>>> data['file_type'] = 'auto' >>>> >>>> data['dbkey'] = '' >>>> >>>> data['upload_option'] = 'upload_paths' >>>> >>>> >>>> >>>> *data['filesystem_paths'] = fullpath* >>>> >>>> data['create_type'] = 'file' >>>> >>>> libset = submit(api_key, api_url + "libraries/%s/contents" % >>>> library_id, data, return_formatted = False) >>>> >>>> time.sleep(5) >>>> >>>> for ds in libset: >>>> >>>> if 'id' in ds: >>>> >>>> wf_data = {} >>>> >>>> wf_data['workflow_id'] = workflow['id'] >>>> >>>> wf_data['history'] = "%s - %s" % (fname, >>>> workflow['name']) >>>> >>>> wf_data['ds_map'] = {} >>>> >>>> for step_id, ds_in in workflow['inputs' >>>> ].iteritems(): >>>> >>>> wf_data['ds_map'][step_id] = {'src':'ld', >>>> 'id':ds['id']} >>>> >>>> res = submit( api_key, api_url + 'workflows', >>>> wf_data, return_formatted=False) >>>> >>>> >>>> >>>> Rob Leclerc, PhD >>>> <http://www.linkedin.com/in/robleclerc><https://twitter.com/#!/robleclerc> >>>> P: (US) +1-(917)-873-3037 >>>> P: (Shanghai) +86-1-(861)-612-5469 >>>> Personal Email: rob.leclerc(a)aya.yale.edu >>>> >>>> ___________________________________________________________ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> http://lists.bx.psu.edu/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>> http://galaxyproject.org/search/mailinglists/ >>>> >>> >>> >> >

4 7

none mapping is not currently available!
by Sameer Dixit 05 Jun '13

05 Jun '13

Hi all, I tried to run a small workflow on Galaxy, which basically attempts to establish a correlation between the Chip-Chip, Polymerase binding and Dnase Hypersensitivity data. The input would be these 3 files, I am initially performing a liftover on the Chip-Chip data to convert from hg18 to hg19 and then using the output to establish the correlation. Here, you can see a screenshot of the pipeline here: As I am running this workflow, the desired output isn't obtained and the following error: "none mapping is not currently available" is obtained. Can someone please explain what this means, and what am I to do after this! Thanks a lot people!! -Sameer

1 0

Error / import rpy module
by Sarah Maman 05 Jun '13

05 Jun '13

Hello, Some Galaxy tools failed to import rpy module and displays the following message: Traceback (most recent call last): File "/usr/local/bioinfo/src/galaxy/galaxy-dist/tools/plotting/histogram.py", line 5, in <module> from rpy import * ImportError: No module named rpy For instance, the tool "Regrouping" (tools/stats/grouping.py) have been refactored in 2011 by Kanwei Li to use numpy instead of rpy. And this tool runs on my local instance of Galaxy. But I would like to be able use rpy module instead of refactor the others tools. Could you, please, help me to understand why the eggs ' import does not allow the use of rpy module? How to debug these tools? Thnaks in advance, Sarah -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

5 7

Creating workflow which includes Multifile upload
by Alex.Khassapov＠csiro.au 05 Jun '13

05 Jun '13

Hi John, One more problem with multifile upload - when I display a workflow which includes multi upload tool, I get: Module workflow_run_mako:476 in render_row_for_param >> <http://140.79.7.98/workflow/run?id=f597429621d6eb2b> __M_writer(unicode(param.get_label())) AttributeError: 'UploadDataset' object has no attribute 'get_label' Ok, I see that UploadDataset class is derived from Group, not ToolParameter. So I tried to add get_label() to the Group class, which returns some string. But then I get: Module workflow_run_mako:476 in render_row_for_param >> <http://140.79.7.98/workflow/run?id=f597429621d6eb2b> __M_writer(unicode(param.get_label())) TypeError: 'str' object is not callable Here my knowledge of Galaxy ends and I need some help please. -Alex

3 4

Local instance - Problem when log in/out
by Stephanie LE GRAS 05 Jun '13

05 Jun '13

Hi everyone, We have set up a local instance of Galaxy within our research institute and we are encountering an error when logging in/out. I've attached a screenshot of what's displayed on the screen when I log out. It is kind of a double galaxy menu and we don't understand why it is like this. [cid:7DC11615-6B60-474B-BD6E-6C118D96826D] After logging out: If you click on the link "go to home page", then you're re-directed to the login page. When logging in, it's working and no more trouble. But if you click on the link "Go back to the page you were visiting", you are re-directed to the login page. And when you log in over, you arrive one more time on a page with a double menu, an error pop up comes out (see attached Error.png) and finally, you're redirected to the login page again. Once you log in again, you can finally start working using Galaxy. Error: [cid:17FFC87F-7D93-43A8-AF3A-35E69332C685] Has someone already encountered such an error? Do you know how we could solve it? We would like to get the same behavior as it is on the Galaxy main instance knowing that only registered user can use our Galaxy local instance. Thank you in advance for your help. Stephanie -- Stephanie Le Gras Bioinformatics engineer High throughput sequencing platform IGBMC 1, rue Laurent Fries 67404 ILLKIRCH Cedex France Tel. : +33 (0)3 88 65 32 73 Tel. Solexa : +33 (0)3 88 65 32 97

2 3

Internal Server Error during installation of ncbi_blast_plus
by Keilwagen, Jens 05 Jun '13

05 Jun '13

Hi guys, we are currently estalishing our own local Galaxy instance. However, when we try to install "Repository 'ncbi_blast_plus'" from the Galaxy main tool shed via the admin pages, we obtain the following error "Internal Server Error Galaxy was unable to sucessfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team." We tried to install on three different days. Hence, we believe there might be another problem. Thanks for any help. Best regards, Jens

3 10

Adding a Tool for PANAMA
by Saket Choudhary 05 Jun '13

05 Jun '13

As part of a warmup, I added a tool for perming eQTL analysis using PANAMA( http://ml.sheffield.ac.uk/qtl/panama/) By default PANAMA writes to stderr and creates a PANAM_results.csv file in the same directory as the input files. I wrote a python wrapper https://bitbucket.org/saketkc/galaxy-central/src/3eb26205af5a0b77fa6177d5ec… for this. I however also came across how stderr text can be handled in Galaxy : http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Cstdio.3E.2C_… and thus wrote a new XML : https://gist.github.com/saketkc/5697388 Apparently the above XML runs too, but I dont get back a CSV. Essentially I need to move the PANAMA_results.csv file to output1. Can this be done without using a wrapper at all , just using the XML files ? Is there a way to nest the commands in the XML so that I could do a <move command> after <panama command> , but how would the paths be specified ? Saket

3 5

mapping Summary from Bowtie
by sridhar srinivasan 04 Jun '13

04 Jun '13

Dear Developers, I am running Bowtie2 through galaxy interface, previously it gives the the mapping summary and convert the format to sam.. but now it directly convert the fastq file to bam and doesnt give the mapping summary. how to get the mapping summary from bowtie2 using galaxy interface.. Thanks Sridhar

2 1