July 2011 - galaxy-dev - lists.galaxyproject.org

[API] Apache error?
by Louise-Amélie Schmitt 02 Aug '11

02 Aug '11

Hello everyone, I'm working on a script that uploads files and launchs workflows on them, but I keep getting errors that appear more or less randomly when the display() and submit() functions are called. In a nutshell, there is a 1/3 chance the calls fail this way. Nevertheless, the actions are properly triggered in Galaxy, surprisingly. Here is an example: When I launch a workflow, I get the following traceback even though the workflow is properly executed: http://localhost/galaxy-dev/api/workflows?key=273c7b4e3aaffd3884ef715aaf780… File "automated_preprocessing.py", line 61, in expandFile result = submit( api_key, api_url + 'workflows', wf_data, return_formatted=False) File "common.py", line 100, in submit r = post( api_key, url, data ) File "common.py", line 44, in post return simplejson.loads( urllib2.urlopen( req ).read() ) File "/g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/socket.py", line 329, in read data = self._sock.recv(rbufsize) File "/g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/httplib.py", line 518, in read return self._read_chunked(amt) File "/g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/httplib.py", line 561, in _read_chunked raise IncompleteRead(''.join(value)) Failed to expand file A: Type: <class 'httplib.IncompleteRead'> IncompleteRead(118 bytes read) Therefore, I cannot get the results within the script. Any idea? Best, L-A

1 1

minor bug: double-quotes in dataset name
by Assaf Gordon 02 Aug '11

02 Aug '11

Hello, There's a small bug when entering a double-quote character in a dataset name: The double-quote confuses some later-on processing (templates and Display-Application XML), resulting in some problems. Example: This history has one BAM dataset with a name of: Hello "world" http://main.g2.bx.psu.edu/u/foobar/h/double-quotes-bug If you try to rename it (by clicking the "pencil" icon), you'll see only the "hello" part. If you try to load the BAM to the UCSC Genome browser, it will fail with this error message: Missing bigDataUrl setting from track of type=bam (Hello ) -gordon

2 1

visual cue for <conditional> group parameters
by Assaf Gordon 02 Aug '11

02 Aug '11

Hi, The following tiny patch adds a light-gray visual cue in the html form, to give a better idea as to which parameters go together. It looks acceptable even with nested <conditional> parameters. See it in action here: http://cancan.cshl.edu/publicgalaxy/root?tool_id=cshl_bowtie_pe1 http://cancan.cshl.edu/publicgalaxy/root?tool_id=cshl_tophat_pe1 It might not be the prettiest thing, but it works nicely with minimal code changes. Patch is here: http://cancan.cshl.edu/labmembers/gordon/files/conditional-group-css2.patch Comments are welcomed, -gordon

2 1

wrong static image URL when clicking re-run
by Assaf Gordon 02 Aug '11

02 Aug '11

Hi, A small bug: When clicking "re-run" to view a tool form, and the tool has embedded images, the URL of the images is not working. When viewing a tool that has embedded images, the relative <img> URL is "./static/images/histogram2.png". The tool's URL comes from: "http://main.g2.bx.psu.edu/tool_runner?tool_id=histogram_rpy" and so the absolute image URL becomes: "http://main.g2.bx.psu.edu/./static/images/histogram2.png" which works OK. But when viewing a tool by clicking "re-run", the tool's URL is: "http://main.g2.bx.psu.edu/galaxy/tool_runner/rerun?id=56" ("rerun" being a method in the "tool_runner" controller), which in turns makes the image absolute URL: "http://main.g2.bx.psu.edu/tool_runner/./static/images/XXXXX.png" - and it doesn't work. Note that the solution of changing all the images URLs in the XML files to start at the root (e.g. "/static/images/XXXX.png") is not optimal, because it will bypass the "prefix" URL that some people use (e.g. with "http://rave.cshl.edu/galaxy" instead of "http://rave.cshl.edu/"). a workaround (when using apache) is to add the following rewrite_rule: RewriteRule ^/tool_runner/static/(.*) /localdata1/galaxy/galaxy_prod/static/$1 [L] Or if using a "/galaxy" prefix: RewriteRule ^/galaxy/tool_runner/static/(.*) /localdata1/galaxy/galaxy_prod/static/$1 [L] -gordon

2 1

3 ideas to improve the user experience for GALAXY
by 孙翰菲 01 Aug '11

01 Aug '11

Hi, I'm a user of Galaxy, I think Galaxy is a good idea except some small parts. 1. Progress Bar for uploading file I know that implement a progress bar for every tool is quite hard. But maybe it's not so hard for displaying the current percent of uploading, it's really helpful and comfortable for users (a countdown timer will be better). 2. Drag and Drop I've used gmail on my firefox, and I really like the function of "Drag & Drop" when I upload attachments. With this function, I can drag multiple small files (for example,'bed' files) in the special region of the page, which is easy to use. 3.Start automatically when uploading finished Some large files, need quite a long time before the uploading finished. We need to wait until the uploading is done because we cannot assign a task to an unfinished uploading file. Maybe we can pre-assign tasks for them, and we can go to other works when we finish doing this, because the operation will automatically apply to them when they have been uploading completely. I can imagine a story like this: I am quite tired after a whole day's busy work, but some sequencing data still need to be analyzed because I will show it to my boss tommorow. Then I think of GALAXY can do it well. I open the GALAXY, drag in several sequencing data, and assign a workflow to deal with the data in 2 minutes. Then I can go to sleep safely because GALAXY will run the workflow automatically when uploading is finished. When I get to GALAXY the next day, all the preliminary results are show on the page of GALAXY. I think it's really cool because it will reduce a lot of repetitive works for the user, and in my opinion, that is the REAL biology analysis platform. Best Wishes, Hanfei Sun

2 1

GALAXY-GBROWSE : setting up incoming/outcomming functionalities (sharing data)
by BM 01 Aug '11

01 Aug '11

Hi, I am trying to set up the galaxy outgoing and incoming funtionalities in GBROWSE and GALAXY to send data between these applications. I have local instances of GALAXY and GBROWSE and I wonder how should I configure these. How this should look in the GBROWSE user interface (in the "share this track" section) ? How this should look in the GALAXY user interface ? My GBROWSE version : 2.38 and my GALAXY version : galaxy-galaxy-central-fd3391bcd903 Here the conf I set up : ------MY GALAXY SETTINGS------ I saw in a previous topic posted by nicki (http://gmod.827538.n3.nabble.com/Send-data-to-GBrowse-td2446346.html) that some files must be edtited : 1. universe_wsgi.ini I edited it as follows : GBrowse servers: tool-data/shared/gbrowse/gbrowse_build_sites.txt gbrowse_display_sites = wormbase,tair,modencode_worm,modencode_fly,yeast_sgd,perso 2. tool-data/shared/gbrowse/gbrowse_build_sites.txt I edited it as follows : #perso perso http://my_host.com/Gbrowse/cgi-bin/gbrowse_details/specie specie 3. datatypes_conf.xml I edited it as follows :  <display file="gbrowse/gbrowse_gff.xml" inherit="True" />  <display file="gbrowse/gbrowse_wig.xml" /> I have got the following files in "display_applications/gbrowse/" : gbrowse_gff.xml gbrowse_interval_as_bed.xml gbrowse_wig.xml ------MY GBROWSE SETTINGS------ I have edited the general section of "/var/www/GBROWSE/conf/specie.conf" according to the documentation http://gmod.org/wiki/GBrowse_Configuration_HOWTO#Track_Sharing_Options: [GENERAL] galaxy outgoing = http:/my_host.com:8080/tool_runner?tool_id=ncbi_blastn_wrapper galaxy incoming = https://my_host.com/Gbrowse/cgi-bin/gbgff ------------- First, I would like to look for the outgoing functionality from GBROWSE to GALAXY. Then, the incoming functionnality : from GALAXY to GBROWSE I hope someone can help me, I'm new to GALAXY. Thanks, greetings Ben

4 3

job status when SGE kills/aborts job
by Shantanu Pavgi 30 Jul '11

30 Jul '11

We are using SGE cluster with our galaxy install. We have specified resource and run-time limits for certain tools using tool specific drmaa URL configuration, e.g.: - run-time (h_rt, s_rt) - memory (vf, h_vmem). This helps scheduler in submitting jobs to an appropriate node and also prevent node from crashing because of excessive memory consumption. However, sometimes a job needs more resources and/or run-time than specified in the drmaa URL configuration. In such cases SGE kills particular job and we get email notification with appropriate job summary. However, the galaxy web interface doesn't show any error for such failures. The job table doesn't contain any related state/info as well. The jobs are shown in green-boxes meaning they completed without any failure. In reality these jobs have been killed/aborted by the scheduler. This is really confusing as there is inconsistency between job status indicated by the galaxy and SGE/drmaa. Has anyone else experienced and/or addressed this issue? Any comments or suggestions will be really helpful. Thanks, Shantanu.

5 6

move dataset from history to server
by colin molter 30 Jul '11

30 Jul '11

Hello, Is there a way to copy/move a dataset located in the history to a given location in the filesystem? As an administrator you can upload data from given directory to your datalibrary. Is there a way to do the opposite? I know that it is possible to download any dataset to your local computer but what I want is just to move the dataset constructed by galaxy to a given repository in my server filesystem. thank you, colin

1 0

exporting environment variables to SGE in galaxy
by Chaolin Zhang 30 Jul '11

30 Jul '11

Hi, I am setting up SGE in our galaxy mirror. One problem I have is that I cannot export environment variables of the specific users running the galaxy service. On command line, I did this by qsub -V script.sh. or add a line #$ -V in script.sh I tried to change /lib/galaxy/jobs/runners/sge.py and lib/galaxy/jobs/runners/drmaae.py by adding a line #$ -V under sge_template = """#!/bin/sh #$ -S /bin/sh but this did not help. Any idea is very appreciated! Chaolin

2 2

"Job output not returned from cluster"
by Edward Kirton 29 Jul '11

29 Jul '11

I've been getting these errors sometimes lately, particularly when the cluster is heavily loaded. The jobs have completed successfully, as I can see the output if I click the pen icon, but the job is in a failed state. Have any other sites been experiencing this problem? Or can the galaxy developers help shed some light on the issue? FYI, I use the outputs_to_working_directory option in universe_wsgi.ini so that i can use a faster/more reliable filesystem to collect output from the cluster. I'm not using the recently discussed patch to run jobs as the unix user. I'll continue to experiment with different filesystems and software settings.

3 4