November 2012 - galaxy-dev - lists.galaxyproject.org

Galaxy and ENCODE
by Scooter Willis 20 Nov '12

20 Nov '12

I found out about Galaxy based on the ENCODE project making a big deal about packaging the software they used on a CD and available for download. I misplaced the CD and following some link on an ENCODE/Nature web site found Galaxy. What I haven't found is any public workflows that would indicate this is what ENCODE was doing to call SNPs or assemble RNAseq data. It appears that number of public workflows are minimal. Maybe I am not pointing to the correct repository. Can someone provide some guidance on ENCODE use of Galaxy and where I can find ENCODE workflows? I seem to remember the ENCODE Nature publication having a CD but never looked at it. If this was the case did ENCODE package a vmware'ish version of Galaxy on the CD? Thanks Scooter

3 3

Re: [galaxy-dev] [galaxy-user] Local Install - Understanding Upload File Concept
by Nate Coraor 19 Nov '12

19 Nov '12

On Nov 16, 2012, at 12:42 PM, greg wrote: > On Fri, Nov 16, 2012 at 12:28 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: >> If you follow the log, you should see messages from galaxy.jobs, and specifically, galaxy.jobs.runners.drmaa. >> > No, I don't see anything like that in the log after I started the > upload. Any ideas on how to see what might be going wrong there? Could you provide the contents of the log from the time you click the execute button on the tool form until it turns red in the UI? I just noticed this thread was on the user list, so I've moved it over to the dev list. --nate > > -Greg

2 4

Sam-to-Bam error
by Nicholas Tucker 19 Nov '12

19 Nov '12

Dear all, I have mapped some Illumina reads to a reference using Bowtie. I am trying to use Sam Tools to convert Sam to Bam on our local galaxy, but it always fails: Traceback (most recent call last): File "/export/galaxy/galaxy-central/lib/galaxy/jobs/runners/local.py", line 155, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/export/galaxy/galaxy-central/lib/galaxy/jobs/__init__.py", line 396, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/export/galaxy/galaxy-central/lib/galaxy/datatypes/binary.py", line 213, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found Tool execution generated the following error message: Unable to finish job The resulting failure always generates a large file, as if it has almost completed…… Anybody got an idea what the error might be and how to solve it please? Thanks in advance, Nick

2 1

Uncaught exception queueing job
by Carlos Borroto 19 Nov '12

19 Nov '12

Hi all, I ran into this error while uploading a ~300mb file. I'm using latest galaxy-dist, Apache proxy and Torque/Maui throught DRMAA. After this error appear in the log, the dataset stays in grey. If I restart the galaxy daemon, the job gets correctly submitted. galaxy.jobs.runners.drmaa ERROR 2012-11-16 16:29:49,833 Uncaught exception queueing job Traceback (most recent call last): File "/local/opt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 145, in run_next self.queue_job( obj ) File "/local/opt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 233, in queue_job job_id = self.ds.runJob(jt) File "build/bdist.linux-x86_64/egg/drmaa/__init__.py", line 331, in runJob _h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate) File "build/bdist.linux-x86_64/egg/drmaa/helpers.py", line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer)))) File "build/bdist.linux-x86_64/egg/drmaa/errors.py", line 90, in error_check raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value)) InternalException: code 1: pbs-drm: unknown error The same error happens even if I don't upload the file directly but use something like scp. Another file of only a few kbs doesn't show this behavior. I have the feeling this might be a memory issue, as the box running galaxy has only 2gb of RAM. Please let me know if there is something else I could do to help figuring out this issue. Thanks, Carlos

1 2

Multiple inputs in a workflow (variable number of inputs) ?
by LUO Yufei 17 Nov '12

17 Nov '12

Hello, The number of inputs is not fixed for my workflow, maybe I have 6 inputs, maybe more than that, the number of inputs depends on what data I have for the analysis. As I know, in a workflow, all inputs should be pre-defined...that is to say, the number of inputs is known before running one workflow. Do you have any idea for this problem? How can I do to make a workflow with the variable number of inputs? In advance, thanks for the answer, Yufei

3 3

Converting __oc__ and __cc__ back to curly braces?
by Samuel Lampa 16 Nov '12

16 Nov '12

Hi, I'm creating a galaxy tool [0] for querying external SPARQL (the Semantic Web query language) endpoints to get back data in tabular formats. I run into a problem though, since SPARQL queries contain curly braces, and Galaxy obviously converts these to an internal representation __oc__ and __cc__ (I found the commit for this at [1]) My SPARQL query for testing looks like this: SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 25 Thus, when I send this to my (python) script as a commmand line argument, the curly braces comes in galaxy's internal representation. Thus I was wondering if there is a way to get the braces in their correct format from galaxy, or if I should just go agead and convert them back in my own script? (i.e. will this behaviour be stable enough to rely on?) Best // Samuel [0] See the sparql_import.* files in https://github.com/samuell/galaxy_semwebtools [1] http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-February/001837.html -- Developer at SNIC-UPPMAX www.uppmax.uu.se Developer at Dept of Pharm Biosciences www.farmbio.uu.se Twitter - twitter.com/samuellampa Blog - saml.rilspace.org G+ - gplus.to/saml

2 3

Re: [galaxy-dev] [galaxy-user] new_file_path setting in universe_wsgi.ini
by Nate Coraor 16 Nov '12

16 Nov '12

On Nov 9, 2012, at 9:24 AM, greg wrote: > I see this setting in the universe_wsgi.ini file: > > # Temporary files are stored in this directory. > new_file_path = /scratch/galaxy > > Is this only used by the web server or will job runners pick up this > setting and use it for temp files? > > > I'm also wondering: > > 1. What sort of files go here? > 2. How much space does it need? Hi Greg, I've moved this over to the galaxy-dev list since it concerns a local installation. Jobs will use this directory if passed the value of $__new_file_path__ on the command line. It's up to the tool to decide whether to use this and how it will be used. How much space it needs will depend entirely on what tools you use and what inputs and parameters you use with those tools. --nate > > Thanks, > > Greg > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

2 2

Is there any timeout in Galaxy
by Jennifer Jackson 16 Nov '12

16 Nov '12

Hi Akshay, Please post all technical questions directly to the galaxy-dev mailing list. I'm sure we'll be able to help out shortly, Jen Galaxy team http://wiki.galaxyproject.org/Support#Mailing_Lists Hello, My name is Akshay and I am a computer science graduate student at UGA in Dr. Kissinger's Lab. We have created an extension to galaxy which allowed users to add web services as tools into galaxy, these could be either used as stand-alone or in a workflow. I wanted to know if Galaxy employs any time out strategy. Consider a case where I have a workflow with BLASTP now it takes some time to run blast and then get result form it. When I try that experiment as standalone (i.e., running the run method first and then after several mins getting result using job id works) however when I create a workflow in which one tool being run method of blast followed by getresult, galaxy seems to time out. Would appreciate your feedback on the same. -Akshay Choche

2 1

"This link may not be followed from within Galaxy" using nginx and upload module.
by Marcelo Timmerman 16 Nov '12

16 Nov '12

Any idea would be appreciated. I get a "This link may not be followed from within Galaxy" after following these steps: Get Data -> Upload File -> File Format: Type "lpe" -> select "lped" from autocomplete list. I've compiled latest stable nginx in CentOS 6.3 with upload module http://www.grid.net.ru/nginx/upload.en.html Galaxy being started by galaxy user nginx started by root user (this scenario reports a nginx worker process owned by galaxy) I'm serving Galaxy at a subdirectory as explained in http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy I'm not using External user authentication I suspect this is a problem in nginx.conf upload module directives. Do you know if it's ok to change location /_x_accel_redirect/ { with location /galaxy/_x_accel_redirect/ { ? Besides I couldn't find an _upload_done directory. Do you have an "_upload_done" or "_upload" directory? Where it's located? This is my nginx.conf server { ... more directives location /galaxy { proxy_pass http://galaxy_app; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } location /galaxy/static { alias /home/galaxy/galaxy-dist/static; expires 24h; } ....... # more directives location /galaxy/_x_accel_redirect/ { internal; alias /; } location /galaxy/_upload { upload_store /home/galaxy/galaxy-dist/database/tmp/upload_store; upload_pass_form_field ""; upload_set_form_field "__${upload_field_name}__is_composite" "true"; upload_set_form_field "__${upload_field_name}__keys" "name path"; upload_set_form_field "${upload_field_name}_name" "$upload_file_name"; upload_set_form_field "${upload_field_name}_path" "$upload_tmp_path"; upload_pass_args on; upload_pass /_upload_done; } location /galaxy/_upload_done { set $dst /tool_runner/index; if ($args ~ nginx_redir=([^&]+)) { set $dst $1; } rewrite "" $dst; } } Cheers

2 1

Scatter plot integration in pages
by Joachim Jacob 16 Nov '12

16 Nov '12

Dear devs, How will the scatterplot in the future be integrated in Galaxy, e.g, for saving and displaying a scatterplot in pages? Kind regards Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

3 3