- galaxy-dev - lists.galaxyproject.org

Move files between histories?
by Thyssen, Gregory - ARS 20 Nov '12

20 Nov '12

Hello Is it possible to move a file from one history to another? I have generated BAM files in separate histories that now must be merged. How can I do this without downloading and then uploading the files? Thanks, Gregory Thyssen, PhD Molecular Biologist Cotton Fiber Bioscience USDA-ARS-Southern Regional Research Center 1100 Robert E Lee Blvd New Orleans, LA 70124 gregory.thyssen(a)ars.usda.gov 504-286-4280 This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

2 1

First questions from a new developer
by Joshua Orvis 20 Nov '12

20 Nov '12

I'm building a pipeline which, initially, will use some existing tools and then I plan on adding some of my own. In running this though I've run into a few questions that might have to do with how Galaxy in general works rather than being tool-specific. I've made it up to running Realigner Target Creator (part of GATK tools) and I don't see a convention of specifying what version of GATK it expects to be installed. Then, running the tool failed with "Running with the -et NO_ET or -et STDOUT option requires a GATK Key file." It seems the -et option is hard-coded into the component without passing a key, which has to be applied for. I didn't want this at all, so I modified the following file: galaxy-dist/tools/gatk/realigner_target_creator.xml and removed the following line: -et "NO_ET" ##ET no phone home Then I added the component and tried to run again, still getting the same error. If I removed that line how is it still part of the call? Is Galaxy caching the xml tool files? Also, is there a way to see the precise commands being executed by Galaxy for any particular tool? Thanks - Joshua

2 2

format sensitive tools not working properly after upgrade
by Langhorst, Brad 20 Nov '12

20 Nov '12

I performed an upgrade over the weekend. Since then, I've observed an odd behavior - bowtie2 will not list the new fastqsanger files in my history. It will work with files that had already been groomed before the upgrade. I also see that the groomer is not recognizing fastq files. The format does seem to be correctly set for these datasets. Is there more than one place that a dataset's format is stored (and therefore get out of sync)? Could I have messed up the data types xml file? It looks ok. Seems unlikely that this would be a global problem since nobody else has reported it… but it does seem a bit subtle (only affecting newly tweaked files) Any ideas? Brad -- Brad Langhorst langhorst(a)neb.com<mailto:langhorst@neb.com>

1 0

Unable to run jobs in Galaxy: runners/local.py # stdout_file.close() # python2.7/tempfile.py # self.unlink(self.name) # OSError: [Errno 2] No such file or directory
by Tim te Beek 20 Nov '12

20 Nov '12

Hi all Since updating to the latest Galaxy-dist version, I'm unable to run any jobs. galaxy.jobs.handler INFO 2012-11-20 12:22:09,018 (6403) Job dispatched galaxy.objectstore CRITICAL 2012-11-20 12:22:15,362 /home/galaxy/galaxy-dist/database/job_working_directory/006/6403 delete error [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/006/6403' CRITICAL 12:22:15 __init__.delete:322 /home/galaxy/galaxy-dist/database/job_working_directory/006/6403 delete error [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/006/6403' galaxy.jobs.runners.local ERROR 2012-11-20 12:22:15,363 failure running job 6403 Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in run_job stdout_file.close() File "/usr/lib/python2.7/tempfile.py", line 410, in close self.unlink(self.name) OSError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/006/6403/tmpKuTwAJ_stdout' ERROR 12:22:15 local.run_job:134 failure running job 6403 Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in run_job stdout_file.close() File "/usr/lib/python2.7/tempfile.py", line 410, in close self.unlink(self.name) OSError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/006/6403/tmpKuTwAJ_stdout' Exception OSError: (2, 'No such file or directory', '/home/galaxy/galaxy-dist/database/job_working_directory/006/6403/tmpK7i5n4_stderr') in <bound method _TemporaryFileWrapper.__del__ of <closed file '<fdopen>', mode 'w+b' at 0x55b8f60>> ignored galaxy.objectstore CRITICAL 2012-11-20 12:22:15,723 /home/galaxy/galaxy-dist/database/job_working_directory/006/6404 delete error [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/006/6404' CRITICAL 12:22:15 __init__.delete:322 /home/galaxy/galaxy-dist/database/job_working_directory/006/6404 delete error [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/006/6404' I'm running the version from ~Nov 6th: 8153:5dcbbdfe1087 tip | Allow rerun to access hidden datasets. I already found this recent issue on the maillinglist which appeared similar, but my problems involve running jobs, not uploading datasets: http://dev.list.galaxyproject.org/Re-Galaxy-at-Duke-td4656946.html My setup involves 1 runner instance and 2 web instances, all inside the same VM, as per my configuration files here: https://trac.nbic.nl/brs2010p33/browser/trunk/galaxy-dist/ The folder /work/project is an sshfs mount to the server running the VM. sshfs#backup@192.168.122.1:/work/odose/ /work/project/ fuse defaults,idmap=user,allow_other,cache=no 0 0 I've setup the following paths to that mounted folder: # Dataset files are stored in this directory. file_path = /work/project/galaxy-dist/database/files # Temporary files are stored in this directory. new_file_path = /work/project/galaxy-dist/database/tmp The job_working_directory is a normal folder in the VM, but I still get the above error. Can anyone tell me what I should do to be able to run jobs in Galaxy again? Best, Tim

1 0

Galaxy and ENCODE
by Scooter Willis 20 Nov '12

20 Nov '12

I found out about Galaxy based on the ENCODE project making a big deal about packaging the software they used on a CD and available for download. I misplaced the CD and following some link on an ENCODE/Nature web site found Galaxy. What I haven't found is any public workflows that would indicate this is what ENCODE was doing to call SNPs or assemble RNAseq data. It appears that number of public workflows are minimal. Maybe I am not pointing to the correct repository. Can someone provide some guidance on ENCODE use of Galaxy and where I can find ENCODE workflows? I seem to remember the ENCODE Nature publication having a CD but never looked at it. If this was the case did ENCODE package a vmware'ish version of Galaxy on the CD? Thanks Scooter

3 3

Re: [galaxy-dev] [galaxy-user] Local Install - Understanding Upload File Concept
by Nate Coraor 19 Nov '12

19 Nov '12

On Nov 16, 2012, at 12:42 PM, greg wrote: > On Fri, Nov 16, 2012 at 12:28 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: >> If you follow the log, you should see messages from galaxy.jobs, and specifically, galaxy.jobs.runners.drmaa. >> > No, I don't see anything like that in the log after I started the > upload. Any ideas on how to see what might be going wrong there? Could you provide the contents of the log from the time you click the execute button on the tool form until it turns red in the UI? I just noticed this thread was on the user list, so I've moved it over to the dev list. --nate > > -Greg

2 4

Sam-to-Bam error
by Nicholas Tucker 19 Nov '12

19 Nov '12

Dear all, I have mapped some Illumina reads to a reference using Bowtie. I am trying to use Sam Tools to convert Sam to Bam on our local galaxy, but it always fails: Traceback (most recent call last): File "/export/galaxy/galaxy-central/lib/galaxy/jobs/runners/local.py", line 155, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/export/galaxy/galaxy-central/lib/galaxy/jobs/__init__.py", line 396, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/export/galaxy/galaxy-central/lib/galaxy/datatypes/binary.py", line 213, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found Tool execution generated the following error message: Unable to finish job The resulting failure always generates a large file, as if it has almost completed…… Anybody got an idea what the error might be and how to solve it please? Thanks in advance, Nick

2 1

Uncaught exception queueing job
by Carlos Borroto 19 Nov '12

19 Nov '12

Hi all, I ran into this error while uploading a ~300mb file. I'm using latest galaxy-dist, Apache proxy and Torque/Maui throught DRMAA. After this error appear in the log, the dataset stays in grey. If I restart the galaxy daemon, the job gets correctly submitted. galaxy.jobs.runners.drmaa ERROR 2012-11-16 16:29:49,833 Uncaught exception queueing job Traceback (most recent call last): File "/local/opt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 145, in run_next self.queue_job( obj ) File "/local/opt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 233, in queue_job job_id = self.ds.runJob(jt) File "build/bdist.linux-x86_64/egg/drmaa/__init__.py", line 331, in runJob _h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate) File "build/bdist.linux-x86_64/egg/drmaa/helpers.py", line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer)))) File "build/bdist.linux-x86_64/egg/drmaa/errors.py", line 90, in error_check raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value)) InternalException: code 1: pbs-drm: unknown error The same error happens even if I don't upload the file directly but use something like scp. Another file of only a few kbs doesn't show this behavior. I have the feeling this might be a memory issue, as the box running galaxy has only 2gb of RAM. Please let me know if there is something else I could do to help figuring out this issue. Thanks, Carlos

1 2

Multiple inputs in a workflow (variable number of inputs) ?
by LUO Yufei 17 Nov '12

17 Nov '12

Hello, The number of inputs is not fixed for my workflow, maybe I have 6 inputs, maybe more than that, the number of inputs depends on what data I have for the analysis. As I know, in a workflow, all inputs should be pre-defined...that is to say, the number of inputs is known before running one workflow. Do you have any idea for this problem? How can I do to make a workflow with the variable number of inputs? In advance, thanks for the answer, Yufei

3 3

Converting __oc__ and __cc__ back to curly braces?
by Samuel Lampa 16 Nov '12

16 Nov '12

Hi, I'm creating a galaxy tool [0] for querying external SPARQL (the Semantic Web query language) endpoints to get back data in tabular formats. I run into a problem though, since SPARQL queries contain curly braces, and Galaxy obviously converts these to an internal representation __oc__ and __cc__ (I found the commit for this at [1]) My SPARQL query for testing looks like this: SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 25 Thus, when I send this to my (python) script as a commmand line argument, the curly braces comes in galaxy's internal representation. Thus I was wondering if there is a way to get the braces in their correct format from galaxy, or if I should just go agead and convert them back in my own script? (i.e. will this behaviour be stable enough to rely on?) Best // Samuel [0] See the sparql_import.* files in https://github.com/samuell/galaxy_semwebtools [1] http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-February/001837.html -- Developer at SNIC-UPPMAX www.uppmax.uu.se Developer at Dept of Pharm Biosciences www.farmbio.uu.se Twitter - twitter.com/samuellampa Blog - saml.rilspace.org G+ - gplus.to/saml

2 3