July 2013 - galaxy-dev - lists.galaxyproject.org

Local Instance: BWA, Sam-to-Bam, file upload, filter Sam
by Moritz Juchler 24 Jul '13

24 Jul '13

Hello Ladies and Gentlemen, I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I have to setup Galaxy to find SNP's in genomes from hcc patients. I have a 64-bit openSuse 11.3 server on which I installed Galaxy locally, since we have a) very large files (>30GB per patient) and b) the data is protection sensitive. I have to run this pipeline: > http://www.nature.com/ng/journal/v44/n6/extref/ng.2256-S1.pdf 1. Which version of BWA should I use? The current gives me the EoF error and the 0.5.9. gives me "missing argument -I" 2. Originally I had this workflow to start with https://main.g2.bx.psu.edu/u/mj--/w/ngs , but I at the sam-to-bam conversion I get the "sequences are not currently available for specified build" error when using "locally cached" and I can't figure out how to use the reference file hg19.fa without actually uploading it to Galaxy, because I do not have enough space on the filesystem where the Galaxy distribution is placed ( /home). The genedata are all on /genedata. So my question here is: How to use the hg19.fa file placed on another filesystem then the galaxydist? 3. Thus I searched the web and found this workaround: https://main.g2.bx.psu.edu/u/mj--/w/sample-workflow-whole-exome-sequencing which runs fine ONLINE at usegalaxy but produces this error in my local instance Dataset generation errors *Dataset 18: Filter SAM on data 7* Tool execution generated the following error message: Traceback (most recent call last): File "/home/trr/galaxy-dist/tools/samtools/sam_bitwise_flag_filter.py", line 148, in <module> if __name__ == "__main__": main() File "/home/trr/galaxy-dist/tools/samtools/sam_bitwise_flag_filter.py", line 137, in main flags = int( fields[flag_col] ) IndexError: list index out of range So thats my 3rd question: How to solve this error? I didnt find anything online. Best Moritz

1 0

GATK fasta file missing ERROR
by Sridhar Srinivasan 24 Jul '13

24 Jul '13

Dear Developers, I am GATK (beta ) Unified Genotyper module to get the indels and SNP. I placed the jar file in (/galaxy-dist/tool-data/shared/jars/gatk) and i link the reference genome in .loc file (gatk_sorted_picard_index.loc). I could see the Name i mentioned in the .loc file is appearing in the galaxy page while choosing the Reference genome. Now while running GATK in galaxy , The command runs for a while and gives ERROR as i am getting ERROR as *(The fasta file you specified (/tmp/tmp-gatk-Jjg9Mi) does not exist.)* Could anybody suggest on the same.. Thanks Sridhar

1 0

Bowtie2 error: Creates a 62 kb file.
by Perez, Ricardo 24 Jul '13

24 Jul '13

Dear all, Today we ran some problems when running bowtie2. When we run bowtie2, the run manages to complete. However, bowtie2 creates a 62kb file. The error log that we get is the following: galaxy.jobs DEBUG 2013-07-23 17:09:42,378 (1483) Persisting job destination (destination id: local:///) galaxy.jobs.handler INFO 2013-07-23 17:09:42,392 (1483) Job dispatched galaxy.tools DEBUG 2013-07-23 17:09:42,556 Building dependency shell command for dependency 'bowtie2' galaxy.tools WARNING 2013-07-23 17:09:42,556 Failed to resolve dependency on 'bowtie2', ignoring galaxy.tools DEBUG 2013-07-23 17:09:42,556 Building dependency shell command for dependency 'samtools' galaxy.tools WARNING 2013-07-23 17:09:42,556 Failed to resolve dependency on 'samtools', ignoring galaxy.jobs.runners.local DEBUG 2013-07-23 17:09:42,626 (1483) executing: bowtie2 --version &> /usr/local/galaxy/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_1483; python /usr/local/galaxy/galaxy-dist/tools/sr_mapping/bowtie2_wrapper.py --num-threads="4" --output=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1673.dat --indexes-path="/data/db/fungidb/Ncrassa_OR74A/bowtie2/Ncrassa_wt_Chr01.Chr07.+ChrM.+rDNA_v12" --single-paired=paired --input1=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1668.dat --input2=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1669.dat -I 0 -X 250 --settings=preSet galaxy.jobs DEBUG 2013-07-23 17:09:42,696 (1483) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-07-23 17:09:42,766 execution finished: bowtie2 --version &> /usr/local/galaxy/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_1483; python /usr/local/galaxy/galaxy-dist/tools/sr_mapping/bowtie2_wrapper.py --num-threads="4" --output=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1673.dat --indexes-path="/data/db/fungidb/Ncrassa_OR74A/bowtie2/Ncrassa_wt_Chr01.Chr07.+ChrM.+rDNA_v12" --single-paired=paired --input1=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1668.dat --input2=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1669.dat -I 0 -X 250 --settings=preSet galaxy.jobs.runners DEBUG 2013-07-23 17:09:42,850 executing external set_meta script for job 1483: /usr/local/galaxy/galaxy-dist/set_metadata.sh ./database/files /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483 . /usr/local/galaxy/galaxy-dist/universe_wsgi.ini /usr/local/galaxy/galaxy-dist/database/tmp/tmpUiqpPD /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/galaxy.json /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_in_HistoryDatasetAssociation_1767_wqhEmr,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_kwds_HistoryDatasetAssociation_1767_qVZjpG,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_out_HistoryDatasetAssociation_1767_vIfpRO,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_results_HistoryDatasetAssociation_1767_UNl6OR,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_override_HistoryDatasetAssociation_1767_MZilrL 165.91.11.135 - - [23/Jul/2013:17:09:43 -0500] "GET /api/histories/ca8efa3c47500414 HTTP/1.1" 200 - "http://galaxy.tamu.edu/history" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.22 (KHTML, like Gecko) Ubuntu Chromium/25.0.1364.160 Chrome/25.0.1364.160 Safari/537.22" 165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /history HTTP/1.1" 200 - "http://galaxy.tamu.edu/tool_runner/index" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36" 165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /api/histories/1ab12f1e464820dd HTTP/1.1" 200 - "http://galaxy.tamu.edu/history" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36" 165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /history/get_display_application_links HTTP/1.1" 200 - "http://galaxy.tamu.edu/history" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36" galaxy.jobs.runners DEBUG 2013-07-23 17:09:44,457 execution of external set_meta for job 1483 finished galaxy.datatypes.metadata DEBUG 2013-07-23 17:09:44,545 loading metadata from file for: HistoryDatasetAssociation 1767 We currently have the most recent version of bowtie2 and samtools. Anyone know how to fix this? Thank you for your time, --Ricardo Perez

1 0

workflow: default parameters
by Keilwagen, Jens 24 Jul '13

24 Jul '13

Hi guys, when building workflows we can choose for each parameter whether to 'set at runtime' or to 'set in advance' (when creating or editing the workflow). We are looking for a possiblity to set a default value that is used (only) for the workflow, i.e., we like to specify a value that might be used by most users, but might be altered by expert users. This value might be different from the default value of the corresponding tool. So far, we did not find any possiblity. Thanks for any help. Best regards, Jens

2 2

Cloudman master cannot connect to volume at start-up
by Ravpreet Setia 23 Jul '13

23 Jul '13

* This is what has been reported by the cluster status log: * * 20:01:14 - Master starting * 20:01:16 - Trouble getting volume reference for volume vol-91cfd1cb: EC2ResponseError: 400 Bad RequestInvalidVolume.NotFoundThe volume 'vol-91cfd1cb' does not exist.9179663c-c5ec-4e90-88ff-ecf606b5e9a6 * 20:01:16 - Attempting to connect to a non-existent volume vol-91cfd1cb * 20:01:16 - Error processing filesystems in existing cluster configuration: 'NoneType' object has no attribute '__getitem__' * 20:01:16 - ***** Manager failed to start ***** Any suggestions?

2 3

RFC: Cheetah template in tool_dependencies.xml
by Björn Grüning 23 Jul '13

23 Jul '13

Hi, I was thinking how to tackle the build failures like the atlas library on OS-X or the ones from xlib and ease the writing of the shell_command a little bit. Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I was also sometimes forced to use some bash tricks. What about to make that more easy and powerful, and as a plus more consistent with the tool-command tag, and use cheetah for that kind of work. Something like that could work easily: #if ${os.environ}.getVar('X11_LIB_DIR', None): ./configure ... --x-includes=.... #else: ./configure #end if Also we could define some reserved variables like $__is64__ and cheetah evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah variable and we do not need to replace it later manually. What do you think any comments? If no one has any objections I can try to implement an <action type="cheetah_command">. Thanks, Björn

4 10

[GSoC2013] Week 5 Update
by Saket Choudhary 23 Jul '13

23 Jul '13

Was stuck up with my approach of wrapping the Workflows in a toolbox and hence this blog post is a bit delayed. http://galaxy-gsoc2013.blogspot.in/2013/07/week-5-update.html Saket

1 0

HTML output with external Javascript
by Sinthong, Phanwadee 23 Jul '13

23 Jul '13

Hi galaxy-dev, I have a tool that generates a HTML output file which uses some javascript library. I cannot view the output file with the eye button in Galaxy, but it still works fine from the HTML file opened out of the file directory. I changed the sanitize_all_html = False and restarted the galaxy server as some earlier post suggested but still couldn't view the output file. What else can I do to execute Javascript in the Galaxy browser? Thanks, Gift

2 2

Two questions
by Gromobir 23 Jul '13

23 Jul '13

Hello list, I have two questions regarding galaxy: 1.) Is it possible to configure the amount of lines listed in the tool standard output (stdout)? On my system only the last five lines of tool output are shown there. 2.) Currently I get the message "This dataset is large and only the first megabyte is shown below" every time I try to display a file, which is larger than one megabyte. Is it possible to change this size, such that I won't get this message anymore for larger files? Thank you very much in advance! Cheers, Gromobir

2 8

Re: [galaxy-dev] Problem in uploading file
by Himanshu Gupta 22 Jul '13

22 Jul '13

Hello, It seems you are right. It has nothing to do with the SQLAlchemy. I checked the Galaxy log and didn't find any other error log. After upgrading the database to MySQL when I uploaded the first input file, job dispatched to local runner and everything was fine. But when I tried to upload another file Galaxy didn't even dispatched the Job. I am pasting below the snapshot of the Galaxy log for your reference. galaxy.jobs DEBUG 2013-07-23 09:36:15,579 job 1 ended galaxy.datatypes.metadata DEBUG 2013-07-23 09:36:15,579 Cleaning up external metadata files 10.180.4.152 - - [23/Jul/2013:09:36:15 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:36:15 +0600] "GET /api/users/current HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:36:15 +0600] "GET /api/histories/f2db41e1fa331b3e/contents?ids=ebfb8f50c6abde6d HTTP/1.1" 200 - "http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:36:33 +0600] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:36:38 +0600] "POST /tool_runner/upload_async_create HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:36:38 +0600] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:01 +0600] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:08 +0600] "POST /tool_runner/upload_async_create HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:08 +0600] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:53 +0600] "GET /history HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:53 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:53 +0600] "GET /history/get_display_application_links HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET / HTTP/1.1" 200 - "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET /root/welcome HTTP/1.1" 302 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET /history HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET /history/get_display_application_links HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:02 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:06 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:10 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:14 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:18 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:22 +0600] "GET /api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" 10.180.4.152 - - [23/Jul/2013:09:38:24 +0600] "POST /datasets/33b43b4e7093c91f/delete_async HTTP/1.1" 200 - " http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4" Thanks On Mon, Jul 22, 2013 at 1:05 PM, James Taylor <james(a)jamestaylor.org> wrote: > The SQLAlchemy warning shouldn't have anything to do with uploads. > Does Galaxy still start correctly and do you see any other errors on > the log? What is logged when you try to do the upload? > > -- > James Taylor, Assistant Professor, Biology/CS, Emory University > > > On Fri, Jul 19, 2013 at 9:19 AM, Himanshu Gupta <himanshucse10(a)gmail.com> > wrote: > > Hello everyone, > > > > I had configured galaxy with MySQL database and for some time it was > working > > fine. But suddenly it started showing following warning whenever I start > > galaxy. > > > ".../galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs2.egg/sqlalchemy/sql/expression.py:1927: > > SAWarning: The IN-predicate on "request_type.id" was invoked with an > empty > > sequence. This results in a contradiction, which nonetheless can be > > expensive to evaluate. Consider alternative strategies for improved > > performance." > > > > And now I am not able to upload any file. Please help me to resolve it. > > > > Thanks > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ >

1 0