November 2012 - galaxy-dev - lists.galaxyproject.org

which .loc file for "SAM to BAM"?
by Andreas Kuntzagk 09 Nov '12

09 Nov '12

Hi, I'm still setting up a local galaxy. Currently I'm testing the setup of NGS tools. If I try "SAM to BAM" for a BAM file that has "hg18" set as build I get a message that "Sequences are not currently available for the specified build." I guess that I have either to manipulate one of the .loc files (but which?) or have to download additional data from rsync server. (I already have the tool-data/shared/hg18 completely) regards, Andreas Btw. Do you have any plans to ease the pain on adding additional builds? Something simpler than having to add one line for each build*tool combo? These lines seem very redundant to me. -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

3 10

GATK fails to create Java Virtual Machine
by Iry Witham 09 Nov '12

09 Nov '12

Hi Folks, I received an email from one of my end users that she was having an issue with the "Realigner Target Creator" under NGS: GATK Tools. Here's the error: An error occurred running this job: "Unable to access jar file /hpcdata/galaxy-setup/galaxy-dist/tool-data/shared/jars/gatk/GenomeAnalysisTK.jar." That was easy to resolve since the jar file is located at either /hpcdata/galaxy-setup/galaxy-dist/tool-data/shared/jars/GATK/1.4.2/ or /hpcdata/galaxy-setup/galaxy-dist/tool-data/shared/jars/GATK/1.6.13/. I modified the xml to point to the /1.4.2 directory and now I get "Could not create the Java virtual machine." I have looked at the wrapper for java settings and restarted the Galaxy server, but am at a loss. Do you have any ideas? Thanks, Iry Witham Scientific Applications Administrator Scientific Computing Group Computational Sciences Dept. The Jackson Laboratory 600 Main Street Bar Harbor, ME 04609 Phone: 207-288-6744 email: iry.witham(a)jax.org The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

1 0

makeblastdb wrapper
by Espin, Almudena 09 Nov '12

09 Nov '12

Hi guys, I am writing a wrapper for makeblastdb. The problem is that Galaxy doesn’t support the blastdb data format, so this has to be added if a makeblastdb wrapper is to work properly. I have problems modifying the registry.py, Could anyone specify which changes have to be done? Thank you

2 3

How to get a compressed output in the history for my own tool
by Philipe Moncuquet 09 Nov '12

09 Nov '12

Hi dev, I have written a perl script to perform some complex demultiplexing of reads. It works well on my computer but the integration to Galaxy is a bit tricky since it's the first time for me. At one point my perl script generates a bunch of fasta files, I want all this fasta files to be compressed and I use a tar command within the script to do so. I am not sure how to do the same within the Galaxy execution. I think it might work out with the "from_work_dir" attributes of the output tag but I don't know exactly where my script should write these fasta files and compress them so that the archive could be retrieve. I don't know either what to put in the "format" attributes since gz is not in the supported datatype or is it ? Should I create a new datatype ? The ending point of my tool would be to have the user be able to download one compressed file that would contains all the sorted fasta files. Cheers, Philippe

1 0

Testing framework and repeat, boolean and conditional parameters
by Simon Gladman 09 Nov '12

09 Nov '12

Hi all, Have looked through the list archives regarding this without much luck. I have a repeat parameter which contains a boolean based conditional. I wrote a simple test case for it but I think the test keeps failing at the boolean parameter.. input section looks like: <repeat name="inputs" title="Input read libraries"> <conditional name="is_separate"> <param name="separate" type="boolean" checked="false" truevalue="-separate" falsevalue="" label="Are the reads paired and in two different files?" /> <when value=""> <param name="forwards_file" type="data" format="fasta,fastq,raw,sam,bam" label="Read dataset"/> </when> <when value="-separate"> <param name="forwards_file" type="data" format="fasta,fastq,raw,sam,bam" label="Read dataset for direction 1"/> <param name="reverse_file" type="data" format="fasta,fastq,raw,sam,bam" label="Read dataset for direction 2"/> </when> </conditional> ....... </repeat> test section looks like: <tests> <test> <param name="start_kmer" value="21"/> <param name="end_kmer" value="29"/> <param name="inputs.is_separate.separate" value="false"/> <param name="inputs.is_separate.forwards_file" value="vopt_test_reads.dat" ftype="fasta"/> <output name="contigs" file="vopt_test_contigs.dat"/> </test> </tests> The output from run_functional tests is: ====================================================================== ERROR: test_tool_000000 (functional.test_toolbox.TestForTool_velvetoptimiser_vlsci) Velvet Optimiser vlsci ( velvetoptimiser_vlsci ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/simon/Galaxy/galaxy-dist/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/home/simon/Galaxy/galaxy-dist/test/functional/test_toolbox.py", line 76, in do_it page_inputs = self.__expand_grouping(testdef.tool.inputs_by_page[0], all_inputs) File "/home/simon/Galaxy/galaxy-dist/test/functional/test_toolbox.py", line 131, in __expand_grouping expanded_inputs.update( self.__expand_grouping( { new_prefix : r_value }, declared_inputs, prefix = new_prefix ) ) File "/home/simon/Galaxy/galaxy-dist/test/functional/test_toolbox.py", line 118, in __expand_grouping if declared_inputs[ value.test_param.name ] == case.value: KeyError: 'separate' -------------------- >> begin captured stdout << --------------------- kmer_step not declared in tool test, will not change default value. filetype not declared in tool test, will not change default value. --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- galaxy.web.framework: DEBUG: Error: this request returned None from get_history(): http://localhost:8326/ galaxy.web.framework: DEBUG: Error: this request returned None from get_history(): http://localhost:8326/ galaxy.web.framework: DEBUG: Error: this request returned None from get_history(): http://localhost:8326/user/logout galaxy.web.framework: DEBUG: Error: this request returned None from get_history(): http://localhost:8326/ --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 0.675s FAILED (errors=1) My question: Can the testing framework handle the nesting of the boolean, conditional and repeat parameters or have I made a mistake in the test xml? Cheers, Simon Gladman VBC/VLSCI Monash University Australia.

1 0

Velvet not working
by Philipe Moncuquet 09 Nov '12

09 Nov '12

Hi developers, We have our own instance running and under continuoius development. I installed velvetg and velveth from the toolshed (revision 0:4afe13ac23b6<http://toolshed.g2.bx.psu.edu/repository/view_changelog?id=9dca5b8f68c509af>) and manually installed binaries for velvet_1.2.08. When I try to test it under the galaxy web interface, the job does not failed but all files are empty except the first one that contains the following error message : [0.000000] Unknown option: -ins_length3 I guess this might come from the version of the binaries and the version the xml wrapper was written for are different. Does anyone knows which binaries version I should use ? How is it that the job does not appears as failed ? Any help will greatly appreciated. Regards, Philippe

3 3

Always seem to see this " [Errno 39] Directory not empty" error when a step fails
by Anthonius deBoer 09 Nov '12

09 Nov '12

Hi, I always see this error pop up in the logs when one of the steps failed... It seems to have a problem deleting the working directory since it claims they are not empty. Ofcourse when I check, the directory is empty, but I am left with a gazilion empty working directories, alhtough no problems with the installation otherwise... Using the latest pull Any ideas? Thanks, Thon galaxy.datatypes.metadata DEBUG 2012-11-08 15:43:29,532 Cleaning up external metadata files galaxy.objectstore CRITICAL 2012-11-08 15:43:29,621 /mnt/ngs/analysis/svcgalaxy/DATA/job_working_directory/000/105 delet e error [Errno 39] Directory not empty: '/mnt/ngs/analysis/svcgalaxy/DATA/job_working_directory/000/105' ls -la /mnt/ngs/analysis/svcgalaxy/DATA/job_working_directory/000/105 total 36K drwxrwsr-x 2 svcgalaxy 0 Nov 8 15:42 . drwxrwsr-x 30 svcgalaxy 565 Nov 8 15:42 .. (galaxy_env)[svcgalaxy@srv151 g]$

1 0

Problems with latest Galaxy 8167:b0c916349070
by Scooter Willis 08 Nov '12

08 Nov '12

Running a CloudMan created instance on Amazon. Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070 Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls ABYSS abyss-adjtodot abyss-joindist abyss-tofastq DistanceEst MergePaths PathConsensus SimpleGraph abyss2ace abyss-fac ABYSS-P AdjList KAligner Overlap PathOverlap abyss2afg abyss-fixmate abyss-pe Consensus MergeContigs ParseAligns PopBubbles Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented. Thanks Scooter Bowtie Stdout Sequence file aligned. ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt' Stderr WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Using toolshed installed Abyss and get the same error in Stdout. Stdout ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt' ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt' Stderr COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrapper.sh: line 9: ABYSS: command not found WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Looking at the util directory based on the expected path of the read build file galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls aliaspickler.py docutils_ext heartbeat.py json.pyc odict.py streamball.py aliaspickler.pyc docutils_template.txt inflection.py lrucache.py odict.pyc streamball.pyc backports expressions.py inflection.pyc lrucache.pyc sanitize_html.py template.py bunch.py expressions.pyc __init__.py memdump.py sanitize_html.pyc template.pyc bunch.pyc hash_util.py __init__.pyc none_like.py shed_util.py topsort.py debugging.py hash_util.pyc json.py none_like.pyc shed_util.pyc topsort.pyc

2 12

BWA error post upgrade - local galaxy
by Kelkar, Hemant 08 Nov '12

08 Nov '12

We recently upgraded our local galaxy install and followed the procedure that now uses "bwa" from the galaxy toolshed. Since the upgrade "bwa" alignments are generating the following error (exact file paths redacted): [bwt_restore_bwt] fail to open file '/path_to/bwa_path/mm9_all.fa.rbwt'. Abort! /bin/sh: line 1: 14044 Aborted (core dumped) bwa aln -t 4 /path_to/bwa_path/mm9_all.fa /path_to/galaxy/002/dataset_2377.dat > /path_to/galaxy-prod/galaxy-dist/database/tmp/tmpxb3Bvr/tmpJeewN0 We are using indexes made by bwa (0.6.0 also tried 0.6.2) but I do not see the "*.rbwt" file mentioned above in the indexes we have. BWA was working prior to the upgrade with the indexes currently in place. We are using "LSF" for job queuing on the cluster. All other tools appear to be working fine. Thanks, Hemant

2 1

Manage libraries permissions with API
by Fabien Mareuil 08 Nov '12

08 Nov '12

Dear all, I would like to know if it is possible to manage libraries permissions with the API galaxy, and how to do that. Thank you for your answer, Best regards, Fabien Mareuil -- Fabien Mareuil | Centre d'Informatique pour la Biologie fabien.mareuil(a)pasteur.fr | Institut Pasteur 25,28 rue du Docteur Roux 75015 Paris, France

1 0