- galaxy-dev - lists.galaxyproject.org

problems with the varscan somatic tool
by Karin Schlangen 11 Sep '14

11 Sep '14

Dear Galaxy team, i tried to run the varscan somatic tool from fcaramia on two pileup files but i got the following error: Unknown Input: COMMAND::java -jar /home/karin/galaxy/dependency_dir/VarScan/2.3.5/fcaramia/varscan/51969e284317/jars/VarScan.v2.3.5.jar somatic The command was the following: Job Command-Line: perl /home/karin/shed_tools/ toolshed.g2.bx.psu.edu/repos/fcaramia/varscan/51969e284317/varscan/varscan_… "COMMAND::java -jar $JAVA_JAR_PATH/VarScan.v2.3.5.jar somatic" "NORMAL::/home/karin/galaxy-dist/database/files/000/dataset_107.dat" "TUMOR::/home/karin/galaxy-dist/database/files/000/dataset_109.dat" "OUTPUT::/home/karin/galaxy-dist/database/files/000/dataset_145.dat" "OPTION::--min-coverage 8" "OPTION::--min-coverage-normal 8" "OPTION::--min-coverage-tumor 8" "OPTION::--min-var-freq 0.1" "OPTION::--min-freq-for-hom 0.75" "OPTION::--normal-purity 1.0" "OPTION::--tumor-purity 1.0" "OPTION::--p-value 0.99" "OPTION::--somatic-p-value 0.05" "OPTION::--strand-filter 1" "OPTION::--validation 0" "OPTION::--output-vcf 1" I also tried to change the varscan_somatic.xml file in such a way that i removed the "COMMAND::" tags etc. and used it without the varscan_somatic.pl file. After doing that, the tool ran without errors but i obtained empty files. Redoing the same command on my local computer ran properly and i obtained the wanted outputs: java -jar $JAVA_JAR_PATH/VarScan.v2.3.5.jar somatic /home/karin/galaxy-dist/database/files/000/dataset_107.dat /home/karin/galaxy-dist/database/files/000/dataset_109.dat dataset_131.dat --output-snp dataset_131.dat --output-indel dataset_132.dat --min-coverage 8 --min-coverage-normal 8 --min-coverage-tumor 8 --min-var-freq 0.1 --min-freq-for-hom 0.75 --normal-purity 1.0 --tumor-purity 1.0 --p-value 0.99 --somatic-p-value 0.05 --strand-filter 1 2> testlog in dataset_131.dat the snps, indels: dataset_132.dat, and some information (varscan directs this info on stderr) in testlog. Any idea how to fix that? i appreciate any information on how this may be fixed! Thanks a lot and best wishes, Karin

1 0

BAM to SAM tool no way to get an unsorted SAM?
by Alistair Chilcott 10 Sep '14

10 Sep '14

Hello all, My users are trying to use a tool called bismark and it requires an unsorted SAM file for one of its steps Previous steps produce BAM files so we would like to include the tool BAM-to-SAM as a step in the workflow unfortunately it seems to be set to sort by default and there is no way to alter this behaviour. Is there another tool that would convert a BAM to a SAM without sorting it or can the BAM-to-SAM tool be altered to include a checkbox for an unsorted result? Regards, Alistair

2 4

How to escape % characters in tool parameters?
by Melissa Cline 09 Sep '14

09 Sep '14

Hi folks, I'm working on a tool that queries a database based on user input, where this user input is a text parameter. The wildcard for the database is the percent (%) character. The user input, which is ultimately translated into the query, should be able to contain a % character. For example, one input field lets the user indicate which cancer cohort he or she is interested in, in a table of available cohorts, and the user should be able to give input like 'TCGA%" to indicate all TCGA cohorts. The problem I'm having is that when % is given in the input field, it's translated (to X) by the time it reaches my tool's python script. I haven't been able to figure out the right way to escape the input so that % is not translated. So, how do I escape this? Thanks! Melissa

2 2

Error executing workflow via API
by Neil.Burdett＠csiro.au 09 Sep '14

09 Sep '14

Hi, I execute a number of workflows via the API which all work fine. However, the longest one I use (consists of three linked tasks), returns an error. I call the workflow in the python script using popen and has the format: /home/galaxy/milxcloud-new/scripts/api/workflow_execute.py d2fcd3feb4c6318c496d55fa8869b67c http://barium-rbh/new/api/workflows f597429621d6eb2b hist_id=f597429621d6eb2b 4=hda=c8c00aa41dc69085 The task appears on the history panel and all tasks eventually complete in the workflow, however, the task that calls the workflow using the above popen call returns the following error: Traceback (most recent call last): File "/home/galaxy/milxcloud-new/scripts/api/workflow_execute.py", line 31, in <module> main() File "/home/galaxy/milxcloud-new/scripts/api/workflow_execute.py", line 28, in main submit( sys.argv[1], sys.argv[2], data ) File "/home/galaxy/milxcloud-new/scripts/api/common.py", line 117, in submit r = post( api_key, url, data ) File "/home/galaxy/milxcloud-new/scripts/api/common.py", line 51, in post return json.loads( urllib2.urlopen( req ).read() ) File "/usr/lib/python2.7/socket.py", line 351, in read data = self._sock.recv(rbufsize) File "/usr/lib/python2.7/httplib.py", line 541, in read return self._read_chunked(amt) File "/usr/lib/python2.7/httplib.py", line 586, in _read_chunked raise IncompleteRead(''.join(value)) httplib.IncompleteRead: IncompleteRead(147 bytes read) This worked on older versions of Galaxy. I need this to work so I can "monitor" the workflow so know when all tasks are complete. has anyone seen this error? Or got an idea how to solve it. hg summary reports: % hg summary parent: 13771:7a4d321c0e38 tip Updated tag latest_2014.06.02 for changeset 8c30e91bc9ae branch: stable commit: (clean) update: (current) Thanks Neil

2 3

Output Excel file
by Patrick Leyshock 08 Sep '14

08 Sep '14

Hello, I am trying to write a Galaxy tool that will output an Excel file. Currently the tool wrapper calls an R script, which uses the "xlsx" package to read and write to and from Excel files. After being invoked by Galaxy, the script is able to successfully read an input Excel file: suppressPackageStartupMessages(library(xlsx, quietly=TRUE)); raw.data <- read.xlsx(commandArgs(trailingOnly=TRUE)[1]), sheetName="input_records"); The script then does its work on the data just read in. Then, when the work is done, I'd like to output the results to an Excel file. Here's where I run into a problem. I first tried to output the result like this: write.xlsx(processed.data, file=commandArgs(trailingOnly=TRUE)[2]); but write.xlsx throws an error. Looks like ".xls" and ".xlsx" are the only legal file extensions for the write.xlsx() function. Inspecting commandArgs(trailingOnly=TRUE)[2] shows that Galaxy provided a filename extension of ".dat". I tried a workaround using this: write.xlsx(processed.data, file=paste(commandArgs(trailingOnly=TRUE)[2]), ".xls", sep=""); The write.xlsx function no longer throws an error (since the file name supplied as a parameter has an acceptable file extension) but now Galaxy won't display the result in the History. If I look in Galaxy's database I see two files there corresponding to my tool's output. Supposing that Galaxy assigned the result the name "dataset_87", then looking in the Galaxy database I see: dataset_87.dat dataset_87.dat.xls "dataset_87.dat" is empty but visible to Galaxy (and so displayed as an empty dataset in the History window). "dataset_87.dat.xls" has the results I want but isn't displayed by Galaxy in the History window. There might be R libraries that can write xls or xlsx files without requiring a ".xls" or ".xlsx" file extension. That's a path I'm willing to explore. That said, is there a configuration option I can set that'll let me continue to use write.xlsx()? I've been working from the suggestions made on this helpful thread: http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-December/007807.html so the relevant entries in datatypes_conf.xml are: <datatype extension="xls" type="galaxy.datatypes.binary.Xls" display_in_upload="true" /> <datatype extension="xlsx" type="galaxy.datatypes.binary.Xlsx" display_in_upload="true" /> and the relevant additions to binary.py are: class Xls(Binary): '''Class describing an Excel 2003 (xls) file''' file_ext='xls' class Xlsx(Binary): '''Class describing an Excel 2007 (xlsx) file''' file_ext='xlsx' Any suggestions appreciated. Thanks, Patrick

3 3

Re: [galaxy-dev] Concept for a Galaxy Versioned Fasta Data Retrieval Tool
by Dooley, Damion 05 Sep '14

05 Sep '14

Well well, thanks very much for that reference! I can see how your system to enable a workflow to process delta (diff) data (and merge the results back with a previous run's output) would greatly lighten the processing power for keeping results current. Interesting choice of technologies too. Damion Message: 5 Date: Fri, 5 Sep 2014 10:41:37 +0000 From: Pedersen Edvard <edvard.pedersen(a)uit.no> To: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Subject: Re: [galaxy-dev] Concept for a Galaxy Versioned Fasta Data Retrieval Tool My PhD work may be of interest for this subject, although the primary focus has been on generating databases comprising the changes from a specific timeframe, and was not designed specifically for Galaxy. The similarities between my system and the system you are proposing are that it can generate a BLAST database from any date (that has been added to the system), as well as "diffs" between two dates, and supports FASTA, the Uniprot EMBL variant, full files (which does not give compression benefits) and several others. The system uses delta compression to make sure that non-updated fields do not take up extra space. It uses the Hadoop stack (HBase, HDFS and MapReduce) for parallelism in generating the databases (the blast database generation from FASTA files is not parallel). You can find one of the publications here: <http://bdps.cs.uit.no/papers/hibb13.pdf> http://bdps.cs.uit.no/papers/hibb13.pdf I hope this can be of some use to you. Regards, Edvard Pedersen

1 0

Join tool failure
by Iry Witham 05 Sep '14

05 Sep '14

Hi Team, I have upgraded to the latest galaxy distribution on my test server. Now I am running into issues and can't figure this one out. I am attempting to run the join.py tool to join two datasets at specified fields. I am getting the following error: Traceback (most recent call last): File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/tools/filters/join.py", line 18, in <module> from galaxy.util.bunch import Bunch File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/__init__.py", line 95, in <module> import galaxy.eggs File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/eggs/__init__.py", line 5, in <module> import os, sys, shutil, glob, urllib, urllib2, ConfigParser, HTMLParser, zipimport, zipfile File "/usr/lib64/python2.6/urllib2.py", line 93, in <module> import hashlib File "/usr/lib64/python2.6/hashlib.py", line 88, in <module> import _hashlib ImportError: /usr/lib64/libssl.so.10: symbol EC_KEY_get0_group, version OPENSSL_1.0.1_EC not defined in file libcrypto.so.10 with link time reference Traceback (most recent call last): File "./scripts/set_metadata.py", line 29, in <module> from galaxy import eggs File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/__init__.py", line 95, in <module> import galaxy.eggs File "/hpcdata/galaxy-test/galaxy-setup/galaxy-dist/lib/galaxy/eggs/__init__.py", line 5, in <module> import os, sys, shutil, glob, urllib, urllib2, ConfigParser, HTMLParser, zipimport, zipfile File "/usr/lib64/python2.6/urllib2.py", line 93, in <module> import hashlib File "/usr/lib64/python2.6/hashlib.py", line 88, in <module> import _hashlib ImportError: /usr/lib64/libssl.so.10: symbol EC_KEY_get0_group, version OPENSSL_1.0.1_EC not defined in file libcrypto.so.10 with link time reference I am at a loss on the resolution for this. Thanks, Iry The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

2 1

Re: [galaxy-dev] Concept for a Galaxy Versioned Fasta Data Retrieval Tool
by Pedersen Edvard 05 Sep '14

05 Sep '14

My PhD work may be of interest for this subject, although the primary focus has been on generating databases comprising the changes from a specific timeframe, and was not designed specifically for Galaxy. The similarities between my system and the system you are proposing are that it can generate a BLAST database from any date (that has been added to the system), as well as "diffs" between two dates, and supports FASTA, the Uniprot EMBL variant, full files (which does not give compression benefits) and several others. The system uses delta compression to make sure that non-updated fields do not take up extra space. It uses the Hadoop stack (HBase, HDFS and MapReduce) for parallelism in generating the databases (the blast database generation from FASTA files is not parallel). You can find one of the publications here: <http://bdps.cs.uit.no/papers/hibb13.pdf> http://bdps.cs.uit.no/papers/hibb13.pdf I hope this can be of some use to you. Regards, Edvard Pedersen

1 0

Re: [galaxy-dev] Concept for a Galaxy Versioned Fasta Data Retrieval Tool
by Dooley, Damion 04 Sep '14

04 Sep '14

Earlier on in the project analysis I was pursuing a Git solution because it seemed all its features would work with documents/code/files of any kind and so would be perfect for scientific reproducibility. But its ability to efficiently archive non-documents is quite hit and miss, and the file size limitation becomes a major problem on top of that when it doesn't. I will try to design the system so that handlers for different types of databases/files can be called into play to retrieve versioned content. Its just that this fall I'll only have time to provide the handlers for fasta file archiving (the key-value database update approach enables fasta versioning and all the spinoff data from that.). The next priority would be a handler for any type of file that needs to be replaced as a whole from version to version (one just needs hard drive space to accommodate this, since caching is pointless). A git handler for well-behaved document content would also be a possibility. Typo: I said yesterday "I wasn't going to leave that as just "fasta" datatype since it seems tools like makeblastdb don't allow anything else ..." - but I meant 'I WAS going to leave that as just "fasta"...' d.

1 0

Examples of Galaxy tools in the toolsheds that install and run JAR files properly?
by Melissa Cline 04 Sep '14

04 Sep '14

Hi folks, I'm attempting something that should be straightforward, but it's not. I have a tool that runs a JAR file, which I have bundled with the tool. I simply want to run the JAR file. And to paraphrase Thomas Edison, I've tried several thousand things that do not work (at least for me), from setting the JAVA_JAR_PATH environment variable in the tool_dependencies.xml file to trying to copy the JAR file into the tool-data/shared/jars subdirectories (which is the closest thing I've got to working). So, at long last I'm doing the sensible thing and looking for one simple working example that I can use as a template. Who can suggest a good toolshed tool (either main or test) that involves running its own JAR file, and that works? Thanks! Melissa

5 6