- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] trinity on galaxy
by Jeremy Goecks 12 Oct '11

12 Oct '11

David, I'm cc'ing the galaxy-dev mailing list; let's keep this thread on the list for archival and community purposes. > >> I had a look through the Trinity.pl script and found that the default butterfly >> heap is set to 1G. >> >> I've changed this to 2G. Would you re-run the trinity job and let me know if this helps. Rather than mucking with the script, It's possible to change the Butterfly memory size using command line parameters: --- Butterfly options: # # --run_butterfly :executes butterfly commands. Do not set this if you want to spawn them on a computing grid. # # --bfly_opts :parameters to pass through to butterfly (see butterfly documentation). # # --bflyHeapSpace :java heap space setting for butterfly (default: 1000M) => yields command java -Xmx1000M -jar Butterfly.jar ... $bfly_opts --- This can be done in the Trinity wrapper at <galaxy-home>/tools/ngs_rna/trinity_all.xml ; there are already some Butterfly options set in the wrapper, so it should be easy to add more. > We see the following at the end of the Trinity logs - I don't know what it means but I thought you might... > number of good components: 1 > number of reads found = 1253 (from total of 1254) which came from 1252 pairs > ## Read PathPair results: 1251 singletons, num pairs: 1, num pairs discarded: 0 > total number of components = 1 > working on subcomponent 0 > method: combineSimilarPathsThatEndAtV(-1) with 1 paths > method: combineSimilarPathsThatEndAtV(2) with 1 paths > method: combineSimilarPathsThatEndAtV(1248) with 1 paths > method: combineSimilarPathsThatEndAtV(1223) with 1 paths > method: combineSimilarPathsThatEndAtV(-2) with 2 paths > > Path Illustration: > > ==== PATH: [-1, 2, 1248, -2] > === Read: [[-1, 2, 1248], []] count: 2 > = Read: [[2], []] count: 1030 > == Read: [[2, 1248], []] count: 23 > === Read: [[2, 1248, -2], []] count: 2 > = Read: [[1248], []] count: 197 > > > Final path reported: comp12043_c0_seq1 len=1658 ~FPKM=29.7 path=[2:0-1220 1248:1221-1657] > FinalPath: [-1, 2, 1248, -2] with support <1254, 1254> > total number of paths reported = 1 from 1 components > Done > SUCCESS[53682]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.6/comp12043 -V 10 --stderr > SUCCESS[53680]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.24/comp48425 -V 10 --stderr > SUCCESS[53681]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.8/comp16349 -V 10 --stderr > > > Sorry, 1 of 53682 jobs failed. Overall, this looks very good - almost all the jobs succeeded. > Failed commands written to file: failed_cmds.27222.txt > > Error, cmd: /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/util/cmd_process_forker.pl -c /gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/database/job_working_directory/679/trinity_out_dir/chrysalis/butterfly_commands.adj --CPU 4 --shuffle died with ret 256 at /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Trinity.pl line 571. This is a bit cryptic to me. You'll want to determine what error code 256 means and see whether it may be a memory issue; you may also want to email the Trinity mailing list and add for help there. Good luck, J.

2 1

cloudman on private cloud
by Rodrigo García Herrera 12 Oct '11

12 Oct '11

Hello! If I setup a private cloud infrastructure with ubuntu, can I setup galaxy cloudman there? Thanks! Rodrigo

2 1

Possible to browse local filesystem
by alessandro albiero 12 Oct '11

12 Oct '11

Dear all, I have a problem with galaxy and I have already tried to write on ther user list.... My problem concern the upload on galaxy of new files. We are setting up a galaxy istance on our server (HPC Cluster with torque and PBS) for internal use. Our problem is that we manage only NGS data (SOLiD and Illumina Data) that are already stored on the server. We tried to set up the user directory in the universe_wgsi.ini file to allow to any user tu upload their local files, but adding new file and re-uploading the dir we found duplicated files (it seems that galaxy re-upload every time all the files ignoring if some are already present). This method is OK for us but how can we avoid the file duplications ? In a presentation found on web (blankenberg_pittsburg_2001), we saw that there is the possibility to automatically upload the sequenced reads on Galaxy (and let them available to all our user). Where can we find information for doing this ? Thank in advance All the best Alessandro -- ****************************** Alessandro Albiero Bioinformatics Lab CRIBI-Padua University Viale G. Colombo, 3 35121 Padova-Italy http://genomics.cribi.unipd.it http://genomics.cribi.unipd.it/index.php/User:Albiero Phone. +39. 049 827-6165 Fax. +39. 049 827-6159 ********************************

2 1

Downloading large files from local galaxy
by Roy Weckiewicz 09 Oct '11

09 Oct '11

Hi, We have a local galaxy instance running on mac osx connected to a mysql database. We are having an issue downloading files which are > 2Gb. I've read posts on this mailing list that mention that there is a browser limit on downloading files of this size, which is fine -- a command-line method would be okay. Furthermore, I've read some solutions that mention using either wget or curl along with the link from the 'disk icon' in the history to download the data. We've tried both curl and wget, which seem to work fine with files < 2Gb, but again fails with the larger files. Wget will tend to time-out, and attempts to retry over and over (wget 'address-from-galaxy-history') and curl will hang forever as well (curl -O 'address-from-galaxy-history'). Attempts have been made to both download the data from the host computer to itself, and from the host to a remote computer. Are we perhaps missing some critical configuration setting? We would like to stay away from a ftp solution if at all possible. Is anyone using a simple method to retrieve these large files? -Roy W. -- Roy Weckiewicz Texas A&M University

3 3

Accept-Ranges: bytes for
by Kip Bodi 08 Oct '11

08 Oct '11

Hi all, I have a local galaxy instance, proxied by Apache, and am trying to get the 'display at UCSC' link to work. I've followed all of the instructions and am nearly there, but now I'm getting this response from UCSC when it tries to load the track: "Byte-range request was ignored by server. Expected Partial Content 206." etc. I think the error is because the PasteWSGIServer isn't allowing byte-range requests. When I look at the header, "Accept-Ranges: bytes" is not present: [galaxy@galaxy galaxy_dist]$ curl --head localhost HTTP/1.1 200 OK Date: Fri, 07 Oct 2011 22:23:49 GMT Server: PasteWSGIServer/0.5 Python/2.6.6 content-type: text/html; charset=UTF-8 Set-Cookie: galaxysession=c6ca0ddb55be603a4e98b4f13a4db8bfb8635f0aa576697114d40b13a8355efe2cf7a075ef81d769; expires=Thu, 05-Jan-2012 17:23:49 GMT; Max-Age=7776000; Path=/; Version=1 Vary: Accept-Encoding Connection: close Whereas on another server, "Accept-Ranges: bytes" appears in the header: [galaxy@galaxy galaxy_dist]$ curl --head apollo.med.tufts.edu HTTP/1.1 403 Forbidden Date: Fri, 07 Oct 2011 22:22:42 GMT Server: Apache/2.2.21 (Fedora) *Accept-Ranges: bytes* Content-Length: 4609 Connection: close Content-Type: text/html; charset=UTF-8 I'm running RHEL6 with the latest version of galaxy-dist. I've tried manually enabling 'accept-ranges' but haven't had any luck. Any suggestions? Can I enable it somehow in universe_wsgi.ini? Best, -Kip

1 0

Datafiles for the Gencode partition tool
by Harald Dahle 08 Oct '11

08 Oct '11

Hello, I've been trying to configure the Gencode Partition tool on our galaxy installation, but have not been able to find any information about what sort of data it expects to find, and my best attempts at guessing still lead to errors being produced. Can anyone help me with a few pointers to the correct datafiles/format of the files? Thanks in advance! Best regards, Harald Dahle

1 0

CloudMan on Eucalyptus
by Andrey Tovchigrechko 07 Oct '11

07 Oct '11

We want to use CloudMan to provision and control a cluster with our own VM image (with Galaxy front-end as well). Our plan was to make the VMs for both our internal Eucalyptus cloud as well as EC2. Will CloudMan work with Eucalyptus? If not, how much would our effort have be in order to extend the source code to make it work? I am seeing in the repo an empty file https://bitbucket.org/galaxy/cloudman/src/1440e0147c85/cm/clouds/eucalyptus… Thanks, Andrey

2 1

bug report submission bug
by Oleksandr Moskalenko 07 Oct '11

07 Oct '11

Someone shared a history with me for the purpose of debugging their Galaxy problem in our local Galaxy instance. I set the "error_email_to" to our bug reporting email address, but I can't submit a bug report using the "bug" icon in the history for datasets in an error state. The traceback is below. The job failure was a missing tool by the way. Has anyone run into this before? 10.244.xxx.xxx - - [03/Oct/2011:16:02:12 -0400] "POST /dataset/report_error HTTP/1.1" 500 - "http://galaxy.hpc.ufl.edu/datase t/errors?id=767" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/534.48.3 (KHTML, like Gecko) Version/5.1 Safari /534.48.3" Error - <type 'exceptions.IndexError'>: list index out of range URL: http://galaxy.hpc.ufl.edu/dataset/report_error File '/galaxy/prod/galaxy/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/galaxy/prod/galaxy/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/galaxy/prod/galaxy/lib/galaxy/web/framework/middleware/remoteuser.py', line 111 in __call__ return self.app( environ, start_response ) File '/galaxy/prod/galaxy/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/galaxy/prod/galaxy/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/galaxy/prod/galaxy/lib/galaxy/web/controllers/dataset.py', line 171 in report_error job = hda.creating_job_associations[0].job IndexError: list index out of range

1 1

help with complicated conditional output format
by Sarah Diehl 06 Oct '11

06 Oct '11

Hello, I'm trying to write a wrapper for the intersectBed tool from the BEDTools suite. The difficulty lies in the possible combinations of input and output data formats. Input can be either bed or bam. When the input is bed, the output is also bed. However, when the input is bam the output can be either bam (default) or bed (by an additional command-line switch). I cannot figure out how to dynamically set the output format correctly. The wrapper always fails in at least one of the three cases. I collect the input as follows: <inputs> <conditional name="intype"> <param name="inselect" type="select" label="Select input file type"> <option value="bed" selected="true">BED, GFF</option> <option value="bam">BAM</option> </param> <when value="bed"> <param name="inputBed" type="data" format="bed,gff" label="Input A" help="Each feature in A is compared to B in search of overlaps."/> </when> <when value="bam"> <param name="inputBam" type="data" format="bam" label="Input A" help="Each BAM alignment in A is compared to B in search of overlaps."/> <param name="bed" type="boolean" truevalue="-bed" falsevalue="" checked="False" label="Write output as BED" help="The default is to write output in BAM when using BAM as input."/> </when> </conditional> <param name="input" type="data" format="bed" label="Input B"/> </inputs> I really want to keep the conditional and the select, because the program behaves slightly different depending on the input format. I tried to define the output like that: <outputs> <data name="output" format="input"/> <change_format> <when input="intype.bed" value="" format="bam"/> </change_format> </outputs> However, this fails when the input is bam and I want a bed output. I tried many different things I found in the wiki and on the mailing list, but nothing worked properly. I'm grateful for any ideas how to solve this problem. Thanks for your help in advance! Sarah

1 1

Galaxy tools with redirected python logging output get stuck past a certain amount of logging output
by Tim te Beek 06 Oct '11

06 Oct '11

Hi all, I've got some Python applications that I've hooked into Galaxy, which use the Python logging module for output. To capture this output in my Galaxy history I've created a wrapper that adds a logging.FileHandler to the root logger, which uses an Galaxy tool output argument as file to write to. This all works fine, up to a certain amount of logging output... I've created a proof of concept Galaxy tool that uses my method of capturing logging output (30 lines in total for tool & config) and added that to this email. When run with the default settings on my Galaxy instance this tool only writes 6625 lines of logging output out of 10,000, after which I have to manually kill the process because it gets stuck. Could you please look into why producing more than a certain amount of logging output can lead to a tool getting stuck, and tell me what I can do to resolve this? (Preferably without changing the way I capture output: I don't think my use case is that much of a stretch) Greets, Tim

1 0