David, I'm cc'ing the galaxy-dev mailing list; let's keep this thread on the list for archival and community purposes.
I had a look through the Trinity.pl script and found that the default butterfly heap is set to 1G.
I've changed this to 2G. Would you re-run the trinity job and let me know if this helps.
Rather than mucking with the script, It's possible to change the Butterfly memory size using command line parameters: --- Butterfly options: # # --run_butterfly :executes butterfly commands. Do not set this if you want to spawn them on a computing grid. # # --bfly_opts :parameters to pass through to butterfly (see butterfly documentation). # # --bflyHeapSpace :java heap space setting for butterfly (default: 1000M) => yields command java -Xmx1000M -jar Butterfly.jar ... $bfly_opts --- This can be done in the Trinity wrapper at <galaxy-home>/tools/ngs_rna/trinity_all.xml ; there are already some Butterfly options set in the wrapper, so it should be easy to add more.
We see the following at the end of the Trinity logs - I don't know what it means but I thought you might... number of good components: 1 number of reads found = 1253 (from total of 1254) which came from 1252 pairs ## Read PathPair results: 1251 singletons, num pairs: 1, num pairs discarded: 0 total number of components = 1 working on subcomponent 0 method: combineSimilarPathsThatEndAtV(-1) with 1 paths method: combineSimilarPathsThatEndAtV(2) with 1 paths method: combineSimilarPathsThatEndAtV(1248) with 1 paths method: combineSimilarPathsThatEndAtV(1223) with 1 paths method: combineSimilarPathsThatEndAtV(-2) with 2 paths
Path Illustration:
==== PATH: [-1, 2, 1248, -2] === Read: [[-1, 2, 1248], []] count: 2 = Read: [[2], []] count: 1030 == Read: [[2, 1248], []] count: 23 === Read: [[2, 1248, -2], []] count: 2 = Read: [[1248], []] count: 197
Final path reported: comp12043_c0_seq1 len=1658 ~FPKM=29.7 path=[2:0-1220 1248:1221-1657] FinalPath: [-1, 2, 1248, -2] with support <1254, 1254> total number of paths reported = 1 from 1 components Done SUCCESS[53682]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.6/comp12043 -V 10 --stderr SUCCESS[53680]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.24/comp48425 -V 10 --stderr SUCCESS[53681]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.8/comp16349 -V 10 --stderr
Sorry, 1 of 53682 jobs failed. Overall, this looks very good - almost all the jobs succeeded. Failed commands written to file: failed_cmds.27222.txt
Error, cmd: /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/util/cmd_process_forker.pl -c /gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/database/job_working_directory/679/trinity_out_dir/chrysalis/butterfly_commands.adj --CPU 4 --shuffle died with ret 256 at /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Trinity.pl line 571. This is a bit cryptic to me. You'll want to determine what error code 256 means and see whether it may be a memory issue; you may also want to email the Trinity mailing list and add for help there.
Good luck, J.
HI, Thanks for the feedback we'll incorporate it into our setup. In the meantime, our fix inside the perl script (which increases the heap from 1GB to 2GB) appears to have worked a treat and we have lots of lovely looking transcripts! So for anyone following this that seems to be a good place to start if you are having issues with Trinity failing in a Galaxy setup. Best Wishes, David. __________________________________ Dr David A. Matthews Senior Lecturer in Virology Room E49 Department of Cellular and Molecular Medicine, School of Medical Sciences University Walk, University of Bristol Bristol. BS8 1TD U.K. Tel. +44 117 3312058 Fax. +44 117 3312091 D.A.Matthews@bristol.ac.uk On 11 Oct 2011, at 02:19, Jeremy Goecks wrote:
David,
I'm cc'ing the galaxy-dev mailing list; let's keep this thread on the list for archival and community purposes.
I had a look through the Trinity.pl script and found that the default butterfly heap is set to 1G.
I've changed this to 2G. Would you re-run the trinity job and let me know if this helps.
Rather than mucking with the script, It's possible to change the Butterfly memory size using command line parameters:
--- Butterfly options: # # --run_butterfly :executes butterfly commands. Do not set this if you want to spawn them on a computing grid. # # --bfly_opts :parameters to pass through to butterfly (see butterfly documentation). # # --bflyHeapSpace :java heap space setting for butterfly (default: 1000M) => yields command java -Xmx1000M -jar Butterfly.jar ... $bfly_opts ---
This can be done in the Trinity wrapper at <galaxy-home>/tools/ngs_rna/trinity_all.xml ; there are already some Butterfly options set in the wrapper, so it should be easy to add more.
We see the following at the end of the Trinity logs - I don't know what it means but I thought you might... number of good components: 1 number of reads found = 1253 (from total of 1254) which came from 1252 pairs ## Read PathPair results: 1251 singletons, num pairs: 1, num pairs discarded: 0 total number of components = 1 working on subcomponent 0 method: combineSimilarPathsThatEndAtV(-1) with 1 paths method: combineSimilarPathsThatEndAtV(2) with 1 paths method: combineSimilarPathsThatEndAtV(1248) with 1 paths method: combineSimilarPathsThatEndAtV(1223) with 1 paths method: combineSimilarPathsThatEndAtV(-2) with 2 paths
Path Illustration:
==== PATH: [-1, 2, 1248, -2] === Read: [[-1, 2, 1248], []] count: 2 = Read: [[2], []] count: 1030 == Read: [[2, 1248], []] count: 23 === Read: [[2, 1248, -2], []] count: 2 = Read: [[1248], []] count: 197
Final path reported: comp12043_c0_seq1 len=1658 ~FPKM=29.7 path=[2:0-1220 1248:1221-1657] FinalPath: [-1, 2, 1248, -2] with support <1254, 1254> total number of paths reported = 1 from 1 components Done SUCCESS[53682]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.6/comp12043 -V 10 --stderr SUCCESS[53680]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.24/comp48425 -V 10 --stderr SUCCESS[53681]: java -Xmx1G -jar /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Butterfly/Butterfly.jar -N 25506261 -L 300 -F 300 -C chrysalis/RawComps.8/comp16349 -V 10 --stderr
Sorry, 1 of 53682 jobs failed. Overall, this looks very good - almost all the jobs succeeded. Failed commands written to file: failed_cmds.27222.txt
Error, cmd: /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/util/cmd_process_forker.pl -c /gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/database/job_working_directory/679/trinity_out_dir/chrysalis/butterfly_commands.adj --CPU 4 --shuffle died with ret 256 at /gpfs/cluster/isys/galaxy/Galaxy/trinityrnaseq_r2011-08-20/Trinity.pl line 571. This is a bit cryptic to me. You'll want to determine what error code 256 means and see whether it may be a memory issue; you may also want to email the Trinity mailing list and add for help there.
Good luck, J.
participants (2)
-
David Matthews -
Jeremy Goecks