Hi, I'm trying to solve an issue I'm having with my local installation of Galaxy (installed on my own computer, rather than on a server). I'm using data in the form of fastq files from an Illumina Hi-seq and I want Galaxy to parse the bar coded sequences out into individual files for me. I've been using the public server in the past, and I'm able to use the Groomer, Joiner, Reverse-Complement, Trimmer, and Barcode Splitter tools just fine. Now I'm trying to do the same thing locally on the same files. Both the large file (38 GB) and a smaller file consisting of the first 10k lines will upload just fine. However, I can only get the Groomer to work on the small file. When I use it on the large file I get an error: "Error executing tool: maximum recursion depth exceeded while calling a Python object."
Any help on this would be greatly appreciated!
- Chris M.
Hi Chris,
Are you able to share a copy of the offending file? Perhaps by uploading to the public Galaxy server, possibly trying the Grooming step there, and then sharing the history with me or by using the bug report option on the Grooming step if it fails?
If not, you could look at the output created (the error dataset) and use e.g. tail to find out the last few fastq blocks that were successfully processed or wc -l to find out the number of lines written, and then use grep -C or a combination of head/tail to look at the original file and see if anything is amiss.
Thanks for using Galaxy,
Dan
On Oct 10, 2012, at 6:28 PM, Chris Merrikh wrote:
Hi, I'm trying to solve an issue I'm having with my local installation of Galaxy (installed on my own computer, rather than on a server). I'm using data in the form of fastq files from an Illumina Hi-seq and I want Galaxy to parse the bar coded sequences out into individual files for me. I've been using the public server in the past, and I'm able to use the Groomer, Joiner, Reverse-Complement, Trimmer, and Barcode Splitter tools just fine. Now I'm trying to do the same thing locally on the same files. Both the large file (38 GB) and a smaller file consisting of the first 10k lines will upload just fine. However, I can only get the Groomer to work on the small file. When I use it on the large file I get an error: "Error executing tool: maximum recursion depth exceeded while calling a Python object."
Any help on this would be greatly appreciated!
- Chris M.
The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Chris,
It sounds like the file is too large for your set up and there is a memory problem. There are a few options:
1 - If you really want to try to use your desktop computer, break the large file into smaller files before running the tools, then merge together data from the same barcode together once split away from the rest. This may get tedious depending on how many barcodes you are working with, but possible. Just double check that the tool you are running makes sense to run on the partial datasets you are providing as input (most of the tools you list will run on any partial dataset). I would suggest running some tests to see if these are small enough to work with in your environment before you invest too much in this strategy (you may find that you run into resource issues again in later steps, like mapping).
2 - Run Galaxy on a different local computer, one with more memory.
3 - Run Galaxy on a cloud instance with sufficient memory. http://usegalaxy.org/cloud
Going forward, the galaxy-dev@bx.psu.edu mailing list would be a good choice for local/cloud install questions: http://wiki.g2.bx.psu.edu/Support#Mailing_Lists
Best,
Jen Galaxy team
On 10/10/12 3:28 PM, Chris Merrikh wrote:
Hi, I'm trying to solve an issue I'm having with my local installation of Galaxy (installed on my own computer, rather than on a server). I'm using data in the form of fastq files from an Illumina Hi-seq and I want Galaxy to parse the bar coded sequences out into individual files for me. I've been using the public server in the past, and I'm able to use the Groomer, Joiner, Reverse-Complement, Trimmer, and Barcode Splitter tools just fine. Now I'm trying to do the same thing locally on the same files. Both the large file (38 GB) and a smaller file consisting of the first 10k lines will upload just fine. However, I can only get the Groomer to work on the small file. When I use it on the large file I get an error: "Error executing tool: maximum recursion depth exceeded while calling a Python object."
Any help on this would be greatly appreciated!
- Chris M.
The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
galaxy-user@lists.galaxyproject.org
-
Chris Merrikh -
Daniel Blankenberg -
Jennifer Jackson