Hi, I have hit a brick wall when trying to convert wig files from the GEO to bigwig files. Each time I try (and I have tried many times since October), I get the same error. For example, here is a downloaded wig file, that I assigned to the mouse mm8 genome, and the error I got when I tried to convert it to a bigwig file. The dataset came from Bing Ren's lab, and its GEO record is here: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM560344 The wig file was uploaded to Galaxy Dec. 8, 2011, and I assigned mm8 rather than mm9 based on the GEO record: 49: GSM560344_03112009_313D2AAXX_B7.wi ~960,000 lines format: wig, database: mm8 Info: uploaded wig file display at UCSC main The details for this upload are as follows: Tool: Upload File Name: GSM560344_03112009_313D2AAXX_B7.wi Created: Dec 08, 2011 Filesize: 12.1 Mb Dbkey: mm8 Format: wig Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Input Parameter Value File Format auto Genome Conditional (files_metadata) 32 Inheritance Chain GSM560344_03112009_313D2AAXX_B7.wi The wig-to-bigWig conversion on data 49 (using the wig to bigwig conversion tool in the convert formats toolbox) was run on March 21, 2012 and gave the following error: 77: Wig-to-bigWig on data 49 0 bytes An error occurred running this job:line 152351 of stdin: chromosome chr13 has 120614378 bases, but item ends at 120614600 line 298005 of stdin: chromosome chr17 has 95177420 bases, but item ends at 95177625 line 325066 of stdin: chromosome chr16 has 98252459 bases, but item ends at 9825252 The details for this operation are as follows: Tool: Wig-to-bigWig Name: Wig-to-bigWig on data 49 Created: Mar 21, 2012 Filesize: 0 bytes Dbkey: mm8 Format: bigwig Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Input Parameter Value Convert 49: GSM560344_03112009_313D2AAXX_B7.wi Conditional (settings) 1 Items to bundle in r-tree 256 Data points bundled at lowest level 1024 Clip chromosome positions True Do not use compression True Inheritance Chain Wig-to-bigWig on data 49 I gather that the chromosome ends are not being snipped off, even though I toggle this option on the galaxy conversion tool. And I know it's doing something, because if I toggle that option off, I get an error that includes "broken pipe" and simply aborts. I apologize for knowing so little about the bioinformatics involved here. And I'm sure I've overlooked something that is likely obvious to others and/or failed to provide some critical bit of info in this email. But any help would be greatly appreciated. Thanks, Mike
Hi Michael, This particular .wig file has a data format problem that is the root cause of the conversion error. Specifically, there is an extra track line in the file. This can be found using unix tools with a grep or in Galaxy with the tool "Filter and Sort -> Select" by matching the pattern "track". Ideally this would be corrected and resubmitted by the data author before use, since how/why this was inserted and what impact it has would need to be examined. Since you noticed problems with other GEO files (conversion problems), verifying the .wig format and making any necessary corrections would also be advised. Hopefully this helps! Best, Jen Galaxy team On 4/13/12 6:19 AM, Michael Sikes wrote:
Hi,
I have hit a brick wall when trying to convert wig files from the GEO to bigwig files. Each time I try (and I have tried many times since October), I get the same error. For example, here is a downloaded wig file, that I assigned to the mouse mm8 genome, and the error I got when I tried to convert it to a bigwig file. The dataset came from Bing Ren's lab, and its GEO record is here: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM560344
The wig file was uploaded to Galaxy Dec. 8, 2011, and I assigned mm8 rather than mm9 based on the GEO record:
49: GSM560344_03112009_313D2AAXX_B7.wi <https://main.g2.bx.psu.edu/history> ~960,000 lines format: wig, database: mm8 Info: uploaded wig file <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/display?to_ext=wig><https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=4746000><https://main.g2.bx.psu.edu/history> <https://main.g2.bx.psu.edu/tag/retag?item_id=47465bc44dbd111e&item_class=HistoryDatasetAssociation><https://main.g2.bx.psu.edu/dataset/annotate?id=47465bc44dbd111e> display at UCSC main <https://main.g2.bx.psu.edu/datasets/4746000/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dmm8%26position%3Dchr11%3A3000251-3023451%26hgt.customText%3D%25s&display_url=https%3A%2F%2Fmain.g2.bx.psu.edu%2Froot%2Fdisplay_as%3Fid%3D4746000%26display_app%3Ducsc%26authz_method%3Ddisplay_at>
The details for this upload are as follows:
Tool: Upload File Name: GSM560344_03112009_313D2AAXX_B7.wi Created: Dec 08, 2011 Filesize: 12.1 Mb Dbkey: mm8 Format: wig Tool Version: Tool Standard Output: stdout <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stdout> Tool Standard Error: stderr <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stderr>
Input Parameter Value File Format auto Genome Conditional (files_metadata) 32
Inheritance Chain
GSM560344_03112009_313D2AAXX_B7.wi
The wig-to-bigWig conversion on data 49 (using the wig to bigwig conversion tool in the convert formats toolbox) was run on March 21, 2012 and gave the following error: 77: Wig-to-bigWig on data 49 <https://main.g2.bx.psu.edu/history> 0 bytes An error occurred running this job:/line 152351 of stdin: chromosome chr13 has 120614378 bases, but item ends at 120614600 line 298005 of stdin: chromosome chr17 has 95177420 bases, but item ends at 95177625 line 325066 of stdin: chromosome chr16 has 98252459 bases, but item ends at 9825252/ <https://main.g2.bx.psu.edu/dataset/errors?id=6041022><https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=6041022> <https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/display/?preview=True><https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/edit><https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/delete?show_deleted_on_refresh=False>
The details for this operation are as follows:
Tool: Wig-to-bigWig Name: Wig-to-bigWig on data 49 Created: Mar 21, 2012 Filesize: 0 bytes Dbkey: mm8 Format: bigwig Tool Version: Tool Standard Output: stdout <https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stdout> Tool Standard Error: stderr <https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stderr>
Input Parameter Value Convert 49: GSM560344_03112009_313D2AAXX_B7.wi Conditional (settings) 1 Items to bundle in r-tree 256 Data points bundled at lowest level 1024 Clip chromosome positions True Do not use compression True
Inheritance Chain
Wig-to-bigWig on data 49
I gather that the chromosome ends are not being snipped off, even though I toggle this option on the galaxy conversion tool. And I know it's doing something, because if I toggle that option off, I get an error that includes "broken pipe" and simply aborts. I apologize for knowing so little about the bioinformatics involved here. And I'm sure I've overlooked something that is likely obvious to others and/or failed to provide some critical bit of info in this email. But any help would be greatly appreciated.
Thanks, Mike
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org http://galaxyproject.org/wiki/Support
Jennifer, Thanks for your help. I ran the filter and sort tool as advised, and then ran the wig to bigwig on the new history item generated by the filter. This time I got a different error: 84: Wig-to-bigWig on data 83 0 bytes An error occurred running this job:stdin is empty of data Error running wigToBigWig. 83: Select on data 49 1 line, 1 comments format: wig, database: mm8 Info: Matching pattern: track Again, I'm sure I left off something obvious. Could you tell me what I did wrong? Thanks, Mike On Apr 13, 2012, at 1:27 PM, Jennifer Jackson wrote:
Hi Michael,
This particular .wig file has a data format problem that is the root cause of the conversion error. Specifically, there is an extra track line in the file. This can be found using unix tools with a grep or in Galaxy with the tool "Filter and Sort -> Select" by matching the pattern "track".
Ideally this would be corrected and resubmitted by the data author before use, since how/why this was inserted and what impact it has would need to be examined.
Since you noticed problems with other GEO files (conversion problems), verifying the .wig format and making any necessary corrections would also be advised.
Hopefully this helps!
Best,
Jen Galaxy team
On 4/13/12 6:19 AM, Michael Sikes wrote:
Hi,
I have hit a brick wall when trying to convert wig files from the GEO to bigwig files. Each time I try (and I have tried many times since October), I get the same error. For example, here is a downloaded wig file, that I assigned to the mouse mm8 genome, and the error I got when I tried to convert it to a bigwig file. The dataset came from Bing Ren's lab, and its GEO record is here: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM560344
The wig file was uploaded to Galaxy Dec. 8, 2011, and I assigned mm8 rather than mm9 based on the GEO record:
49: GSM560344_03112009_313D2AAXX_B7.wi <https://main.g2.bx.psu.edu/history
~960,000 lines format: wig, database: mm8 Info: uploaded wig file <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/display?to_ext=wig
<https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=4746000 <https://main.g2.bx.psu.edu/history> <https://main.g2.bx.psu.edu/tag/retag?item_id=47465bc44dbd111e&item_class=HistoryDatasetAssociation <https://main.g2.bx.psu.edu/dataset/annotate?id=47465bc44dbd111e> display at UCSC main <https://main.g2.bx.psu.edu/datasets/4746000/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dmm8%26position%3Dchr11%3A3000251-3023451%26hgt.customText%3D%25s&display_url=https%3A%2F%2Fmain.g2.bx.psu.edu%2Froot%2Fdisplay_as%3Fid%3D4746000%26display_app%3Ducsc%26authz_method%3Ddisplay_at
The details for this upload are as follows:
Tool: Upload File Name: GSM560344_03112009_313D2AAXX_B7.wi Created: Dec 08, 2011 Filesize: 12.1 Mb Dbkey: mm8 Format: wig Tool Version: Tool Standard Output: stdout <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stdout> Tool Standard Error: stderr <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stderr>
Input Parameter Value File Format auto Genome Conditional (files_metadata) 32
Inheritance Chain
GSM560344_03112009_313D2AAXX_B7.wi
The wig-to-bigWig conversion on data 49 (using the wig to bigwig conversion tool in the convert formats toolbox) was run on March 21, 2012 and gave the following error: 77: Wig-to-bigWig on data 49 <https://main.g2.bx.psu.edu/history> 0 bytes An error occurred running this job:/line 152351 of stdin: chromosome chr13 has 120614378 bases, but item ends at 120614600 line 298005 of stdin: chromosome chr17 has 95177420 bases, but item ends at 95177625 line 325066 of stdin: chromosome chr16 has 98252459 bases, but item ends at 9825252/ <https://main.g2.bx.psu.edu/dataset/errors?id=6041022><https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/show_params
<https://main.g2.bx.psu.edu/tool_runner/rerun?id=6041022> <https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/display/?preview=True <https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/edit><https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/delete?show_deleted_on_refresh=False
The details for this operation are as follows:
Tool: Wig-to-bigWig Name: Wig-to-bigWig on data 49 Created: Mar 21, 2012 Filesize: 0 bytes Dbkey: mm8 Format: bigwig Tool Version: Tool Standard Output: stdout <https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stdout> Tool Standard Error: stderr <https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stderr>
Input Parameter Value Convert 49: GSM560344_03112009_313D2AAXX_B7.wi Conditional (settings) 1 Items to bundle in r-tree 256 Data points bundled at lowest level 1024 Clip chromosome positions True Do not use compression True
Inheritance Chain
Wig-to-bigWig on data 49
I gather that the chromosome ends are not being snipped off, even though I toggle this option on the galaxy conversion tool. And I know it's doing something, because if I toggle that option off, I get an error that includes "broken pipe" and simply aborts. I apologize for knowing so little about the bioinformatics involved here. And I'm sure I've overlooked something that is likely obvious to others and/or failed to provide some critical bit of info in this email. But any help would be greatly appreciated.
Thanks, Mike
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org http://galaxyproject.org/wiki/Support
Michael Sikes, Ph.D. Associate Professor of Immunology North Carolina State University Microbiology Department 4524A Gardner Hall Campus Box 7615 Raleigh, NC 27695 Ph: 919-513-0528 Fax: 919-515-7867 email: mlsikes@ncsu.edu
Hi Mike, I apologize if I wasn't clear, but the 'Select' was to show you how to identify the multi-track group wig files. I wanted to give you a way to screen similar files going forward. The wig-to-bigWig program in Galaxy comes from UCSC. It accepts .wig files with a single track group as input: http://genome.ucsc.edu/goldenPath/help/bigWig.html (see step #1) The data author lab can either submit the data as single track group .wig files, or, if you are confident that the multiple track group .wig format is expected and OK from this source, split the file. There are no specific tools in Galaxy to do this, but something like this would work: - Text Manipulation -> "Add column", "1", Iterate? = yes - "Select", "track" - note the line number of track lines - "Remove beginning of a file", using line numbers, and the -original- .wig file, to break up into individual .wig files. Good luck! Jen Galaxy team On 4/16/12 6:57 AM, Michael Sikes wrote:
Jennifer,
Thanks for your help. I ran the filter and sort tool as advised, and then ran the wig to bigwig on the new history item generated by the filter. This time I got a different error: 84: Wig-to-bigWig on data 83 <https://main.g2.bx.psu.edu/history> 0 bytes An error occurred running this job:/stdin is empty of data Error running wigToBigWig. / <https://main.g2.bx.psu.edu/dataset/errors?id=6818347><https://main.g2.bx.psu.edu/datasets/0f70746579b165e2/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=6818347> <https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/display/?preview=True><https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/edit><https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/delete?show_deleted_on_refresh=False> 83: Select on data 49 <https://main.g2.bx.psu.edu/history> 1 line, 1 comments format: wig, database: mm8 Info: Matching pattern: track <https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/display?to_ext=wig><https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=6818275><https://main.g2.bx.psu.edu/history> <https://main.g2.bx.psu.edu/tag/retag?item_id=b4fb2e8c767b4258&item_class=HistoryDatasetAssociation><https://main.g2.bx.psu.edu/dataset/annotate?id=b4fb2e8c767b4258>
Again, I'm sure I left off something obvious. Could you tell me what I did wrong?
Thanks, Mike
On Apr 13, 2012, at 1:27 PM, Jennifer Jackson wrote:
Hi Michael,
This particular .wig file has a data format problem that is the root cause of the conversion error. Specifically, there is an extra track line in the file. This can be found using unix tools with a grep or in Galaxy with the tool "Filter and Sort -> Select" by matching the pattern "track".
Ideally this would be corrected and resubmitted by the data author before use, since how/why this was inserted and what impact it has would need to be examined.
Since you noticed problems with other GEO files (conversion problems), verifying the .wig format and making any necessary corrections would also be advised.
Hopefully this helps!
Best,
Jen Galaxy team
On 4/13/12 6:19 AM, Michael Sikes wrote:
Hi,
I have hit a brick wall when trying to convert wig files from the GEO to bigwig files. Each time I try (and I have tried many times since October), I get the same error. For example, here is a downloaded wig file, that I assigned to the mouse mm8 genome, and the error I got when I tried to convert it to a bigwig file. The dataset came from Bing Ren's lab, and its GEO record is here: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM560344
The wig file was uploaded to Galaxy Dec. 8, 2011, and I assigned mm8 rather than mm9 based on the GEO record:
49: GSM560344_03112009_313D2AAXX_B7.wi <https://main.g2.bx.psu.edu/history> ~960,000 lines format: wig, database: mm8 Info: uploaded wig file <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/display?to_ext=wig><https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=4746000><https://main.g2.bx.psu.edu/history> <https://main.g2.bx.psu.edu/tag/retag?item_id=47465bc44dbd111e&item_class=HistoryDatasetAssociation <https://main.g2.bx.psu.edu/tag/retag?item_id=47465bc44dbd111e&item_class=HistoryDatasetAssociation>><https://main.g2.bx.psu.edu/dataset/annotate?id=47465bc44dbd111e> display at UCSC main <https://main.g2.bx.psu.edu/datasets/4746000/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dmm8%26position%3Dchr11%3A3000251-3023451%26hgt.customText%3D%25s&display_url=https%3A%2F%2Fmain.g2.bx.psu.edu%2Froot%2Fdisplay_as%3Fid%3D4746000%26display_app%3Ducsc%26authz_method%3Ddisplay_at <https://main.g2.bx.psu.edu/datasets/4746000/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dmm8%26position%3Dchr11%3A3000251-3023451%26hgt.customText%3D%25s&display_url=https%3A%2F%2Fmain.g2.bx.psu.edu%2Froot%2Fdisplay_as%3Fid%3D4746000%26display_app%3Ducsc%26authz_method%3Ddisplay_at>>
The details for this upload are as follows:
Tool: Upload File Name:GSM560344_03112009_313D2AAXX_B7.wi Created:Dec 08, 2011 Filesize:12.1 Mb Dbkey:mm8 Format:wig Tool Version: Tool Standard Output:stdout <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stdout> Tool Standard Error:stderr <https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stderr>
Input ParameterValue File Formatauto Genome Conditional (files_metadata)32
Inheritance Chain
GSM560344_03112009_313D2AAXX_B7.wi
The wig-to-bigWig conversion on data 49 (using the wig to bigwig conversion tool in the convert formats toolbox) was run on March 21, 2012 and gave the following error: 77: Wig-to-bigWig on data 49 <https://main.g2.bx.psu.edu/history> 0 bytes An error occurred running this job:/line 152351 of stdin: chromosome chr13 has 120614378 bases, but item ends at 120614600 line 298005 of stdin: chromosome chr17 has 95177420 bases, but item ends at 95177625 line 325066 of stdin: chromosome chr16 has 98252459 bases, but item ends at 9825252/ <https://main.g2.bx.psu.edu/dataset/errors?id=6041022><https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/show_params><https://main.g2.bx.psu.edu/tool_runner/rerun?id=6041022> <https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/display/?preview=True><https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/edit><https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/delete?show_deleted_on_refresh=False>
The details for this operation are as follows:
Tool: Wig-to-bigWig Name:Wig-to-bigWig on data 49 Created:Mar 21, 2012 Filesize:0 bytes Dbkey:mm8 Format:bigwig Tool Version: Tool Standard Output:stdout <https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stdout> Tool Standard Error:stderr <https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stderr>
Input ParameterValue Convert49: GSM560344_03112009_313D2AAXX_B7.wi Conditional (settings)1 Items to bundle in r-tree256 Data points bundled at lowest level1024 Clip chromosome positionsTrue Do not use compressionTrue
Inheritance Chain
Wig-to-bigWig on data 49
I gather that the chromosome ends are not being snipped off, even though I toggle this option on the galaxy conversion tool. And I know it's doing something, because if I toggle that option off, I get an error that includes "broken pipe" and simply aborts. I apologize for knowing so little about the bioinformatics involved here. And I'm sure I've overlooked something that is likely obvious to others and/or failed to provide some critical bit of info in this email. But any help would be greatly appreciated.
Thanks, Mike
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org http://galaxyproject.org/wiki/Support
Michael Sikes, Ph.D. Associate Professor of Immunology North Carolina State University Microbiology Department 4524A Gardner Hall Campus Box 7615 Raleigh, NC 27695 Ph: 919-513-0528 Fax: 919-515-7867 email: mlsikes@ncsu.edu <mailto:mlsikes@ncsu.edu>
-- Jennifer Jackson http://galaxyproject.org
participants (2)
-
Jennifer Jackson -
Michael Sikes