Incorporating Velvet and Interproscan into Galaxy
I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan. However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy. Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter. I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment. Any advice would be much appreciated. All the very best, Konrad. Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK. http://biosciences.exeter.ac.uk/facilities/sequencing/
Konrad, I think some folks at my facility are working on velvet and have gotten pretty far along. I've cc:ed one of them as I'm not sure if they're reading this list. Ideally we'd be able to upload our work to the galaxy list to provide you at least a starting point. I do know that they went down the composite datatype route and were able to represent the data thusly. On Thu, May 27, 2010 at 12:35:28PM +0100, Paszkiewicz, Konrad wrote:
I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan.
However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy.
Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter.
I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment.
Any advice would be much appreciated.
All the very best,
Konrad.
Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK.
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
-- Ry4an Brase 612-626-6575 University of Minnesota Supercomputing Institute for Advanced Computational Research http://www.msi.umn.edu
I hope I'm close to submitted velvet to the community site: http://community.g2.bx.psu.edu/ I still need to add test cases. Here's what I've done so far: I attached the a zip : $ zip -r velvet.zip * adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/velvet.py (deflated 74%) adding: tools/ (stored 0%) adding: tools/velvet/ (stored 0%) adding: tools/velvet/velvetg.xml (deflated 76%) adding: tools/velvet/velvetg_wrapper.py (deflated 63%) adding: tools/velvet/velveth.xml (deflated 63%) adding: tools/velvet/velveth_wrapper.py (deflated 65%) I also had to Add the import of velvet datatypes into lib/galaxy/datatypes/registry.py $ grep velvet lib/galaxy/datatypes/registry.py import data, tabular, interval, images, sequence, qualityscore, genetics, xml, coverage, tracks, chrominfo, binary, velvet Add velvet to datatypes_conf.xml $ grep velvet datatypes_conf.xml <datatype extension="Sequences" type="galaxy.datatypes.velvet:Sequences" display_in_upload="true"/> <datatype extension="Roadmaps" type="galaxy.datatypes.velvet:Roadmaps" display_in_upload="true"/> <datatype extension="velvet" type="galaxy.datatypes.velvet:Velvet" display_in_upload="true"/> <sniffer type="galaxy.datatypes.velvet:Sequences"/> <sniffer type="galaxy.datatypes.velvet:Roadmaps"/> Add velvet to tool_conf.xml $ grep velvet tool_conf.xml <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/> <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/> On 5/27/10 8:41 AM, Ry4an Brase wrote:
Konrad, I think some folks at my facility are working on velvet and have gotten pretty far along. I've cc:ed one of them as I'm not sure if they're reading this list. Ideally we'd be able to upload our work to the galaxy list to provide you at least a starting point. I do know that they went down the composite datatype route and were able to represent the data thusly.
On Thu, May 27, 2010 at 12:35:28PM +0100, Paszkiewicz, Konrad wrote:
I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan.
However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy.
Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter.
I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment.
Any advice would be much appreciated.
All the very best,
Konrad.
Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK.
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
I think I've finished a first version of velvet tools for galaxy. I moved velvet to tools/sr_assembly/ dir at the suggestion of Kevin Silverstein at UMN, he's working on the Roche gsAssembler for that category. As this contains additonal datatypes, I could use some advice on packaging this for the community website: http://community.g2.bx.psu.edu/ In the meantime, I've attached a zip : $ zip -r velvet.zip * adding: README (deflated 71%) adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/assembly.py (deflated 75%) adding: test-data/ (stored 0%) adding: test-data/velvet_test_long.fa (deflated 72%) adding: test-data/velvet_test_reads.fa (deflated 76%) adding: test-data/velvet_test_reference.fa (deflated 69%) adding: test-data/velvetg_test1/ (stored 0%) adding: test-data/velvetg_test1/amos.afg (deflated 81%) adding: test-data/velvetg_test1/contigs.fa (deflated 69%) adding: test-data/velvetg_test1/lastgraph.txt (deflated 62%) adding: test-data/velvetg_test1/stats.csv (deflated 42%) adding: test-data/velvetg_test1/unusedreads.fa (deflated 75%) adding: test-data/velveth_test1/ (stored 0%) adding: test-data/velveth_test1/Log (deflated 38%) adding: test-data/velveth_test1/output.html (deflated 41%) adding: test-data/velveth_test1/Roadmaps (deflated 69%) adding: test-data/velveth_test1/Sequences (deflated 75%) adding: tools/ (stored 0%) adding: tools/sr_assembly/ (stored 0%) adding: tools/sr_assembly/velvetg.xml (deflated 77%) adding: tools/sr_assembly/velvetg_wrapper.py (deflated 66%) adding: tools/sr_assembly/velveth.xml (deflated 63%) adding: tools/sr_assembly/velveth_wrapper.py (deflated 56%) On 5/27/10 9:02 AM, Jim Johnson wrote:
I hope I'm close to submitted velvet to the community site: http://community.g2.bx.psu.edu/
I still need to add test cases.
Here's what I've done so far:
I attached the a zip : $ zip -r velvet.zip * adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/velvet.py (deflated 74%) adding: tools/ (stored 0%) adding: tools/velvet/ (stored 0%) adding: tools/velvet/velvetg.xml (deflated 76%) adding: tools/velvet/velvetg_wrapper.py (deflated 63%) adding: tools/velvet/velveth.xml (deflated 63%) adding: tools/velvet/velveth_wrapper.py (deflated 65%)
I also had to Add the import of velvet datatypes into lib/galaxy/datatypes/registry.py $ grep velvet lib/galaxy/datatypes/registry.py import data, tabular, interval, images, sequence, qualityscore, genetics, xml, coverage, tracks, chrominfo, binary, velvet
Add velvet to datatypes_conf.xml $ grep velvet datatypes_conf.xml <datatype extension="Sequences" type="galaxy.datatypes.velvet:Sequences" display_in_upload="true"/> <datatype extension="Roadmaps" type="galaxy.datatypes.velvet:Roadmaps" display_in_upload="true"/> <datatype extension="velvet" type="galaxy.datatypes.velvet:Velvet" display_in_upload="true"/> <sniffer type="galaxy.datatypes.velvet:Sequences"/> <sniffer type="galaxy.datatypes.velvet:Roadmaps"/>
Add velvet to tool_conf.xml $ grep velvet tool_conf.xml <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/> <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/>
On 5/27/10 8:41 AM, Ry4an Brase wrote:
Konrad, I think some folks at my facility are working on velvet and have gotten pretty far along. I've cc:ed one of them as I'm not sure if they're reading this list. Ideally we'd be able to upload our work to the galaxy list to provide you at least a starting point. I do know that they went down the composite datatype route and were able to represent the data thusly.
On Thu, May 27, 2010 at 12:35:28PM +0100, Paszkiewicz, Konrad wrote:
I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan.
However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy.
Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter.
I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment.
Any advice would be much appreciated.
All the very best,
Konrad.
Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK.
http://biosciences.exeter.ac.uk/facilities/sequencing/
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Jim, If you agree, we would like to include your tools in the Galaxy distribution, rather than hosting them on the community space. If you can create a patch and send it, I'll apply it to the main code base. If you would rather have me just apply the files you sent, that would be fine as well. What do you think about this approach? Thanks Jim, Greg On Jun 3, 2010, at 12:03 PM, Jim Johnson wrote:
I think I've finished a first version of velvet tools for galaxy. I moved velvet to tools/sr_assembly/ dir at the suggestion of Kevin Silverstein at UMN, he's working on the Roche gsAssembler for that category.
As this contains additonal datatypes, I could use some advice on packaging this for the community website: http://community.g2.bx.psu.edu/
In the meantime, I've attached a zip : $ zip -r velvet.zip * adding: README (deflated 71%) adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/assembly.py (deflated 75%) adding: test-data/ (stored 0%) adding: test-data/velvet_test_long.fa (deflated 72%) adding: test-data/velvet_test_reads.fa (deflated 76%) adding: test-data/velvet_test_reference.fa (deflated 69%) adding: test-data/velvetg_test1/ (stored 0%) adding: test-data/velvetg_test1/amos.afg (deflated 81%) adding: test-data/velvetg_test1/contigs.fa (deflated 69%) adding: test-data/velvetg_test1/lastgraph.txt (deflated 62%) adding: test-data/velvetg_test1/stats.csv (deflated 42%) adding: test-data/velvetg_test1/unusedreads.fa (deflated 75%) adding: test-data/velveth_test1/ (stored 0%) adding: test-data/velveth_test1/Log (deflated 38%) adding: test-data/velveth_test1/output.html (deflated 41%) adding: test-data/velveth_test1/Roadmaps (deflated 69%) adding: test-data/velveth_test1/Sequences (deflated 75%) adding: tools/ (stored 0%) adding: tools/sr_assembly/ (stored 0%) adding: tools/sr_assembly/velvetg.xml (deflated 77%) adding: tools/sr_assembly/velvetg_wrapper.py (deflated 66%) adding: tools/sr_assembly/velveth.xml (deflated 63%) adding: tools/sr_assembly/velveth_wrapper.py (deflated 56%)
On 5/27/10 9:02 AM, Jim Johnson wrote:
I hope I'm close to submitted velvet to the community site: http://community.g2.bx.psu.edu/
I still need to add test cases.
Here's what I've done so far:
I attached the a zip : $ zip -r velvet.zip * adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/velvet.py (deflated 74%) adding: tools/ (stored 0%) adding: tools/velvet/ (stored 0%) adding: tools/velvet/velvetg.xml (deflated 76%) adding: tools/velvet/velvetg_wrapper.py (deflated 63%) adding: tools/velvet/velveth.xml (deflated 63%) adding: tools/velvet/velveth_wrapper.py (deflated 65%)
I also had to Add the import of velvet datatypes into lib/galaxy/datatypes/registry.py $ grep velvet lib/galaxy/datatypes/registry.py import data, tabular, interval, images, sequence, qualityscore, genetics, xml, coverage, tracks, chrominfo, binary, velvet
Add velvet to datatypes_conf.xml $ grep velvet datatypes_conf.xml <datatype extension="Sequences" type="galaxy.datatypes.velvet:Sequences" display_in_upload="true"/> <datatype extension="Roadmaps" type="galaxy.datatypes.velvet:Roadmaps" display_in_upload="true"/> <datatype extension="velvet" type="galaxy.datatypes.velvet:Velvet" display_in_upload="true"/> <sniffer type="galaxy.datatypes.velvet:Sequences"/> <sniffer type="galaxy.datatypes.velvet:Roadmaps"/>
Add velvet to tool_conf.xml $ grep velvet tool_conf.xml <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/> <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/>
On 5/27/10 8:41 AM, Ry4an Brase wrote:
Konrad, I think some folks at my facility are working on velvet and have gotten pretty far along. I've cc:ed one of them as I'm not sure if they're reading this list. Ideally we'd be able to upload our work to the galaxy list to provide you at least a starting point. I do know that they went down the composite datatype route and were able to represent the data thusly.
On Thu, May 27, 2010 at 12:35:28PM +0100, Paszkiewicz, Konrad wrote:
I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan.
However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy.
Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter.
I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment.
Any advice would be much appreciated.
All the very best,
Konrad.
Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK.
http://biosciences.exeter.ac.uk/facilities/sequencing/
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
<README.flv><velvet.zip>_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Greg Von Kuster Galaxy Development Team greg@bx.psu.edu
That would be great, thank. JJ On 6/8/10 3:10 PM, Greg Von Kuster wrote:
Jim,
If you agree, we would like to include your tools in the Galaxy distribution, rather than hosting them on the community space. If you can create a patch and send it, I'll apply it to the main code base. If you would rather have me just apply the files you sent, that would be fine as well. What do you think about this approach?
Thanks Jim,
Greg
On Jun 3, 2010, at 12:03 PM, Jim Johnson wrote:
I think I've finished a first version of velvet tools for galaxy. I moved velvet to tools/sr_assembly/ dir at the suggestion of Kevin Silverstein at UMN, he's working on the Roche gsAssembler for that category.
As this contains additonal datatypes, I could use some advice on packaging this for the community website: http://community.g2.bx.psu.edu/
In the meantime, I've attached a zip : $ zip -r velvet.zip * adding: README (deflated 71%) adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/assembly.py (deflated 75%) adding: test-data/ (stored 0%) adding: test-data/velvet_test_long.fa (deflated 72%) adding: test-data/velvet_test_reads.fa (deflated 76%) adding: test-data/velvet_test_reference.fa (deflated 69%) adding: test-data/velvetg_test1/ (stored 0%) adding: test-data/velvetg_test1/amos.afg (deflated 81%) adding: test-data/velvetg_test1/contigs.fa (deflated 69%) adding: test-data/velvetg_test1/lastgraph.txt (deflated 62%) adding: test-data/velvetg_test1/stats.csv (deflated 42%) adding: test-data/velvetg_test1/unusedreads.fa (deflated 75%) adding: test-data/velveth_test1/ (stored 0%) adding: test-data/velveth_test1/Log (deflated 38%) adding: test-data/velveth_test1/output.html (deflated 41%) adding: test-data/velveth_test1/Roadmaps (deflated 69%) adding: test-data/velveth_test1/Sequences (deflated 75%) adding: tools/ (stored 0%) adding: tools/sr_assembly/ (stored 0%) adding: tools/sr_assembly/velvetg.xml (deflated 77%) adding: tools/sr_assembly/velvetg_wrapper.py (deflated 66%) adding: tools/sr_assembly/velveth.xml (deflated 63%) adding: tools/sr_assembly/velveth_wrapper.py (deflated 56%)
On 5/27/10 9:02 AM, Jim Johnson wrote:
I hope I'm close to submitted velvet to the community site: http://community.g2.bx.psu.edu/
I still need to add test cases.
Here's what I've done so far:
I attached the a zip : $ zip -r velvet.zip * adding: lib/ (stored 0%) adding: lib/galaxy/ (stored 0%) adding: lib/galaxy/datatypes/ (stored 0%) adding: lib/galaxy/datatypes/velvet.py (deflated 74%) adding: tools/ (stored 0%) adding: tools/velvet/ (stored 0%) adding: tools/velvet/velvetg.xml (deflated 76%) adding: tools/velvet/velvetg_wrapper.py (deflated 63%) adding: tools/velvet/velveth.xml (deflated 63%) adding: tools/velvet/velveth_wrapper.py (deflated 65%)
I also had to Add the import of velvet datatypes into lib/galaxy/datatypes/registry.py $ grep velvet lib/galaxy/datatypes/registry.py import data, tabular, interval, images, sequence, qualityscore, genetics, xml, coverage, tracks, chrominfo, binary, velvet
Add velvet to datatypes_conf.xml $ grep velvet datatypes_conf.xml <datatype extension="Sequences" type="galaxy.datatypes.velvet:Sequences" display_in_upload="true"/> <datatype extension="Roadmaps" type="galaxy.datatypes.velvet:Roadmaps" display_in_upload="true"/> <datatype extension="velvet" type="galaxy.datatypes.velvet:Velvet" display_in_upload="true"/> <sniffer type="galaxy.datatypes.velvet:Sequences"/> <sniffer type="galaxy.datatypes.velvet:Roadmaps"/>
Add velvet to tool_conf.xml $ grep velvet tool_conf.xml <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/> <tool file="velvet/velveth.xml"/> <tool file="velvet/velvetg.xml"/>
On 5/27/10 8:41 AM, Ry4an Brase wrote:
Konrad, I think some folks at my facility are working on velvet and have gotten pretty far along. I've cc:ed one of them as I'm not sure if they're reading this list. Ideally we'd be able to upload our work to the galaxy list to provide you at least a starting point. I do know that they went down the composite datatype route and were able to represent the data thusly.
On Thu, May 27, 2010 at 12:35:28PM +0100, Paszkiewicz, Konrad wrote:
I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan.
However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy.
Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter.
I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment.
Any advice would be much appreciated.
All the very best,
Konrad.
Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK.
http://biosciences.exeter.ac.uk/facilities/sequencing/
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
<README.flv><velvet.zip>_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Greg Von Kuster Galaxy Development Team greg@bx.psu.edu
participants (4)
-
Greg Von Kuster -
Jim Johnson -
Paszkiewicz, Konrad -
Ry4an Brase