How to add megablast database?
by Kelly Vincent
Begin forwarded message:
> From: Kelly Vincent <kpvincent(a)bx.psu.edu>
> Date: July 26, 2010 11:23:47 AM EDT
> To: buiduyminh(a)gmail.com
> Cc: Galaxy Lab <galaxy-lab(a)bx.psu.edu>
> Subject: Re: [galaxy-lab] [galaxy-dev] How to add megablast database?
>
> Hi,
>
> See the wiki page http://bitbucket.org/galaxy/galaxy-central/wiki/NGSLocalSetup
> for information on getting the database files in place.
>
> If you want to use blastn instead of megablast, you would just need
> to modify the wrapper slightly. You will at minimum need to modify
> tools/metag_tools/megablast_wrapper.py, by changing the command-line
> call (line 72) appropriately. If you need only the same parameters
> as megablast uses, you can just change the python file that is
> called in the command tag in tools/metag_tools/
> megablast_wrapper.xml. Otherwise, you'll need to add relevant
> parameters to the XML and python files.
>
> Let us know if you need any further assistance.
>
> Regards,
> Kelly
>
>
>
>
>
>
> On Jul 24, 2010, at 10:56 AM, Minh Bui wrote:
>
>> Hi,
>>
>> I am using local Galaxy on my computer. I would like to use
>> megablast but I don't know how to add wgs and nt database to local
>> galaxy (I am new to galaxy).
>>
>> 1. Where can I download those databases (if you dont mind, can you
>> give me the link?) ?
>> 2. How can I add those databases to galaxy?
>> 3. What if I want to do BLASTN instead of megablast, how can I do
>> that?
>>
>> Thank you very much,
>> _______________________________________________
>> galaxy-dev mailing list
>> galaxy-dev(a)lists.bx.psu.edu
>> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> _______________________________________________
> galaxy-lab mailing list
> galaxy-lab(a)lists.bx.psu.edu
> http://lists.bx.psu.edu/listinfo/galaxy-lab
9 years, 6 months
How to add megablast database?
by Minh Bui
Hi,
I am using local Galaxy on my computer. I would like to use megablast but I
don't know how to add wgs and nt database to local galaxy (I am new to
galaxy).
1. Where can I download those databases (if you dont mind, can you give me
the link?) ?
2. How can I add those databases to galaxy?
3. What if I want to do BLASTN instead of megablast, how can I do that?
Thank you very much,
9 years, 6 months
Huge memory usage in latest Galaxy
by Assaf Gordon
Hello all,
After upgrading to a recent version (47411bb09df4), memory usage reaches 12GB per python process after less then 48 hours (with normal usage by users).
It only happens with the web-side processes, the job-runner process and the reports process are at their usual usage level.
Debug is off and Interactive is off,
The major change is that I've started using SGE (with DRMAA).
Any tips would be very appreciated.
-gordon
9 years, 6 months
Re: [galaxy-dev] [galaxy-bugs] Fwd: [galaxy-user] Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
by Anton Nekrutenko
Ameena:
454 gives two types of files - reads in fasta format:
>FWV11AZ02JY4XI length=110 xy=3973_2596 region=2 run=R_2009_06_02_15_16_12_TTCATGGAGAAGGCTTAGTGCTCTTTACACAGCCACTAACAACAGGTGAGGTGCTTATAAGTCTCAATCAGAAACAGAGTCTTACCCTTCGTTATTCTAAACAAACTACT
>FWV11AZ02G3SEQ length=134 xy=2796_1792 region=2 run=R_2009_06_02_15_16_12_TTCTATTTAAGGTGCACATGGTTGAGCTCTAGGGCTTTTTAAGACACTGTGAATCTCAAATTATTTGCATGACTTGAAAGGGCACTTTGAATTAGTGAAAAGTCTTGGGAATACCAAAAAAATTTTTAATTTTT
and associated quality scores:
>FWV11AZ02JY4XI length=110 xy=3973_2596 region=2 run=R_2009_06_02_15_16_12_37 37 37 37 37 37 37 37 37 37 37 40 40 40 40 40 40 40 40 40 40 40 40 39 39 39 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 37 37 37 37 37 37 37 37 37 37 35 33 33 33 32 31 31 24 24 15 15 15 15 15 29 30 28 27 27 26 26 30 23 23 23 18 17 14 19 13 13 14 17 17 19 16 16 17 16 20 16 16 16 17 14 12 14 14
>FWV11AZ02G3SEQ length=134 xy=2796_1792 region=2 run=R_2009_06_02_15_16_12_37 37 37 37 37 37 37 37 37 37 37 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 31 31 31 40 28 28 28 28 28 40 40 40 40 40 40 40 40 40 40 40 40 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 30 30 30 37 37 37 37 37 37 37 37 37 36 36 36 36 37 37 37 37 37 37 37 37 37 37 37 35 35 34 16 16 17 17 17 14 15 15 15 12 12 12 10 10 11 11 11 11 11
In the case of metagenomic analysis you only need quality scores if you will be doing selection of high quality segments from your reads, which you can skip if you do not have quality files handy. However, before you begin make sure you understand what is involved in this analysis. A good place to start would a galaxy page for our Genome Research publication:
http://main.g2.bx.psu.edu/u/aun1/p/windshield-splatter
Thanks,
anton
galaxy team
On Jul 23, 2010, at 12:44 PM, Anton Nekrutenko wrote:
> Ameena:
>
> I am forwarding this to galaxy-bugs mailing list, which more appropriate for your question.
>
> anton
> galaxy team
>
> Begin forwarded message:
>
>> From: "Ameena Nalim" <fanalim(a)nmsu.edu>
>> Date: July 23, 2010 11:30:48 AM EDT
>> To: galaxy-dev(a)bx.psu.edu
>> Subject: Re: [galaxy-dev] [galaxy-user] Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
>>
>>
>> Hi!
>> I am a new user. I have tried unsuccessfully to follow the steps in the
>> screencasts for Metagenome Analysis with Galaxy.
>> I have successfully uploaded the fasta file in .fas and .fna format from
>> 454 sequencing. The dataset is some 150k plus sequences.
>> I am unable to create a quality score format to proceed.
>> Is this something to do with my fasta file?
>>
>> My goal is to draw a phylogeny like that possible with MEGAN
>> Would you be able to suggest a workflow that will work?
>> I have added this email to share my history.
>> My public name is fanalim
>> the history is the only one saved
>>
>> thanks
>> Ameena
>>
>>
>>> Hello Galaxy Community,
>>>
>>> The Galaxy Community Space has been renamed to be the Galaxy Tool Shed (
>> contributed by Dave Clements ), the location is still
>>> http://usegalaxy.org/community.
>>>
>>> The site now includes the ability to upload an archive consisting of a
>> suite of tools in addition to a single tool archive.
>>>
>>> Here is the information about uploading both of these types of archives.
>> Please start sharing tools, and let us know if you encounter any
>> problems.
>>> Also, let us know what new features you would like to see in the tool
>>> shed. Our next update will include the ability to rate tools.
>>>
>>>
>>> Creating an archive containing a tool or a suite of tools
>>> A tool or tool suite archive is a tar-format file (bzipped or gzipped
>> tar are valid) containing all the files necessary to load the tool(s)
>> into a Galaxy instance.
>>>
>>> Tool Suite Archive
>>>
>>> A tools suite must include a file named suite_config.xml which provides
>> information about the id, name, version and description of the tool
>> suite, as well as the id, name, version and description of each tool in
>> the suite. Here is an example suite_config.xml file.
>>>
>>>
>>> <suite id="lastz_toolsuite" name="Suite of Lastz tools"
>>> version="1.0.0">
>>> <description>This suite contains all of my Lastz tools for
>>> Galaxy</description>
>>> <tool id="lastz_wrapper_2" name="Lastz" version="1.1.0">
>>> <description> map short reads against reference
>>> sequence</description>
>>> </tool>
>>> <tool id="lastz_paired_reads_wrapper" name="Lastz paired reads"
>>> version="1.0.0">
>>> <description> map short paired reads against reference
>>> sequence</description>
>>> </tool>
>>> </suite>
>>>
>>>
>>> New versions of the suite can be uploaded, replacing an older version of
>> the suite, but the version attribute of the tag must be altered the same
>> way that the version attribute of a single tool config must be altered
>> if uploading a new version of a tool.
>>>
>>> The id, name and version attributes of each tag in the suite_config.xml
>> file must exactly match the same attributes in each associated tool
>> config in the archive or you will not be allowed to upload the archive.
>>>
>>> In addition to the suite_config.xml file, the archive must include all
>> tool config files, executables, functional test data (if your tool
>> config includes functional tests) and other files needed for each of the
>> tools in your suite to function within Galaxy. See the information about
>> single tool archives below for additional hints to enable ease-of-use
>> when others download your suite of tools.
>>>
>>> For example, to package the above Lastz suite of tools:
>>>
>>> user@host:~% tar jcvf ~/Desktop/galaxy_lastz_toolsuite.tar.bz2
>>> lastzsuite
>>> lastzsuite/
>>> lastzsuite/README
>>> lastzsuite/suite_config.xml
>>> lastzsuite/lastz_paired_reads_wrapper.py
>>> lastzsuite/lastz_paired_reads_wrapper.xml
>>> lastzsuite/lastz_wrapper.py
>>> lastzsuite/lastz_wrapper.xml
>>> lastzsuite/lastz-distrib-1.02.00/
>>> lastzsuite/lastz-distrib-1.02.00/src/
>>> lastzsuite/lastz-distrib-1.02.00/src/Makefile
>>> lastzsuite/lastz-distrib-1.02.00/src/version.mak
>>> lastzsuite/lastz-distrib-1.02.00/src/lastz.c
>>> lastzsuite/lastz-distrib-1.02.00/src/lastz.h
>>> ...
>>> ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded.
>>>
>>> Single Tool Archive
>>>
>>> A single tool archive must include a tool config file and will probably
>> also include a tool script. If any steps are necessary to install your
>> tool beyond the basic instructions below, include a README file to
>> provide details. If the tool (or parts of it) are written in C, the
>> source code can be included (or put links to the source in the README).
>> Do not include pre-compiled binaries without source since Galaxy is run
>> on a wide variety of platforms. Also, if you are only wrapping or
>> providing a Galaxy config for a tool that is not your own, be sure the
>> license allows for
>>> redistribution before including any part of that tool in the archive.
>>>
>>> There are no requirements about the directory structure inside the
>> archive, but for ease of use it's generally a good idea to put
>> everything inside a sub-directory, instead of directly at the top level.
>>>
>>> For example, to package the Lastz tool's config file, Galaxy wrapper,
>> and the C source:
>>>
>>> user@host:~% tar jcvf ~/Desktop/galaxy_lastz_tool.tar.bz2 lastz lastz/
>>> lastz/README
>>> lastz/lastz_wrapper.py
>>> lastz/lastz_wrapper.xml
>>> lastz/lastz-distrib-1.02.00/
>>> lastz/lastz-distrib-1.02.00/src/
>>> lastz/lastz-distrib-1.02.00/src/Makefile
>>> lastz/lastz-distrib-1.02.00/src/version.mak
>>> lastz/lastz-distrib-1.02.00/src/lastz.c
>>> lastz/lastz-distrib-1.02.00/src/lastz.h
>>> ...
>>> ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded.
>>>
>>> Editing Information, Categories, and Submitting For Approval
>>>
>>> Simply uploading a tool to the Galaxy too shed will not allow other
>> users to find and download your tool. It will need to be approved by an
>> administrator before it appears in the tool list.
>>>
>>> After your archive has successfully uploaded, you will be redirected to
>> the Edit Tool page. Provide a detailed description of what the tool does
>> - this will be used by administrators to understand the tool before
>> approving it for display on the site. Once approved, this information
>> will be displayed to users who view your tool. In addition, the site
>>> administrators will have configured a number of categories with which
>> you can associate your tool to make it easy to find by users looking to
>> solve specific problems. Associate as many categories as are relevant to
>> your tool. You may change the description and associated categories as
>> often as you'd like until you click the "Submit for approval" button.
>> Once submitted, the tool will be approved or rejected by an
>> administrator. If the tool is rejected, you will see information about
>> why it was rejected, and you can make appropriate changes to the archive
>> and re-submit it for approval. When it is approved, your archive will be
>> visible to everyone. At that point, the description and associated
>> categories can only be changed by an administrator.
>>>
>>> When the tool has been approved or rejected, you may upload a new
>> version by browsing to the tool's "View Tool" page, clicking the "Tool
>> actions" menu in the upper right corner of the page, and selecting
>> "Upload a new version" from the menu.
>>>
>>> Downloading and Installing Tools
>>>
>>> A tool's download link will send you the tool archive. Once downloaded,
>> unpack the tool on your local Galaxy instance's server:
>>>
>>> user@host:~% tar xvf galaxy_lastz_tool.tar
>>> ...
>>> user@host:~% tar zxvf galaxy_lastz_tool.tar.gz
>>> ...
>>> user@host:~% tar jxvf galaxy_lastz_tool.tar.bz2
>>> ...
>>> If the archive includes a README file, consult it for installation
>> instructions. If not, follow these basic steps:
>>> Create a directory under galaxy_dist/tools/ to house downloaded tool(s).
>> In the new directory, place the XML and any script file(s) which were
>> contained in the archive.
>>> If the tool includes binaries, you'll need to copy them to a directory
>> on your $PATH. If the tool depends on C binaries but does not come with
>> them (only source), you'll need to compile the source first.
>>> Add the tool to galaxy_dist/tool_conf.xml.
>>> Restart your Galaxy server process.
>>>
>>> In the near future, we plan to implement a more direct method to install
>> tools via the Galaxy administrator user interface instead of placing
>> files on the filesystem and manually managing the tool_conf.xml file.
>>>
>>>
>>>
>>> Greg Von Kuster
>>> Galaxy Development Team
>>> greg(a)bx.psu.edu
>>>
>>>
>>>
>>> _______________________________________________
>>> galaxy-user mailing list
>>> galaxy-user(a)lists.bx.psu.edu
>>> http://lists.bx.psu.edu/listinfo/galaxy-user
>>>
>>
>>
>> --
>> F. Ameena Nalim, Ph.D
>> Department of Plant & Environmental Sciences
>> Adj. Faculty, Molecular Biology Program
>> New Mexico State University
>> Las Cruces, NM 88003
>> e-mail:fanalim@nmsu.edu
>> http://aces.nmsu.edu/academics/adrianunc/labmembers.html
>> http://research.nmsu.edu/molbio/
>>
>>
>>
>> --
>> F. Ameena Nalim, Ph.D
>> Department of Plant & Environmental Sciences
>> Adj. Faculty, Molecular Biology Program
>> New Mexico State University
>> Las Cruces, NM 88003
>> e-mail:fanalim@nmsu.edu
>> http://aces.nmsu.edu/academics/adrianunc/labmembers.html
>> http://research.nmsu.edu/molbio/
>>
>> _______________________________________________
>> galaxy-dev mailing list
>> galaxy-dev(a)lists.bx.psu.edu
>> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> Anton Nekrutenko
> http://nekrut.bx.psu.edu
> http://usegalaxy.org
>
>
>
> _______________________________________________
> galaxy-bugs mailing list
> galaxy-bugs(a)lists.bx.psu.edu
> http://lists.bx.psu.edu/listinfo/galaxy-bugs
Anton Nekrutenko
http://nekrut.bx.psu.edu
http://usegalaxy.org
9 years, 6 months
Re: [galaxy-dev] [galaxy-user] Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
by Ameena Nalim
Hi!
I am a new user. I have tried unsuccessfully to follow the steps in the
screencasts for Metagenome Analysis with Galaxy.
I have successfully uploaded the fasta file in .fas and .fna format from
454 sequencing. The dataset is some 150k plus sequences.
I am unable to create a quality score format to proceed.
Is this something to do with my fasta file?
My goal is to draw a phylogeny like that possible with MEGAN
Would you be able to suggest a workflow that will work?
I have added this email to share my history.
My public name is fanalim
the history is the only one saved
thanks
Ameena
> Hello Galaxy Community,
>
> The Galaxy Community Space has been renamed to be the Galaxy Tool Shed (
contributed by Dave Clements ), the location is still
> http://usegalaxy.org/community.
>
> The site now includes the ability to upload an archive consisting of a
suite of tools in addition to a single tool archive.
>
> Here is the information about uploading both of these types of archives.
Please start sharing tools, and let us know if you encounter any
problems.
> Also, let us know what new features you would like to see in the tool
> shed. Our next update will include the ability to rate tools.
>
>
> Creating an archive containing a tool or a suite of tools
> A tool or tool suite archive is a tar-format file (bzipped or gzipped
tar are valid) containing all the files necessary to load the tool(s)
into a Galaxy instance.
>
> Tool Suite Archive
>
> A tools suite must include a file named suite_config.xml which provides
information about the id, name, version and description of the tool
suite, as well as the id, name, version and description of each tool in
the suite. Here is an example suite_config.xml file.
>
>
> <suite id="lastz_toolsuite" name="Suite of Lastz tools"
> version="1.0.0">
> <description>This suite contains all of my Lastz tools for
> Galaxy</description>
> <tool id="lastz_wrapper_2" name="Lastz" version="1.1.0">
> <description> map short reads against reference
> sequence</description>
> </tool>
> <tool id="lastz_paired_reads_wrapper" name="Lastz paired reads"
> version="1.0.0">
> <description> map short paired reads against reference
> sequence</description>
> </tool>
> </suite>
>
>
> New versions of the suite can be uploaded, replacing an older version of
the suite, but the version attribute of the tag must be altered the same
way that the version attribute of a single tool config must be altered
if uploading a new version of a tool.
>
> The id, name and version attributes of each tag in the suite_config.xml
file must exactly match the same attributes in each associated tool
config in the archive or you will not be allowed to upload the archive.
>
> In addition to the suite_config.xml file, the archive must include all
tool config files, executables, functional test data (if your tool
config includes functional tests) and other files needed for each of the
tools in your suite to function within Galaxy. See the information about
single tool archives below for additional hints to enable ease-of-use
when others download your suite of tools.
>
> For example, to package the above Lastz suite of tools:
>
> user@host:~% tar jcvf ~/Desktop/galaxy_lastz_toolsuite.tar.bz2
> lastzsuite
> lastzsuite/
> lastzsuite/README
> lastzsuite/suite_config.xml
> lastzsuite/lastz_paired_reads_wrapper.py
> lastzsuite/lastz_paired_reads_wrapper.xml
> lastzsuite/lastz_wrapper.py
> lastzsuite/lastz_wrapper.xml
> lastzsuite/lastz-distrib-1.02.00/
> lastzsuite/lastz-distrib-1.02.00/src/
> lastzsuite/lastz-distrib-1.02.00/src/Makefile
> lastzsuite/lastz-distrib-1.02.00/src/version.mak
> lastzsuite/lastz-distrib-1.02.00/src/lastz.c
> lastzsuite/lastz-distrib-1.02.00/src/lastz.h
> ...
> ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded.
>
> Single Tool Archive
>
> A single tool archive must include a tool config file and will probably
also include a tool script. If any steps are necessary to install your
tool beyond the basic instructions below, include a README file to
provide details. If the tool (or parts of it) are written in C, the
source code can be included (or put links to the source in the README).
Do not include pre-compiled binaries without source since Galaxy is run
on a wide variety of platforms. Also, if you are only wrapping or
providing a Galaxy config for a tool that is not your own, be sure the
license allows for
> redistribution before including any part of that tool in the archive.
>
> There are no requirements about the directory structure inside the
archive, but for ease of use it's generally a good idea to put
everything inside a sub-directory, instead of directly at the top level.
>
> For example, to package the Lastz tool's config file, Galaxy wrapper,
and the C source:
>
> user@host:~% tar jcvf ~/Desktop/galaxy_lastz_tool.tar.bz2 lastz lastz/
> lastz/README
> lastz/lastz_wrapper.py
> lastz/lastz_wrapper.xml
> lastz/lastz-distrib-1.02.00/
> lastz/lastz-distrib-1.02.00/src/
> lastz/lastz-distrib-1.02.00/src/Makefile
> lastz/lastz-distrib-1.02.00/src/version.mak
> lastz/lastz-distrib-1.02.00/src/lastz.c
> lastz/lastz-distrib-1.02.00/src/lastz.h
> ...
> ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded.
>
> Editing Information, Categories, and Submitting For Approval
>
> Simply uploading a tool to the Galaxy too shed will not allow other
users to find and download your tool. It will need to be approved by an
administrator before it appears in the tool list.
>
> After your archive has successfully uploaded, you will be redirected to
the Edit Tool page. Provide a detailed description of what the tool does
- this will be used by administrators to understand the tool before
approving it for display on the site. Once approved, this information
will be displayed to users who view your tool. In addition, the site
> administrators will have configured a number of categories with which
you can associate your tool to make it easy to find by users looking to
solve specific problems. Associate as many categories as are relevant to
your tool. You may change the description and associated categories as
often as you'd like until you click the "Submit for approval" button.
Once submitted, the tool will be approved or rejected by an
administrator. If the tool is rejected, you will see information about
why it was rejected, and you can make appropriate changes to the archive
and re-submit it for approval. When it is approved, your archive will be
visible to everyone. At that point, the description and associated
categories can only be changed by an administrator.
>
> When the tool has been approved or rejected, you may upload a new
version by browsing to the tool's "View Tool" page, clicking the "Tool
actions" menu in the upper right corner of the page, and selecting
"Upload a new version" from the menu.
>
> Downloading and Installing Tools
>
> A tool's download link will send you the tool archive. Once downloaded,
unpack the tool on your local Galaxy instance's server:
>
> user@host:~% tar xvf galaxy_lastz_tool.tar
> ...
> user@host:~% tar zxvf galaxy_lastz_tool.tar.gz
> ...
> user@host:~% tar jxvf galaxy_lastz_tool.tar.bz2
> ...
> If the archive includes a README file, consult it for installation
instructions. If not, follow these basic steps:
> Create a directory under galaxy_dist/tools/ to house downloaded tool(s).
In the new directory, place the XML and any script file(s) which were
contained in the archive.
> If the tool includes binaries, you'll need to copy them to a directory
on your $PATH. If the tool depends on C binaries but does not come with
them (only source), you'll need to compile the source first.
> Add the tool to galaxy_dist/tool_conf.xml.
> Restart your Galaxy server process.
>
> In the near future, we plan to implement a more direct method to install
tools via the Galaxy administrator user interface instead of placing
files on the filesystem and manually managing the tool_conf.xml file.
>
>
>
> Greg Von Kuster
> Galaxy Development Team
> greg(a)bx.psu.edu
>
>
>
> _______________________________________________
> galaxy-user mailing list
> galaxy-user(a)lists.bx.psu.edu
> http://lists.bx.psu.edu/listinfo/galaxy-user
>
--
F. Ameena Nalim, Ph.D
Department of Plant & Environmental Sciences
Adj. Faculty, Molecular Biology Program
New Mexico State University
Las Cruces, NM 88003
e-mail:fanalim@nmsu.edu
http://aces.nmsu.edu/academics/adrianunc/labmembers.html
http://research.nmsu.edu/molbio/
--
F. Ameena Nalim, Ph.D
Department of Plant & Environmental Sciences
Adj. Faculty, Molecular Biology Program
New Mexico State University
Las Cruces, NM 88003
e-mail:fanalim@nmsu.edu
http://aces.nmsu.edu/academics/adrianunc/labmembers.html
http://research.nmsu.edu/molbio/
9 years, 6 months
Gnu Parallel
by Jesse Erdmann
http://savannah.gnu.org/projects/parallel/
Has anyone considered using something like Parallel under the covers
in Galaxy for simple parallel processing like Bowtie. Essentially it
works like xargs but can execute across multiple hosts and cores. I
thought I would check with the community before spending any serious
time evaluating it. Thanks!
--
Jesse Erdmann
Bioinformatics Analyst
Masonic Cancer Center
University of Minnesota
jerdmann(a)umn.edu
612-626-3123
jesse(a)jesseerdmann.com
Twitter: http://twitter.com/jesseerdmann
9 years, 6 months
Galaxy NGS Tools
by Jacob Biesinger
Hi!
In our lab, we've worked on several ChipSeq projects and have developed
dozens of scripts for analyzing the results, including running several peak
finders, several motif discovery tools, various data munging techniques,
some parameter optimization for the above programs, calculating the genomic
distribution of peaks, generating several summary graphs, calculating motif
distributions within peaks, performing gene ontology analysis, etc.
We've been thinking about making all of this into a standalone tool,
possibly a web service, and have been considering Galaxy as a vehicle for
automating the entire process and opening up the tools to a biologist
community. From what I've seen in Galaxy Main and the recent inclusion of
e.g., the MACS wrapper, it seems like the things I've listed would be of
interest to the galaxy community at large.
So I'm looking for feedback and possibly advice. Ideally, we'd like to be
able to run the entire pipeline, look at the results, possibly change a few
parameters in some of the steps (e.g., minimum FDR cutoff) and rerun only
what needs to be rerun. Galaxy workflows are easy to create, but don't seem
to have the flexibility that we're looking for. Perhaps several workflows
tied together would do the job (i.e., have separate workflows for the major
parts of the analysis) which we could tie together (possible in galaxy?)
into one uber-pipeline.
Has work like this already been done? Are there sample workflows that go
beyond just calling peaks? Would the community be interested in the code +
wrappers?
Thanks for the help!
--
Jake Biesinger
Graduate Student
Xie Lab, UC Irvine
(949) 231-7587
9 years, 6 months
Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
by Greg Von Kuster
Hello Galaxy Community,
The Galaxy Community Space has been renamed to be the Galaxy Tool Shed ( contributed by Dave Clements ), the location is still http://usegalaxy.org/community.
The site now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive.
Here is the information about uploading both of these types of archives. Please start sharing tools, and let us know if you encounter any problems. Also, let us know what new features you would like to see in the tool shed. Our next update will include the ability to rate tools.
Creating an archive containing a tool or a suite of tools
A tool or tool suite archive is a tar-format file (bzipped or gzipped tar are valid) containing all the files necessary to load the tool(s) into a Galaxy instance.
Tool Suite Archive
A tools suite must include a file named suite_config.xml which provides information about the id, name, version and description of the tool suite, as well as the id, name, version and description of each tool in the suite. Here is an example suite_config.xml file.
<suite id="lastz_toolsuite" name="Suite of Lastz tools" version="1.0.0">
<description>This suite contains all of my Lastz tools for Galaxy</description>
<tool id="lastz_wrapper_2" name="Lastz" version="1.1.0">
<description> map short reads against reference sequence</description>
</tool>
<tool id="lastz_paired_reads_wrapper" name="Lastz paired reads" version="1.0.0">
<description> map short paired reads against reference sequence</description>
</tool>
</suite>
New versions of the suite can be uploaded, replacing an older version of the suite, but the version attribute of the tag must be altered the same way that the version attribute of a single tool config must be altered if uploading a new version of a tool.
The id, name and version attributes of each tag in the suite_config.xml file must exactly match the same attributes in each associated tool config in the archive or you will not be allowed to upload the archive.
In addition to the suite_config.xml file, the archive must include all tool config files, executables, functional test data (if your tool config includes functional tests) and other files needed for each of the tools in your suite to function within Galaxy. See the information about single tool archives below for additional hints to enable ease-of-use when others download your suite of tools.
For example, to package the above Lastz suite of tools:
user@host:~% tar jcvf ~/Desktop/galaxy_lastz_toolsuite.tar.bz2 lastzsuite
lastzsuite/
lastzsuite/README
lastzsuite/suite_config.xml
lastzsuite/lastz_paired_reads_wrapper.py
lastzsuite/lastz_paired_reads_wrapper.xml
lastzsuite/lastz_wrapper.py
lastzsuite/lastz_wrapper.xml
lastzsuite/lastz-distrib-1.02.00/
lastzsuite/lastz-distrib-1.02.00/src/
lastzsuite/lastz-distrib-1.02.00/src/Makefile
lastzsuite/lastz-distrib-1.02.00/src/version.mak
lastzsuite/lastz-distrib-1.02.00/src/lastz.c
lastzsuite/lastz-distrib-1.02.00/src/lastz.h
...
~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded.
Single Tool Archive
A single tool archive must include a tool config file and will probably also include a tool script. If any steps are necessary to install your tool beyond the basic instructions below, include a README file to provide details. If the tool (or parts of it) are written in C, the source code can be included (or put links to the source in the README). Do not include pre-compiled binaries without source since Galaxy is run on a wide variety of platforms. Also, if you are only wrapping or providing a Galaxy config for a tool that is not your own, be sure the license allows for redistribution before including any part of that tool in the archive.
There are no requirements about the directory structure inside the archive, but for ease of use it's generally a good idea to put everything inside a sub-directory, instead of directly at the top level.
For example, to package the Lastz tool's config file, Galaxy wrapper, and the C source:
user@host:~% tar jcvf ~/Desktop/galaxy_lastz_tool.tar.bz2 lastz
lastz/
lastz/README
lastz/lastz_wrapper.py
lastz/lastz_wrapper.xml
lastz/lastz-distrib-1.02.00/
lastz/lastz-distrib-1.02.00/src/
lastz/lastz-distrib-1.02.00/src/Makefile
lastz/lastz-distrib-1.02.00/src/version.mak
lastz/lastz-distrib-1.02.00/src/lastz.c
lastz/lastz-distrib-1.02.00/src/lastz.h
...
~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded.
Editing Information, Categories, and Submitting For Approval
Simply uploading a tool to the Galaxy too shed will not allow other users to find and download your tool. It will need to be approved by an administrator before it appears in the tool list.
After your archive has successfully uploaded, you will be redirected to the Edit Tool page. Provide a detailed description of what the tool does - this will be used by administrators to understand the tool before approving it for display on the site. Once approved, this information will be displayed to users who view your tool. In addition, the site administrators will have configured a number of categories with which you can associate your tool to make it easy to find by users looking to solve specific problems. Associate as many categories as are relevant to your tool. You may change the description and associated categories as often as you'd like until you click the "Submit for approval" button. Once submitted, the tool will be approved or rejected by an administrator. If the tool is rejected, you will see information about why it was rejected, and you can make appropriate changes to the archive and re-submit it for approval. When it is approved, your archive will be visible to everyone. At that point, the description and associated categories can only be changed by an administrator.
When the tool has been approved or rejected, you may upload a new version by browsing to the tool's "View Tool" page, clicking the "Tool actions" menu in the upper right corner of the page, and selecting "Upload a new version" from the menu.
Downloading and Installing Tools
A tool's download link will send you the tool archive. Once downloaded, unpack the tool on your local Galaxy instance's server:
user@host:~% tar xvf galaxy_lastz_tool.tar
...
user@host:~% tar zxvf galaxy_lastz_tool.tar.gz
...
user@host:~% tar jxvf galaxy_lastz_tool.tar.bz2
...
If the archive includes a README file, consult it for installation instructions. If not, follow these basic steps:
Create a directory under galaxy_dist/tools/ to house downloaded tool(s).
In the new directory, place the XML and any script file(s) which were contained in the archive.
If the tool includes binaries, you'll need to copy them to a directory on your $PATH. If the tool depends on C binaries but does not come with them (only source), you'll need to compile the source first.
Add the tool to galaxy_dist/tool_conf.xml.
Restart your Galaxy server process.
In the near future, we plan to implement a more direct method to install tools via the Galaxy administrator user interface instead of placing files on the filesystem and manually managing the tool_conf.xml file.
Greg Von Kuster
Galaxy Development Team
greg(a)bx.psu.edu
9 years, 6 months
Show STDOUT in reports page
by Assaf Gordon
Hi,
May I ask to add STDOUT to the job-info page of the very cool new reports webapp ?
Attached a patch for that.
Thanks,
-gordon
9 years, 6 months
