July 2010 - galaxy-dev - lists.galaxyproject.org

July 26, 2010 Galaxy Development News Brief
by Greg Von Kuster 26 Jul '10

26 Jul '10

The latest news brief is available at the following link. http://bitbucket.org/galaxy/galaxy-central/wiki/Features/DevNewsBrief/2010_… Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

How to add megablast database?
by Kelly Vincent 26 Jul '10

26 Jul '10

Begin forwarded message: > From: Kelly Vincent <kpvincent(a)bx.psu.edu> > Date: July 26, 2010 11:23:47 AM EDT > To: buiduyminh(a)gmail.com > Cc: Galaxy Lab <galaxy-lab(a)bx.psu.edu> > Subject: Re: [galaxy-lab] [galaxy-dev] How to add megablast database? > > Hi, > > See the wiki page http://bitbucket.org/galaxy/galaxy-central/wiki/NGSLocalSetup > for information on getting the database files in place. > > If you want to use blastn instead of megablast, you would just need > to modify the wrapper slightly. You will at minimum need to modify > tools/metag_tools/megablast_wrapper.py, by changing the command-line > call (line 72) appropriately. If you need only the same parameters > as megablast uses, you can just change the python file that is > called in the command tag in tools/metag_tools/ > megablast_wrapper.xml. Otherwise, you'll need to add relevant > parameters to the XML and python files. > > Let us know if you need any further assistance. > > Regards, > Kelly > > > > > > > On Jul 24, 2010, at 10:56 AM, Minh Bui wrote: > >> Hi, >> >> I am using local Galaxy on my computer. I would like to use >> megablast but I don't know how to add wgs and nt database to local >> galaxy (I am new to galaxy). >> >> 1. Where can I download those databases (if you dont mind, can you >> give me the link?) ? >> 2. How can I add those databases to galaxy? >> 3. What if I want to do BLASTN instead of megablast, how can I do >> that? >> >> Thank you very much, >> _______________________________________________ >> galaxy-dev mailing list >> galaxy-dev(a)lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-dev > > _______________________________________________ > galaxy-lab mailing list > galaxy-lab(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-lab

1 0

How to add megablast database?
by Minh Bui 24 Jul '10

24 Jul '10

Hi, I am using local Galaxy on my computer. I would like to use megablast but I don't know how to add wgs and nt database to local galaxy (I am new to galaxy). 1. Where can I download those databases (if you dont mind, can you give me the link?) ? 2. How can I add those databases to galaxy? 3. What if I want to do BLASTN instead of megablast, how can I do that? Thank you very much,

1 0

Huge memory usage in latest Galaxy
by Assaf Gordon 23 Jul '10

23 Jul '10

Hello all, After upgrading to a recent version (47411bb09df4), memory usage reaches 12GB per python process after less then 48 hours (with normal usage by users). It only happens with the web-side processes, the job-runner process and the reports process are at their usual usage level. Debug is off and Interactive is off, The major change is that I've started using SGE (with DRMAA). Any tips would be very appreciated. -gordon

2 3

Re: [galaxy-dev] [galaxy-bugs] Fwd: [galaxy-user] Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
by Anton Nekrutenko 23 Jul '10

23 Jul '10

Ameena: 454 gives two types of files - reads in fasta format: >FWV11AZ02JY4XI length=110 xy=3973_2596 region=2 run=R_2009_06_02_15_16_12_TTCATGGAGAAGGCTTAGTGCTCTTTACACAGCCACTAACAACAGGTGAGGTGCTTATAAGTCTCAATCAGAAACAGAGTCTTACCCTTCGTTATTCTAAACAAACTACT >FWV11AZ02G3SEQ length=134 xy=2796_1792 region=2 run=R_2009_06_02_15_16_12_TTCTATTTAAGGTGCACATGGTTGAGCTCTAGGGCTTTTTAAGACACTGTGAATCTCAAATTATTTGCATGACTTGAAAGGGCACTTTGAATTAGTGAAAAGTCTTGGGAATACCAAAAAAATTTTTAATTTTT and associated quality scores: >FWV11AZ02JY4XI length=110 xy=3973_2596 region=2 run=R_2009_06_02_15_16_12_37 37 37 37 37 37 37 37 37 37 37 40 40 40 40 40 40 40 40 40 40 40 40 39 39 39 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 37 37 37 37 37 37 37 37 37 37 35 33 33 33 32 31 31 24 24 15 15 15 15 15 29 30 28 27 27 26 26 30 23 23 23 18 17 14 19 13 13 14 17 17 19 16 16 17 16 20 16 16 16 17 14 12 14 14 >FWV11AZ02G3SEQ length=134 xy=2796_1792 region=2 run=R_2009_06_02_15_16_12_37 37 37 37 37 37 37 37 37 37 37 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 31 31 31 40 28 28 28 28 28 40 40 40 40 40 40 40 40 40 40 40 40 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 37 30 30 30 37 37 37 37 37 37 37 37 37 36 36 36 36 37 37 37 37 37 37 37 37 37 37 37 35 35 34 16 16 17 17 17 14 15 15 15 12 12 12 10 10 11 11 11 11 11 In the case of metagenomic analysis you only need quality scores if you will be doing selection of high quality segments from your reads, which you can skip if you do not have quality files handy. However, before you begin make sure you understand what is involved in this analysis. A good place to start would a galaxy page for our Genome Research publication: http://main.g2.bx.psu.edu/u/aun1/p/windshield-splatter Thanks, anton galaxy team On Jul 23, 2010, at 12:44 PM, Anton Nekrutenko wrote: > Ameena: > > I am forwarding this to galaxy-bugs mailing list, which more appropriate for your question. > > anton > galaxy team > > Begin forwarded message: > >> From: "Ameena Nalim" <fanalim(a)nmsu.edu> >> Date: July 23, 2010 11:30:48 AM EDT >> To: galaxy-dev(a)bx.psu.edu >> Subject: Re: [galaxy-dev] [galaxy-user] Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive >> >> >> Hi! >> I am a new user. I have tried unsuccessfully to follow the steps in the >> screencasts for Metagenome Analysis with Galaxy. >> I have successfully uploaded the fasta file in .fas and .fna format from >> 454 sequencing. The dataset is some 150k plus sequences. >> I am unable to create a quality score format to proceed. >> Is this something to do with my fasta file? >> >> My goal is to draw a phylogeny like that possible with MEGAN >> Would you be able to suggest a workflow that will work? >> I have added this email to share my history. >> My public name is fanalim >> the history is the only one saved >> >> thanks >> Ameena >> >> >>> Hello Galaxy Community, >>> >>> The Galaxy Community Space has been renamed to be the Galaxy Tool Shed ( >> contributed by Dave Clements ), the location is still >>> http://usegalaxy.org/community. >>> >>> The site now includes the ability to upload an archive consisting of a >> suite of tools in addition to a single tool archive. >>> >>> Here is the information about uploading both of these types of archives. >> Please start sharing tools, and let us know if you encounter any >> problems. >>> Also, let us know what new features you would like to see in the tool >>> shed. Our next update will include the ability to rate tools. >>> >>> >>> Creating an archive containing a tool or a suite of tools >>> A tool or tool suite archive is a tar-format file (bzipped or gzipped >> tar are valid) containing all the files necessary to load the tool(s) >> into a Galaxy instance. >>> >>> Tool Suite Archive >>> >>> A tools suite must include a file named suite_config.xml which provides >> information about the id, name, version and description of the tool >> suite, as well as the id, name, version and description of each tool in >> the suite. Here is an example suite_config.xml file. >>> >>> >>> <suite id="lastz_toolsuite" name="Suite of Lastz tools" >>> version="1.0.0"> >>> <description>This suite contains all of my Lastz tools for >>> Galaxy</description> >>> <tool id="lastz_wrapper_2" name="Lastz" version="1.1.0"> >>> <description> map short reads against reference >>> sequence</description> >>> </tool> >>> <tool id="lastz_paired_reads_wrapper" name="Lastz paired reads" >>> version="1.0.0"> >>> <description> map short paired reads against reference >>> sequence</description> >>> </tool> >>> </suite> >>> >>> >>> New versions of the suite can be uploaded, replacing an older version of >> the suite, but the version attribute of the tag must be altered the same >> way that the version attribute of a single tool config must be altered >> if uploading a new version of a tool. >>> >>> The id, name and version attributes of each tag in the suite_config.xml >> file must exactly match the same attributes in each associated tool >> config in the archive or you will not be allowed to upload the archive. >>> >>> In addition to the suite_config.xml file, the archive must include all >> tool config files, executables, functional test data (if your tool >> config includes functional tests) and other files needed for each of the >> tools in your suite to function within Galaxy. See the information about >> single tool archives below for additional hints to enable ease-of-use >> when others download your suite of tools. >>> >>> For example, to package the above Lastz suite of tools: >>> >>> user@host:~% tar jcvf ~/Desktop/galaxy_lastz_toolsuite.tar.bz2 >>> lastzsuite >>> lastzsuite/ >>> lastzsuite/README >>> lastzsuite/suite_config.xml >>> lastzsuite/lastz_paired_reads_wrapper.py >>> lastzsuite/lastz_paired_reads_wrapper.xml >>> lastzsuite/lastz_wrapper.py >>> lastzsuite/lastz_wrapper.xml >>> lastzsuite/lastz-distrib-1.02.00/ >>> lastzsuite/lastz-distrib-1.02.00/src/ >>> lastzsuite/lastz-distrib-1.02.00/src/Makefile >>> lastzsuite/lastz-distrib-1.02.00/src/version.mak >>> lastzsuite/lastz-distrib-1.02.00/src/lastz.c >>> lastzsuite/lastz-distrib-1.02.00/src/lastz.h >>> ... >>> ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded. >>> >>> Single Tool Archive >>> >>> A single tool archive must include a tool config file and will probably >> also include a tool script. If any steps are necessary to install your >> tool beyond the basic instructions below, include a README file to >> provide details. If the tool (or parts of it) are written in C, the >> source code can be included (or put links to the source in the README). >> Do not include pre-compiled binaries without source since Galaxy is run >> on a wide variety of platforms. Also, if you are only wrapping or >> providing a Galaxy config for a tool that is not your own, be sure the >> license allows for >>> redistribution before including any part of that tool in the archive. >>> >>> There are no requirements about the directory structure inside the >> archive, but for ease of use it's generally a good idea to put >> everything inside a sub-directory, instead of directly at the top level. >>> >>> For example, to package the Lastz tool's config file, Galaxy wrapper, >> and the C source: >>> >>> user@host:~% tar jcvf ~/Desktop/galaxy_lastz_tool.tar.bz2 lastz lastz/ >>> lastz/README >>> lastz/lastz_wrapper.py >>> lastz/lastz_wrapper.xml >>> lastz/lastz-distrib-1.02.00/ >>> lastz/lastz-distrib-1.02.00/src/ >>> lastz/lastz-distrib-1.02.00/src/Makefile >>> lastz/lastz-distrib-1.02.00/src/version.mak >>> lastz/lastz-distrib-1.02.00/src/lastz.c >>> lastz/lastz-distrib-1.02.00/src/lastz.h >>> ... >>> ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded. >>> >>> Editing Information, Categories, and Submitting For Approval >>> >>> Simply uploading a tool to the Galaxy too shed will not allow other >> users to find and download your tool. It will need to be approved by an >> administrator before it appears in the tool list. >>> >>> After your archive has successfully uploaded, you will be redirected to >> the Edit Tool page. Provide a detailed description of what the tool does >> - this will be used by administrators to understand the tool before >> approving it for display on the site. Once approved, this information >> will be displayed to users who view your tool. In addition, the site >>> administrators will have configured a number of categories with which >> you can associate your tool to make it easy to find by users looking to >> solve specific problems. Associate as many categories as are relevant to >> your tool. You may change the description and associated categories as >> often as you'd like until you click the "Submit for approval" button. >> Once submitted, the tool will be approved or rejected by an >> administrator. If the tool is rejected, you will see information about >> why it was rejected, and you can make appropriate changes to the archive >> and re-submit it for approval. When it is approved, your archive will be >> visible to everyone. At that point, the description and associated >> categories can only be changed by an administrator. >>> >>> When the tool has been approved or rejected, you may upload a new >> version by browsing to the tool's "View Tool" page, clicking the "Tool >> actions" menu in the upper right corner of the page, and selecting >> "Upload a new version" from the menu. >>> >>> Downloading and Installing Tools >>> >>> A tool's download link will send you the tool archive. Once downloaded, >> unpack the tool on your local Galaxy instance's server: >>> >>> user@host:~% tar xvf galaxy_lastz_tool.tar >>> ... >>> user@host:~% tar zxvf galaxy_lastz_tool.tar.gz >>> ... >>> user@host:~% tar jxvf galaxy_lastz_tool.tar.bz2 >>> ... >>> If the archive includes a README file, consult it for installation >> instructions. If not, follow these basic steps: >>> Create a directory under galaxy_dist/tools/ to house downloaded tool(s). >> In the new directory, place the XML and any script file(s) which were >> contained in the archive. >>> If the tool includes binaries, you'll need to copy them to a directory >> on your $PATH. If the tool depends on C binaries but does not come with >> them (only source), you'll need to compile the source first. >>> Add the tool to galaxy_dist/tool_conf.xml. >>> Restart your Galaxy server process. >>> >>> In the near future, we plan to implement a more direct method to install >> tools via the Galaxy administrator user interface instead of placing >> files on the filesystem and manually managing the tool_conf.xml file. >>> >>> >>> >>> Greg Von Kuster >>> Galaxy Development Team >>> greg(a)bx.psu.edu >>> >>> >>> >>> _______________________________________________ >>> galaxy-user mailing list >>> galaxy-user(a)lists.bx.psu.edu >>> http://lists.bx.psu.edu/listinfo/galaxy-user >>> >> >> >> -- >> F. Ameena Nalim, Ph.D >> Department of Plant & Environmental Sciences >> Adj. Faculty, Molecular Biology Program >> New Mexico State University >> Las Cruces, NM 88003 >> e-mail:fanalim@nmsu.edu >> http://aces.nmsu.edu/academics/adrianunc/labmembers.html >> http://research.nmsu.edu/molbio/ >> >> >> >> -- >> F. Ameena Nalim, Ph.D >> Department of Plant & Environmental Sciences >> Adj. Faculty, Molecular Biology Program >> New Mexico State University >> Las Cruces, NM 88003 >> e-mail:fanalim@nmsu.edu >> http://aces.nmsu.edu/academics/adrianunc/labmembers.html >> http://research.nmsu.edu/molbio/ >> >> _______________________________________________ >> galaxy-dev mailing list >> galaxy-dev(a)lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-dev > > Anton Nekrutenko > http://nekrut.bx.psu.edu > http://usegalaxy.org > > > > _______________________________________________ > galaxy-bugs mailing list > galaxy-bugs(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-bugs Anton Nekrutenko http://nekrut.bx.psu.edu http://usegalaxy.org

1 0

Re: [galaxy-dev] [galaxy-user] Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
by Ameena Nalim 23 Jul '10

23 Jul '10

Hi! I am a new user. I have tried unsuccessfully to follow the steps in the screencasts for Metagenome Analysis with Galaxy. I have successfully uploaded the fasta file in .fas and .fna format from 454 sequencing. The dataset is some 150k plus sequences. I am unable to create a quality score format to proceed. Is this something to do with my fasta file? My goal is to draw a phylogeny like that possible with MEGAN Would you be able to suggest a workflow that will work? I have added this email to share my history. My public name is fanalim the history is the only one saved thanks Ameena > Hello Galaxy Community, > > The Galaxy Community Space has been renamed to be the Galaxy Tool Shed ( contributed by Dave Clements ), the location is still > http://usegalaxy.org/community. > > The site now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive. > > Here is the information about uploading both of these types of archives. Please start sharing tools, and let us know if you encounter any problems. > Also, let us know what new features you would like to see in the tool > shed. Our next update will include the ability to rate tools. > > > Creating an archive containing a tool or a suite of tools > A tool or tool suite archive is a tar-format file (bzipped or gzipped tar are valid) containing all the files necessary to load the tool(s) into a Galaxy instance. > > Tool Suite Archive > > A tools suite must include a file named suite_config.xml which provides information about the id, name, version and description of the tool suite, as well as the id, name, version and description of each tool in the suite. Here is an example suite_config.xml file. > > > <suite id="lastz_toolsuite" name="Suite of Lastz tools" > version="1.0.0"> > <description>This suite contains all of my Lastz tools for > Galaxy</description> > <tool id="lastz_wrapper_2" name="Lastz" version="1.1.0"> > <description> map short reads against reference > sequence</description> > </tool> > <tool id="lastz_paired_reads_wrapper" name="Lastz paired reads" > version="1.0.0"> > <description> map short paired reads against reference > sequence</description> > </tool> > </suite> > > > New versions of the suite can be uploaded, replacing an older version of the suite, but the version attribute of the tag must be altered the same way that the version attribute of a single tool config must be altered if uploading a new version of a tool. > > The id, name and version attributes of each tag in the suite_config.xml file must exactly match the same attributes in each associated tool config in the archive or you will not be allowed to upload the archive. > > In addition to the suite_config.xml file, the archive must include all tool config files, executables, functional test data (if your tool config includes functional tests) and other files needed for each of the tools in your suite to function within Galaxy. See the information about single tool archives below for additional hints to enable ease-of-use when others download your suite of tools. > > For example, to package the above Lastz suite of tools: > > user@host:~% tar jcvf ~/Desktop/galaxy_lastz_toolsuite.tar.bz2 > lastzsuite > lastzsuite/ > lastzsuite/README > lastzsuite/suite_config.xml > lastzsuite/lastz_paired_reads_wrapper.py > lastzsuite/lastz_paired_reads_wrapper.xml > lastzsuite/lastz_wrapper.py > lastzsuite/lastz_wrapper.xml > lastzsuite/lastz-distrib-1.02.00/ > lastzsuite/lastz-distrib-1.02.00/src/ > lastzsuite/lastz-distrib-1.02.00/src/Makefile > lastzsuite/lastz-distrib-1.02.00/src/version.mak > lastzsuite/lastz-distrib-1.02.00/src/lastz.c > lastzsuite/lastz-distrib-1.02.00/src/lastz.h > ... > ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded. > > Single Tool Archive > > A single tool archive must include a tool config file and will probably also include a tool script. If any steps are necessary to install your tool beyond the basic instructions below, include a README file to provide details. If the tool (or parts of it) are written in C, the source code can be included (or put links to the source in the README). Do not include pre-compiled binaries without source since Galaxy is run on a wide variety of platforms. Also, if you are only wrapping or providing a Galaxy config for a tool that is not your own, be sure the license allows for > redistribution before including any part of that tool in the archive. > > There are no requirements about the directory structure inside the archive, but for ease of use it's generally a good idea to put everything inside a sub-directory, instead of directly at the top level. > > For example, to package the Lastz tool's config file, Galaxy wrapper, and the C source: > > user@host:~% tar jcvf ~/Desktop/galaxy_lastz_tool.tar.bz2 lastz lastz/ > lastz/README > lastz/lastz_wrapper.py > lastz/lastz_wrapper.xml > lastz/lastz-distrib-1.02.00/ > lastz/lastz-distrib-1.02.00/src/ > lastz/lastz-distrib-1.02.00/src/Makefile > lastz/lastz-distrib-1.02.00/src/version.mak > lastz/lastz-distrib-1.02.00/src/lastz.c > lastz/lastz-distrib-1.02.00/src/lastz.h > ... > ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded. > > Editing Information, Categories, and Submitting For Approval > > Simply uploading a tool to the Galaxy too shed will not allow other users to find and download your tool. It will need to be approved by an administrator before it appears in the tool list. > > After your archive has successfully uploaded, you will be redirected to the Edit Tool page. Provide a detailed description of what the tool does - this will be used by administrators to understand the tool before approving it for display on the site. Once approved, this information will be displayed to users who view your tool. In addition, the site > administrators will have configured a number of categories with which you can associate your tool to make it easy to find by users looking to solve specific problems. Associate as many categories as are relevant to your tool. You may change the description and associated categories as often as you'd like until you click the "Submit for approval" button. Once submitted, the tool will be approved or rejected by an administrator. If the tool is rejected, you will see information about why it was rejected, and you can make appropriate changes to the archive and re-submit it for approval. When it is approved, your archive will be visible to everyone. At that point, the description and associated categories can only be changed by an administrator. > > When the tool has been approved or rejected, you may upload a new version by browsing to the tool's "View Tool" page, clicking the "Tool actions" menu in the upper right corner of the page, and selecting "Upload a new version" from the menu. > > Downloading and Installing Tools > > A tool's download link will send you the tool archive. Once downloaded, unpack the tool on your local Galaxy instance's server: > > user@host:~% tar xvf galaxy_lastz_tool.tar > ... > user@host:~% tar zxvf galaxy_lastz_tool.tar.gz > ... > user@host:~% tar jxvf galaxy_lastz_tool.tar.bz2 > ... > If the archive includes a README file, consult it for installation instructions. If not, follow these basic steps: > Create a directory under galaxy_dist/tools/ to house downloaded tool(s). In the new directory, place the XML and any script file(s) which were contained in the archive. > If the tool includes binaries, you'll need to copy them to a directory on your $PATH. If the tool depends on C binaries but does not come with them (only source), you'll need to compile the source first. > Add the tool to galaxy_dist/tool_conf.xml. > Restart your Galaxy server process. > > In the near future, we plan to implement a more direct method to install tools via the Galaxy administrator user interface instead of placing files on the filesystem and manually managing the tool_conf.xml file. > > > > Greg Von Kuster > Galaxy Development Team > greg(a)bx.psu.edu > > > > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user > -- F. Ameena Nalim, Ph.D Department of Plant & Environmental Sciences Adj. Faculty, Molecular Biology Program New Mexico State University Las Cruces, NM 88003 e-mail:fanalim@nmsu.edu http://aces.nmsu.edu/academics/adrianunc/labmembers.html http://research.nmsu.edu/molbio/ -- F. Ameena Nalim, Ph.D Department of Plant & Environmental Sciences Adj. Faculty, Molecular Biology Program New Mexico State University Las Cruces, NM 88003 e-mail:fanalim@nmsu.edu http://aces.nmsu.edu/academics/adrianunc/labmembers.html http://research.nmsu.edu/molbio/

1 0

Gnu Parallel
by Jesse Erdmann 23 Jul '10

23 Jul '10

http://savannah.gnu.org/projects/parallel/ Has anyone considered using something like Parallel under the covers in Galaxy for simple parallel processing like Bowtie. Essentially it works like xargs but can execute across multiple hosts and cores. I thought I would check with the community before spending any serious time evaluating it. Thanks! -- Jesse Erdmann Bioinformatics Analyst Masonic Cancer Center University of Minnesota jerdmann(a)umn.edu 612-626-3123 jesse(a)jesseerdmann.com Twitter: http://twitter.com/jesseerdmann

1 0

Galaxy NGS Tools
by Jacob Biesinger 22 Jul '10

22 Jul '10

Hi! In our lab, we've worked on several ChipSeq projects and have developed dozens of scripts for analyzing the results, including running several peak finders, several motif discovery tools, various data munging techniques, some parameter optimization for the above programs, calculating the genomic distribution of peaks, generating several summary graphs, calculating motif distributions within peaks, performing gene ontology analysis, etc. We've been thinking about making all of this into a standalone tool, possibly a web service, and have been considering Galaxy as a vehicle for automating the entire process and opening up the tools to a biologist community. From what I've seen in Galaxy Main and the recent inclusion of e.g., the MACS wrapper, it seems like the things I've listed would be of interest to the galaxy community at large. So I'm looking for feedback and possibly advice. Ideally, we'd like to be able to run the entire pipeline, look at the results, possibly change a few parameters in some of the steps (e.g., minimum FDR cutoff) and rerun only what needs to be rerun. Galaxy workflows are easy to create, but don't seem to have the flexibility that we're looking for. Perhaps several workflows tied together would do the job (i.e., have separate workflows for the major parts of the analysis) which we could tie together (possible in galaxy?) into one uber-pipeline. Has work like this already been done? Are there sample workflows that go beyond just calling peaks? Would the community be interested in the code + wrappers? Thanks for the help! -- Jake Biesinger Graduate Student Xie Lab, UC Irvine (949) 231-7587

5 5

Galaxy Tool Shed now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive
by Greg Von Kuster 22 Jul '10

22 Jul '10

Hello Galaxy Community, The Galaxy Community Space has been renamed to be the Galaxy Tool Shed ( contributed by Dave Clements ), the location is still http://usegalaxy.org/community. The site now includes the ability to upload an archive consisting of a suite of tools in addition to a single tool archive. Here is the information about uploading both of these types of archives. Please start sharing tools, and let us know if you encounter any problems. Also, let us know what new features you would like to see in the tool shed. Our next update will include the ability to rate tools. Creating an archive containing a tool or a suite of tools A tool or tool suite archive is a tar-format file (bzipped or gzipped tar are valid) containing all the files necessary to load the tool(s) into a Galaxy instance. Tool Suite Archive A tools suite must include a file named suite_config.xml which provides information about the id, name, version and description of the tool suite, as well as the id, name, version and description of each tool in the suite. Here is an example suite_config.xml file. <suite id="lastz_toolsuite" name="Suite of Lastz tools" version="1.0.0"> <description>This suite contains all of my Lastz tools for Galaxy</description> <tool id="lastz_wrapper_2" name="Lastz" version="1.1.0"> <description> map short reads against reference sequence</description> </tool> <tool id="lastz_paired_reads_wrapper" name="Lastz paired reads" version="1.0.0"> <description> map short paired reads against reference sequence</description> </tool> </suite> New versions of the suite can be uploaded, replacing an older version of the suite, but the version attribute of the tag must be altered the same way that the version attribute of a single tool config must be altered if uploading a new version of a tool. The id, name and version attributes of each tag in the suite_config.xml file must exactly match the same attributes in each associated tool config in the archive or you will not be allowed to upload the archive. In addition to the suite_config.xml file, the archive must include all tool config files, executables, functional test data (if your tool config includes functional tests) and other files needed for each of the tools in your suite to function within Galaxy. See the information about single tool archives below for additional hints to enable ease-of-use when others download your suite of tools. For example, to package the above Lastz suite of tools: user@host:~% tar jcvf ~/Desktop/galaxy_lastz_toolsuite.tar.bz2 lastzsuite lastzsuite/ lastzsuite/README lastzsuite/suite_config.xml lastzsuite/lastz_paired_reads_wrapper.py lastzsuite/lastz_paired_reads_wrapper.xml lastzsuite/lastz_wrapper.py lastzsuite/lastz_wrapper.xml lastzsuite/lastz-distrib-1.02.00/ lastzsuite/lastz-distrib-1.02.00/src/ lastzsuite/lastz-distrib-1.02.00/src/Makefile lastzsuite/lastz-distrib-1.02.00/src/version.mak lastzsuite/lastz-distrib-1.02.00/src/lastz.c lastzsuite/lastz-distrib-1.02.00/src/lastz.h ... ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded. Single Tool Archive A single tool archive must include a tool config file and will probably also include a tool script. If any steps are necessary to install your tool beyond the basic instructions below, include a README file to provide details. If the tool (or parts of it) are written in C, the source code can be included (or put links to the source in the README). Do not include pre-compiled binaries without source since Galaxy is run on a wide variety of platforms. Also, if you are only wrapping or providing a Galaxy config for a tool that is not your own, be sure the license allows for redistribution before including any part of that tool in the archive. There are no requirements about the directory structure inside the archive, but for ease of use it's generally a good idea to put everything inside a sub-directory, instead of directly at the top level. For example, to package the Lastz tool's config file, Galaxy wrapper, and the C source: user@host:~% tar jcvf ~/Desktop/galaxy_lastz_tool.tar.bz2 lastz lastz/ lastz/README lastz/lastz_wrapper.py lastz/lastz_wrapper.xml lastz/lastz-distrib-1.02.00/ lastz/lastz-distrib-1.02.00/src/ lastz/lastz-distrib-1.02.00/src/Makefile lastz/lastz-distrib-1.02.00/src/version.mak lastz/lastz-distrib-1.02.00/src/lastz.c lastz/lastz-distrib-1.02.00/src/lastz.h ... ~/Desktop/galaxy_lastz_tool.tar.bz2 is now ready to be uploaded. Editing Information, Categories, and Submitting For Approval Simply uploading a tool to the Galaxy too shed will not allow other users to find and download your tool. It will need to be approved by an administrator before it appears in the tool list. After your archive has successfully uploaded, you will be redirected to the Edit Tool page. Provide a detailed description of what the tool does - this will be used by administrators to understand the tool before approving it for display on the site. Once approved, this information will be displayed to users who view your tool. In addition, the site administrators will have configured a number of categories with which you can associate your tool to make it easy to find by users looking to solve specific problems. Associate as many categories as are relevant to your tool. You may change the description and associated categories as often as you'd like until you click the "Submit for approval" button. Once submitted, the tool will be approved or rejected by an administrator. If the tool is rejected, you will see information about why it was rejected, and you can make appropriate changes to the archive and re-submit it for approval. When it is approved, your archive will be visible to everyone. At that point, the description and associated categories can only be changed by an administrator. When the tool has been approved or rejected, you may upload a new version by browsing to the tool's "View Tool" page, clicking the "Tool actions" menu in the upper right corner of the page, and selecting "Upload a new version" from the menu. Downloading and Installing Tools A tool's download link will send you the tool archive. Once downloaded, unpack the tool on your local Galaxy instance's server: user@host:~% tar xvf galaxy_lastz_tool.tar ... user@host:~% tar zxvf galaxy_lastz_tool.tar.gz ... user@host:~% tar jxvf galaxy_lastz_tool.tar.bz2 ... If the archive includes a README file, consult it for installation instructions. If not, follow these basic steps: Create a directory under galaxy_dist/tools/ to house downloaded tool(s). In the new directory, place the XML and any script file(s) which were contained in the archive. If the tool includes binaries, you'll need to copy them to a directory on your $PATH. If the tool depends on C binaries but does not come with them (only source), you'll need to compile the source first. Add the tool to galaxy_dist/tool_conf.xml. Restart your Galaxy server process. In the near future, we plan to implement a more direct method to install tools via the Galaxy administrator user interface instead of placing files on the filesystem and manually managing the tool_conf.xml file. Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Show STDOUT in reports page
by Assaf Gordon 22 Jul '10

22 Jul '10

Hi, May I ask to add STDOUT to the job-info page of the very cool new reports webapp ? Attached a patch for that. Thanks, -gordon

2 3