Hello Aaron,
On Nov 23, 2011, at 8:20 AM, Aaron Quinlan wrote:
Hi Greg,
Thanks again for your help. One last question (famous last words). In the CADDSuite example, it appears that they are distributing compiled binaries in their Tool Shed repo. I am hesitant to do this, owing to the fact that my tools are in C and C++, so compilation is platform dependent.
Yes, including binary dependencies in the repository is not an ideal approach.
As such, I am planning to just release XML wrappers for BEDTools and assume that the users have already installed the appropriate release on their system. Does this meet with your expectations for tools that require compilation?
Yes, for now this is the best solution. In the future, the tool shed will include support for handling small fabric scripts that can be included in the tool shed repository for tools that have requirements like this. These fabric scripts will point to the location where the dependency source can be downloaded and compiled on the platform on which the local Galaxy instance resides. As part of the automatic Galaxy tool installation process from the tool shed to the local Galaxy instance, these dependencies will be automatically downloaded and compiled for tools installed from the tool shed. Until this automated process is functional, 3rd party dependencies must still be manually installed just like they are for all of the Galaxy tools (that have requirements) that come in the distribution. The tool wrappers themselves, however can currently be automatically installed from the tool shed to a local Galaxy instance.
Here is my current repo including just one tool:
http://testtoolshed.g2.bx.psu.edu/repository/browse_repositories?sort=name&a...
This looks good, but a couple of comments:
1. There is no longer any use for suite_config.xml - it was used in the old version of the tool shed, but is no longer used.
2. Including tool_conf.xml is not necessary - nothing is done with it.
3. Support for data types used by your tools that are not included in the Galaxy distribution will be available soon, so if your tools use them, you can include code files that include classes that support them. More information about this will soon be available in the tool shed wiki, but let me know if you have questions about this. Your current tool doesn't fall into this category.
4. Make sure that your tool configs include the <requirements> tag sets if the tool has a dependency. This is the tag set that will enable the use of the future fabric scripts discussed above. See the section of our tool config syntax wiki for details: http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Crequirement...
Thanks again, Aaron
On Nov 18, 2011, at 12:07 PM, Greg Von Kuster wrote:
Hello Aaron,
On Nov 18, 2011, at 10:37 AM, Aaron Quinlan wrote:
Hi Greg,
I am in the process of trying to develop a set of Galaxy wrappers for the tools in my bedtools suite. I have read the Toolshed wiki, but I am still struggling with some very basic questions that I hope you can help me with.
Thanks for your plans to contribute your Galaxy tools to the tool shed - the Galaxy community will appreciate this!
- How should a tarball repository be structured for the toolshed? For example, bedtools currently includes a Makefile plus a src/ directory and sub-directories for each tool. The compiled objects go to obj/ and the binaries to bin/.
The subdirectory hierarchy is completely up to you as the developer - the tools shed really doesn't care what this looks like (although I've included some thoughts on this in the following list). The intent of the Galaxy tool shed to be a place for sharing tools that have been tested to be functionally correct when loaded into a Galaxy instance. This means that you have developed the tools (with Galaxy wrappers) in your own local Galaxy development environment (and proved that they work within Galaxy) before uploading them to a Galaxy tool shed.
Keep in mind that your tools will ultimately be installed for use in other's local Galaxy instances, so here are some "rules of thumb" with regard to your tool shed repository that you create to contain your tools.
Keep your repository clean - do not upload items like .git or .svn subdirectories if you use these tools for development. Upload only those files that make the tools functionally correct within a Galaxy instance.
For simple tools that have no requirements ( see the <requirements> tag set in our Tool config syntax wiki at http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax ), your tarball can consist of a single directory that includes your tool files. Here's an example:
Contents: digital_dge DGE.xml DGEtest.xls DGEtest1out.html rgDGE.py
- For tools that require an index files (.loc.sample), you can add subdirectories if you want (again, not required - it's up to you). Here's an example:
Contents: blast2go tool-data/ blast2go.loc.sample tools/ ncbi_blast_plus/ blast2go.py blast2go.txt blast2go.xml
- For tools (like those you've described) that have 3rd party dependencies, you should place them in some directory within the hierarchy - what you've described is fine.
By reading the wiki I gather that I also need to add a new directory that contains an XML wrapper for each tool. Is that correct? Is there a specific name for said directory that is expected by the Toolshed?
Yes, you should do this and make sure the tools are functional within your local galaxy instance before uploading them to the tool shed.
- Do you have an example repository that has a similar structure to what is described in #1? This would be very helpful as a means to follow the lead of something that already works.
You can browse the directory hierarchy of any of the repositories in the tool shed using the "Browse repository tip files" pop-up menu option. Here's a good example that may be similar to yours:
Contents: caddsuite_linux_x86_64 CADDSuite-1.0.1/ AntitargetRescorer AutoModel BindingDBCleaner CADDSuite-description.txt CombiLibGenerator ConstraintsFinder Converter DBExporter DBImporter DockResultMerger EvenSplit FeatureSelector GalaxyConfigGenerator GridBuilder IMGDock InputPartitioner InputReader InteractionConstraintDefiner LigCheck Ligand3DGenerator LigandFileSplitter ModelCreator MolCombine MolDepict MolFilter MolPredictor PDBCutter PDBDownload PartialChargesCopy PocketDetector Predictor PropertyModifier PropertyPlotter ProteinCheck ProteinProtonator README RMSDCalculator ScoreAnalyzer SimilarityAnalyzer SimpleRescorer SpatialConstraintDefiner TaGRes TaGRes-train Validator VendorFinder WaterFinder bin/ changelog.txt data/ galaxyconfigs/ install.sh lib/ license.txt setPathes.sh suite_config.xml
I apologize if answers to these questions are made abundantly clear somewhere yet I have somehow missed it.
Best, Aaron
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Greg Von Kuster Galaxy Development Team greg@bx.psu.edu
Greg Von Kuster Galaxy Development Team greg@bx.psu.edu