Tool Shed packages for BLAST+ binaries
Hi Greg et al, For the NCBI BLAST+ wrappers, Nicola has suggested we split out the binaries themselves into a separate Tool Shed package, e.g. package_blast_plus_2_2_26 and similar for the later releases. https://github.com/peterjc/galaxy_blast/issues/7 I think this is a good idea. I'd like to check if there are any complications foreseeable from the fact that the NCBI BLAST+ wrappers were migrated out of the Galaxy core and therefore have migration scripts we need to consider. Potentially this could be held under the shared IUC account as planned for other commonly used packages? Or would the migration issues require it to also live under devteam? Also, we may wish to add a dependency on BOOST, e.g. http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 once this is on the main Tool Shed. Regards, Peter
Hi Peter,
Hi Greg et al,
For the NCBI BLAST+ wrappers, Nicola has suggested we split out the binaries themselves into a separate Tool Shed package, e.g. package_blast_plus_2_2_26 and similar for the later releases.
https://github.com/peterjc/galaxy_blast/issues/7
I think this is a good idea.
Yes, me too!
I'd like to check if there are any complications foreseeable from the fact that the NCBI BLAST+ wrappers were migrated out of the Galaxy core and therefore have migration scripts we need to consider.
Potentially this could be held under the shared IUC account as planned for other commonly used packages? Or would the migration issues require it to also live under devteam?
Also, we may wish to add a dependency on BOOST, e.g. http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 once this is on the main Tool Shed.
Ok, time to step up and migrate all the stuff to main. Cheers, Bjoern
Regards,
Peter _______________________________________________ galaxy-iuc mailing list galaxy-iuc@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-iuc
Hello Bjoern, I have the export repository ( and optionally all repoository dependencies ) feature working on the test tool shed. The feature allows you to chose a specific installable revision (if in the browser, by being on the Manage repository page for that revision or if using the API by passing the revision on the command line) for export. An example API call for this is ( from the ~/tool_shed/scripts/api directory): python ./export.py --url http://testtoolshed.g2.bx.psu.edu --name chemicaltoolbox --owner bgruening --revision 4133dbf7ff4d --export_repository_dependencies True --download_dir /tmp This will create a file named the following on your local disk: capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar.gz If you extract it, you'll have the following: lily:repo gvk$ tar -xvf capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar x package_graphicsmagick_1_3-34f511310435.tar.gz x package_eigen_2_0-7293f118efef.tar.gz x package_atlas_3_10-68df9078bf69.tar.gz x package_chemfp_1_1-616ee8e4abf4.tar.gz x package_freetype_2_4-4e54e357ac25.tar.gz x package_lapack_3_4-a6b8f46acca7.tar.gz x package_bzlib_1_0-58a38de96f9b.tar.gz x package_eigen_3_1-b72b882b30b4.tar.gz x natural_product_likeness-da45aa9141cf.tar.gz x chemical_data_sources-c065b806ff8c.tar.gz x package_openbabel_2_3-3a993ff17df5.tar.gz x osra-e5e3d723a675.tar.gz x confab-df8c4f142ae3.tar.gz x package_boost_1_53-f5b13e56a321.tar.gz x molecule_datatypes-18ff2d6e1dca.tar.gz x package_numpy_1_7-40db34c904df.tar.gz x openbabel-b4ba90dcfde0.tar.gz x package_scipy_0_12-cd2e7872a8eb.tar.gz x package_matplotlib_1_2-dd81b71dff1b.tar.gz x package_scikit_learn_0_13-ad4341b759f2.tar.gz x ctb_machine_learning-bf0014ee0871.tar.gz x opsin-ee73c8d529b1.tar.gz x package_rdkit_2012_12-4663c03127f7.tar.gz x rdkit-aa46f08461f9.tar.gz x chemfp-ee02ddefe21b.tar.gz x silicos_it-1c218c2a4bd8.tar.gz x chemicaltoolbox-4133dbf7ff4d.tar.gz x simsearch-b4eaf75cac77.tar.gz x export_info.xml x manifest.xml The manifest.xml file defines the order in which the various tarballs should be uploaded to the tool shed. It also includes the categories associated with each repository. The dependency definitions within each of these tarballs that contain <repository> tags have had the tool_shed and change set_revision attributes eliminated, so when they are loaded into a tool shed, they should be automatically populated. I do not yet have the import feature implemented, but I'm working on it. Even though the import is not finished, you may find the export useful in helping you migrate the chemicaltolbox over to the main tool shed. However, I have not tested anything yet with regard to importing or uploading these tarball, so use a local tool shed first if you can! Let me know if you have any questions or bump into any problems! Here is the manifest.xml file for the chemicaltolbox repository: <?xml version="1.0"?> <repository name="package_graphicsmagick_1_3" type="tool_dependency_definition" username="bgruening"> <description>GraphicsMagick Image Processing System</description> <long_description>GraphicsMagick is the swiss army knife of image processing and it provides a robust and efficient collection of tools and libraries which support reading, writing, and manipulating an image in over 88 major formats including important formats like DPX, GIF, JPEG, JPEG-2000, PNG, PDF, PNM, and TIFF. </long_description> <archive>package_graphicsmagick_1_3-34f511310435.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_eigen_2_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and extract version 2.0.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_2_0-7293f118efef.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_atlas_3_10" type="tool_dependency_definition" username="bgruening"> <description>Automatically Tuned Linear Algebra Software</description> <long_description>The ATLAS (Automatically Tuned Linear Algebra Software) project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance. At present, it provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK. http://math-atlas.sourceforge.net/</long_description> <archive>package_atlas_3_10-68df9078bf69.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_chemfp_1_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.1.x of the the chemfp package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit... This is the version 1.1.x of chemfp. http://chemfp.com/ chemfp provides toolkit independent generation of fingerprints and high-performance similarity search. The tools support the FPS fingerprint file format.</long_description> <archive>package_chemfp_1_1-616ee8e4abf4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_freetype_2_4" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.4 of the freetype font library.</description> <long_description>FreeType is a freely available software library to render fonts. It is written in C, designed to be small, efficient, highly customizable, and portable while capable of producing high-quality output (glyph images) of most vector and bitmap font formats. http://www.freetype.org/</long_description> <archive>package_freetype_2_4-4e54e357ac25.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_lapack_3_4" type="tool_dependency_definition" username="bgruening"> <description>Linear Algebra PACKage</description> <long_description>Lapack provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. http://www.netlib.org You need a Fortran 90 compiler, e.g. gfortran, to build that package.</long_description> <archive>package_lapack_3_4-a6b8f46acca7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_bzlib_1_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.0 of the bzlib library.</description> <long_description>bzip2 is a freely available, patent free (see below), high-quality data compressor. It typically compresses files to within 10X to 15X of the best available techniques (the PPM family of statistical compressors), whilst being around twice as fast at compression and six times faster at decompression. http://www.bzip.org</long_description> <archive>package_bzlib_1_0-58a38de96f9b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_eigen_3_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads, compiles and extracts version 3.1.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_3_1-b72b882b30b4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="natural_product_likeness" type="unrestricted" username="bgruening"> <description>Natural product likeness scorer</description> <long_description>The natural product likeness scorer implemented by Peter Ertl was originally devised to screen large compound libraries for natural product likeness in drug designing studies. His work is re-implemented using open source and open data to validate natural product likeness of theoretical metabolite structures in computer assisted structure elucidation. http://sourceforge.net/p/np-likeness</long_description> <archive>natural_product_likeness-da45aa9141cf.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="chemical_data_sources" type="unrestricted" username="bgruening"> <description>Data sources for small molecules</description> <archive>chemical_data_sources-c065b806ff8c.tar.gz</archive> <categories> <category>Data Source</category> </categories> </repository> <repository name="package_openbabel_2_3" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.3.2 of the Open Babel package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit... This is the version 2.3.2 of Open Babel. http://openbabel.org Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>package_openbabel_2_3-3a993ff17df5.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="osra" type="unrestricted" username="bgruening"> <description>OSRA - Optical Structure Recognition Application</description> <long_description>OSRA is a utility designed to convert graphical representations of chemical structures, as they appear in journal articles, patent documents, textbooks, trade magazines etc., into SMILES.</long_description> <archive>osra-e5e3d723a675.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="confab" type="unrestricted" username="bgruening"> <description>Systematic conformation generator</description> <long_description>Confab is an open source conformation generator whose goal is the systematic coverage of conformational space.</long_description> <archive>confab-df8c4f142ae3.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="package_boost_1_53" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.53 of the boost C++ libraries</description> <long_description>Boost provides free peer-reviewed portable C++ source libraries. http://www.boost.org/</long_description> <archive>package_boost_1_53-f5b13e56a321.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="molecule_datatypes" type="unrestricted" username="bgruening"> <description>Datatypes and converter for cheminformatics</description> <long_description>Molecule datatypes and converters. Including SDF, MOL, SMILES, InChI, MOL2, PDB ...</long_description> <archive>molecule_datatypes-18ff2d6e1dca.tar.gz</archive> <categories> <category>Computational chemistry</category> <category>Convert Formats</category> </categories> </repository> <repository name="package_numpy_1_7" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.7 of the the python numpy package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit... This is the version 1.7 of numpy. www.numpy.org/ NumPy is the fundamental package for scientific computing with Python.</long_description> <archive>package_numpy_1_7-40db34c904df.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="openbabel" type="unrestricted" username="bgruening"> <description>Open Babel tools</description> <long_description>Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>openbabel-b4ba90dcfde0.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="package_scipy_0_12" type="tool_dependency_definition" username="bgruening"> <description>Scientific Tools for Python</description> <long_description>SciPy is open-source software for mathematics, science, and engineering. The SciPy library is built to work with NumPy arrays, and provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization.</long_description> <archive>package_scipy_0_12-cd2e7872a8eb.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_matplotlib_1_2" type="unrestricted" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.2.x of the the python matplotlib package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit... This is the version 1.2.x of matplotlib. www.matplotlib.org/ matplotlib is a python 2D plotting library which produces publication quality figures.</long_description> <archive>package_matplotlib_1_2-dd81b71dff1b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="package_scikit_learn_0_13" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 0.13.x of the the scikit-learn package.</description> <long_description>Easy-to-use and general-purpose machine learning in Python Scikit-learn integrates machine learning algorithms in the tightly-knit scientific Python world, building upon numpy, scipy, and matplotlib. As a machine-learning module, it provides versatile tools for data mining and analysis in any field of science and engineering. It strives to be simple and efficient, accessible to everybody, and reusable in various contexts. http://scikit-learn.org/</long_description> <archive>package_scikit_learn_0_13-ad4341b759f2.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="ctb_machine_learning" type="unrestricted" username="bgruening"> <description>machine learning scripts for the chemicaltoolbox</description> <long_description>This repository contains small tools dealing with machine learning techniques inside the chemicaltoolbox.</long_description> <archive>ctb_machine_learning-bf0014ee0871.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="opsin" type="unrestricted" username="bgruening"> <description>Open Parser for Systematic IUPAC nomenclature</description> <long_description>OPSIN is library for IUPAC name-to-structure conversion offering high recall and precision on organic chemical nomenclature. https://bitbucket.org/dan2097/opsin</long_description> <archive>opsin-ee73c8d529b1.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="package_rdkit_2012_12" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2012-12 of the RDKit cheminformatics and machine-learning package.</description> <long_description>A collection of cheminformatics and machine-learning software written in C++ and Python. The core algorithms and data structures are written in C++. Wrappers are provided to use the toolkit from either Python or Java. Additionally, the RDKit distribution includes a PostgreSQL-based cartridge that allows molecules to be stored in relational database and retrieved via substructure and similarity searches. http://www.rdkit.org/</long_description> <archive>package_rdkit_2012_12-4663c03127f7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository> <repository name="rdkit" type="unrestricted" username="bgruening"> <description>RDKit tools.</description> <long_description>Various tools that are using the RDKit library.</long_description> <archive>rdkit-aa46f08461f9.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="chemfp" type="unrestricted" username="bgruening"> <description>Chemfp tools.</description> <long_description>chemfp is a set of tools for working with cheminformatics fingerprints in the FPS format.</long_description> <archive>chemfp-ee02ddefe21b.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="silicos_it" type="unrestricted" username="bgruening"> <description>silicos</description> <archive>silicos_it-1c218c2a4bd8.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="chemicaltoolbox" type="unrestricted" username="bgruening"> <description>ChemicalToolBoX Metapackage</description> <long_description>That metapackage will install several cheminformatic tool wrappers (e.g. chemfp, openbabel, osra, confab, rdkit)</long_description> <archive>chemicaltoolbox-4133dbf7ff4d.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> <repository name="simsearch" type="unrestricted" username="bgruening"> <description>Similarity search with openbabel or chemfp</description> <long_description>Similarity search with openbabel or chemfp</long_description> <archive>simsearch-b4eaf75cac77.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository> On Aug 7, 2013, at 10:28 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Ok, time to step up and migrate all the stuff to main.
Cheers, Bjoern
Hi Greg, wonderful will test it locally asap and report any error. The most urgent packages I will migrate manually for the time being. I promised it far to long. Thanks Greg!!! Bjoern
Hello Bjoern,
I have the export repository ( and optionally all repoository dependencies ) feature working on the test tool shed. The feature allows you to chose a specific installable revision (if in the browser, by being on the Manage repository page for that revision or if using the API by passing the revision on the command line) for export.
An example API call for this is ( from the ~/tool_shed/scripts/api directory):
python ./export.py --url http://testtoolshed.g2.bx.psu.edu --name chemicaltoolbox --owner bgruening --revision 4133dbf7ff4d --export_repository_dependencies True --download_dir /tmp
This will create a file named the following on your local disk:
capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar.gz
If you extract it, you'll have the following:
lily:repo gvk$ tar -xvf capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar x package_graphicsmagick_1_3-34f511310435.tar.gz x package_eigen_2_0-7293f118efef.tar.gz x package_atlas_3_10-68df9078bf69.tar.gz x package_chemfp_1_1-616ee8e4abf4.tar.gz x package_freetype_2_4-4e54e357ac25.tar.gz x package_lapack_3_4-a6b8f46acca7.tar.gz x package_bzlib_1_0-58a38de96f9b.tar.gz x package_eigen_3_1-b72b882b30b4.tar.gz x natural_product_likeness-da45aa9141cf.tar.gz x chemical_data_sources-c065b806ff8c.tar.gz x package_openbabel_2_3-3a993ff17df5.tar.gz x osra-e5e3d723a675.tar.gz x confab-df8c4f142ae3.tar.gz x package_boost_1_53-f5b13e56a321.tar.gz x molecule_datatypes-18ff2d6e1dca.tar.gz x package_numpy_1_7-40db34c904df.tar.gz x openbabel-b4ba90dcfde0.tar.gz x package_scipy_0_12-cd2e7872a8eb.tar.gz x package_matplotlib_1_2-dd81b71dff1b.tar.gz x package_scikit_learn_0_13-ad4341b759f2.tar.gz x ctb_machine_learning-bf0014ee0871.tar.gz x opsin-ee73c8d529b1.tar.gz x package_rdkit_2012_12-4663c03127f7.tar.gz x rdkit-aa46f08461f9.tar.gz x chemfp-ee02ddefe21b.tar.gz x silicos_it-1c218c2a4bd8.tar.gz x chemicaltoolbox-4133dbf7ff4d.tar.gz x simsearch-b4eaf75cac77.tar.gz x export_info.xml x manifest.xml
The manifest.xml file defines the order in which the various tarballs should be uploaded to the tool shed. It also includes the categories associated with each repository. The dependency definitions within each of these tarballs that contain <repository> tags have had the tool_shed and change set_revision attributes eliminated, so when they are loaded into a tool shed, they should be automatically populated.
I do not yet have the import feature implemented, but I'm working on it. Even though the import is not finished, you may find the export useful in helping you migrate the chemicaltolbox over to the main tool shed. However, I have not tested anything yet with regard to importing or uploading these tarball, so use a local tool shed first if you can!
Let me know if you have any questions or bump into any problems!
Here is the manifest.xml file for the chemicaltolbox repository:
<?xml version="1.0"?> <repository name="package_graphicsmagick_1_3" type="tool_dependency_definition" username="bgruening"> <description>GraphicsMagick Image Processing System</description> <long_description>GraphicsMagick is the swiss army knife of image processing and it provides a robust and efficient collection of tools and libraries which support reading, writing, and manipulating an image in over 88 major formats including important formats like DPX, GIF, JPEG, JPEG-2000, PNG, PDF, PNM, and TIFF. </long_description> <archive>package_graphicsmagick_1_3-34f511310435.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_eigen_2_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and extract version 2.0.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_2_0-7293f118efef.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_atlas_3_10" type="tool_dependency_definition" username="bgruening"> <description>Automatically Tuned Linear Algebra Software</description> <long_description>The ATLAS (Automatically Tuned Linear Algebra Software) project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance. At present, it provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK.
http://math-atlas.sourceforge.net/</long_description> <archive>package_atlas_3_10-68df9078bf69.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_chemfp_1_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.1.x of the the chemfp package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.1.x of chemfp. http://chemfp.com/
chemfp provides toolkit independent generation of fingerprints and high-performance similarity search. The tools support the FPS fingerprint file format.</long_description> <archive>package_chemfp_1_1-616ee8e4abf4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_freetype_2_4" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.4 of the freetype font library.</description> <long_description>FreeType is a freely available software library to render fonts.
It is written in C, designed to be small, efficient, highly customizable, and portable while capable of producing high-quality output (glyph images) of most vector and bitmap font formats. http://www.freetype.org/</long_description> <archive>package_freetype_2_4-4e54e357ac25.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_lapack_3_4" type="tool_dependency_definition" username="bgruening"> <description>Linear Algebra PACKage</description> <long_description>Lapack provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.
You need a Fortran 90 compiler, e.g. gfortran, to build that package.</long_description> <archive>package_lapack_3_4-a6b8f46acca7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_bzlib_1_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.0 of the bzlib library.</description> <long_description>bzip2 is a freely available, patent free (see below), high-quality data compressor. It typically compresses files to within 10X to 15X of the best available techniques (the PPM family of statistical compressors), whilst being around twice as fast at compression and six times faster at decompression. http://www.bzip.org</long_description> <archive>package_bzlib_1_0-58a38de96f9b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_eigen_3_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads, compiles and extracts version 3.1.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_3_1-b72b882b30b4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="natural_product_likeness" type="unrestricted" username="bgruening"> <description>Natural product likeness scorer</description> <long_description>The natural product likeness scorer implemented by Peter Ertl was originally devised to screen large compound libraries for natural product likeness in drug designing studies. His work is re-implemented using open source and open data to validate natural product likeness of theoretical metabolite structures in computer assisted structure elucidation.
http://sourceforge.net/p/np-likeness</long_description> <archive>natural_product_likeness-da45aa9141cf.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemical_data_sources" type="unrestricted" username="bgruening"> <description>Data sources for small molecules</description> <archive>chemical_data_sources-c065b806ff8c.tar.gz</archive> <categories> <category>Data Source</category> </categories> </repository>
<repository name="package_openbabel_2_3" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.3.2 of the Open Babel package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 2.3.2 of Open Babel. http://openbabel.org
Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>package_openbabel_2_3-3a993ff17df5.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="osra" type="unrestricted" username="bgruening"> <description>OSRA - Optical Structure Recognition Application</description> <long_description>OSRA is a utility designed to convert graphical representations of chemical structures, as they appear in journal articles, patent documents, textbooks, trade magazines etc., into SMILES.</long_description> <archive>osra-e5e3d723a675.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="confab" type="unrestricted" username="bgruening"> <description>Systematic conformation generator</description> <long_description>Confab is an open source conformation generator whose goal is the systematic coverage of conformational space.</long_description> <archive>confab-df8c4f142ae3.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_boost_1_53" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.53 of the boost C++ libraries</description> <long_description>Boost provides free peer-reviewed portable C++ source libraries. http://www.boost.org/</long_description> <archive>package_boost_1_53-f5b13e56a321.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="molecule_datatypes" type="unrestricted" username="bgruening"> <description>Datatypes and converter for cheminformatics</description> <long_description>Molecule datatypes and converters. Including SDF, MOL, SMILES, InChI, MOL2, PDB ...</long_description> <archive>molecule_datatypes-18ff2d6e1dca.tar.gz</archive> <categories> <category>Computational chemistry</category> <category>Convert Formats</category> </categories> </repository>
<repository name="package_numpy_1_7" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.7 of the the python numpy package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.7 of numpy. www.numpy.org/
NumPy is the fundamental package for scientific computing with Python.</long_description> <archive>package_numpy_1_7-40db34c904df.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="openbabel" type="unrestricted" username="bgruening"> <description>Open Babel tools</description> <long_description>Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>openbabel-b4ba90dcfde0.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_scipy_0_12" type="tool_dependency_definition" username="bgruening"> <description>Scientific Tools for Python</description> <long_description>SciPy is open-source software for mathematics, science, and engineering. The SciPy library is built to work with NumPy arrays, and provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization.</long_description> <archive>package_scipy_0_12-cd2e7872a8eb.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_matplotlib_1_2" type="unrestricted" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.2.x of the the python matplotlib package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.2.x of matplotlib. www.matplotlib.org/
matplotlib is a python 2D plotting library which produces publication quality figures.</long_description> <archive>package_matplotlib_1_2-dd81b71dff1b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_scikit_learn_0_13" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 0.13.x of the the scikit-learn package.</description> <long_description>Easy-to-use and general-purpose machine learning in Python
Scikit-learn integrates machine learning algorithms in the tightly-knit scientific Python world, building upon numpy, scipy, and matplotlib. As a machine-learning module, it provides versatile tools for data mining and analysis in any field of science and engineering. It strives to be simple and efficient, accessible to everybody, and reusable in various contexts.
http://scikit-learn.org/</long_description> <archive>package_scikit_learn_0_13-ad4341b759f2.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="ctb_machine_learning" type="unrestricted" username="bgruening"> <description>machine learning scripts for the chemicaltoolbox</description> <long_description>This repository contains small tools dealing with machine learning techniques inside the chemicaltoolbox.</long_description> <archive>ctb_machine_learning-bf0014ee0871.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="opsin" type="unrestricted" username="bgruening"> <description>Open Parser for Systematic IUPAC nomenclature</description> <long_description>OPSIN is library for IUPAC name-to-structure conversion offering high recall and precision on organic chemical nomenclature. https://bitbucket.org/dan2097/opsin</long_description> <archive>opsin-ee73c8d529b1.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_rdkit_2012_12" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2012-12 of the RDKit cheminformatics and machine-learning package.</description> <long_description>A collection of cheminformatics and machine-learning software written in C++ and Python.
The core algorithms and data structures are written in C++. Wrappers are provided to use the toolkit from either Python or Java.
Additionally, the RDKit distribution includes a PostgreSQL-based cartridge that allows molecules to be stored in relational database and retrieved via substructure and similarity searches. http://www.rdkit.org/</long_description> <archive>package_rdkit_2012_12-4663c03127f7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="rdkit" type="unrestricted" username="bgruening"> <description>RDKit tools.</description> <long_description>Various tools that are using the RDKit library.</long_description> <archive>rdkit-aa46f08461f9.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemfp" type="unrestricted" username="bgruening"> <description>Chemfp tools.</description> <long_description>chemfp is a set of tools for working with cheminformatics fingerprints in the FPS format.</long_description> <archive>chemfp-ee02ddefe21b.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="silicos_it" type="unrestricted" username="bgruening"> <description>silicos</description> <archive>silicos_it-1c218c2a4bd8.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemicaltoolbox" type="unrestricted" username="bgruening"> <description>ChemicalToolBoX Metapackage</description> <long_description>That metapackage will install several cheminformatic tool wrappers (e.g. chemfp, openbabel, osra, confab, rdkit)</long_description> <archive>chemicaltoolbox-4133dbf7ff4d.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="simsearch" type="unrestricted" username="bgruening"> <description>Similarity search with openbabel or chemfp</description> <long_description>Similarity search with openbabel or chemfp</long_description> <archive>simsearch-b4eaf75cac77.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
On Aug 7, 2013, at 10:28 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Ok, time to step up and migrate all the stuff to main.
Cheers, Bjoern
Thanks Bjoern, My hope is that the current export process will help you with whatever you are manually migrating. If you export the chemicaltoolbox and all of it's dependencies, you will have a nice tarball of each repository that you want to manually upload to the main tool shed. Of course, you'll need to be looged in as the owner that you want for each of the uploaded repositories. The order that you'll want to upload each of them manually is defined in the manifest.xml file, so that should help as well. And automatically populating the "unpopulated" toolshed and changeset_revision attributes for <repository> tags should make things easier. Thanks! Greg On Aug 7, 2013, at 11:20 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Hi Greg,
wonderful will test it locally asap and report any error. The most urgent packages I will migrate manually for the time being. I promised it far to long.
Thanks Greg!!! Bjoern
Hello Bjoern,
I have the export repository ( and optionally all repoository dependencies ) feature working on the test tool shed. The feature allows you to chose a specific installable revision (if in the browser, by being on the Manage repository page for that revision or if using the API by passing the revision on the command line) for export.
An example API call for this is ( from the ~/tool_shed/scripts/api directory):
python ./export.py --url http://testtoolshed.g2.bx.psu.edu --name chemicaltoolbox --owner bgruening --revision 4133dbf7ff4d --export_repository_dependencies True --download_dir /tmp
This will create a file named the following on your local disk:
capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar.gz
If you extract it, you'll have the following:
lily:repo gvk$ tar -xvf capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar x package_graphicsmagick_1_3-34f511310435.tar.gz x package_eigen_2_0-7293f118efef.tar.gz x package_atlas_3_10-68df9078bf69.tar.gz x package_chemfp_1_1-616ee8e4abf4.tar.gz x package_freetype_2_4-4e54e357ac25.tar.gz x package_lapack_3_4-a6b8f46acca7.tar.gz x package_bzlib_1_0-58a38de96f9b.tar.gz x package_eigen_3_1-b72b882b30b4.tar.gz x natural_product_likeness-da45aa9141cf.tar.gz x chemical_data_sources-c065b806ff8c.tar.gz x package_openbabel_2_3-3a993ff17df5.tar.gz x osra-e5e3d723a675.tar.gz x confab-df8c4f142ae3.tar.gz x package_boost_1_53-f5b13e56a321.tar.gz x molecule_datatypes-18ff2d6e1dca.tar.gz x package_numpy_1_7-40db34c904df.tar.gz x openbabel-b4ba90dcfde0.tar.gz x package_scipy_0_12-cd2e7872a8eb.tar.gz x package_matplotlib_1_2-dd81b71dff1b.tar.gz x package_scikit_learn_0_13-ad4341b759f2.tar.gz x ctb_machine_learning-bf0014ee0871.tar.gz x opsin-ee73c8d529b1.tar.gz x package_rdkit_2012_12-4663c03127f7.tar.gz x rdkit-aa46f08461f9.tar.gz x chemfp-ee02ddefe21b.tar.gz x silicos_it-1c218c2a4bd8.tar.gz x chemicaltoolbox-4133dbf7ff4d.tar.gz x simsearch-b4eaf75cac77.tar.gz x export_info.xml x manifest.xml
The manifest.xml file defines the order in which the various tarballs should be uploaded to the tool shed. It also includes the categories associated with each repository. The dependency definitions within each of these tarballs that contain <repository> tags have had the tool_shed and change set_revision attributes eliminated, so when they are loaded into a tool shed, they should be automatically populated.
I do not yet have the import feature implemented, but I'm working on it. Even though the import is not finished, you may find the export useful in helping you migrate the chemicaltolbox over to the main tool shed. However, I have not tested anything yet with regard to importing or uploading these tarball, so use a local tool shed first if you can!
Let me know if you have any questions or bump into any problems!
Here is the manifest.xml file for the chemicaltolbox repository:
<?xml version="1.0"?> <repository name="package_graphicsmagick_1_3" type="tool_dependency_definition" username="bgruening"> <description>GraphicsMagick Image Processing System</description> <long_description>GraphicsMagick is the swiss army knife of image processing and it provides a robust and efficient collection of tools and libraries which support reading, writing, and manipulating an image in over 88 major formats including important formats like DPX, GIF, JPEG, JPEG-2000, PNG, PDF, PNM, and TIFF. </long_description> <archive>package_graphicsmagick_1_3-34f511310435.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_eigen_2_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and extract version 2.0.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_2_0-7293f118efef.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_atlas_3_10" type="tool_dependency_definition" username="bgruening"> <description>Automatically Tuned Linear Algebra Software</description> <long_description>The ATLAS (Automatically Tuned Linear Algebra Software) project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance. At present, it provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK.
http://math-atlas.sourceforge.net/</long_description> <archive>package_atlas_3_10-68df9078bf69.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_chemfp_1_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.1.x of the the chemfp package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.1.x of chemfp. http://chemfp.com/
chemfp provides toolkit independent generation of fingerprints and high-performance similarity search. The tools support the FPS fingerprint file format.</long_description> <archive>package_chemfp_1_1-616ee8e4abf4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_freetype_2_4" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.4 of the freetype font library.</description> <long_description>FreeType is a freely available software library to render fonts.
It is written in C, designed to be small, efficient, highly customizable, and portable while capable of producing high-quality output (glyph images) of most vector and bitmap font formats. http://www.freetype.org/</long_description> <archive>package_freetype_2_4-4e54e357ac25.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_lapack_3_4" type="tool_dependency_definition" username="bgruening"> <description>Linear Algebra PACKage</description> <long_description>Lapack provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.
You need a Fortran 90 compiler, e.g. gfortran, to build that package.</long_description> <archive>package_lapack_3_4-a6b8f46acca7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_bzlib_1_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.0 of the bzlib library.</description> <long_description>bzip2 is a freely available, patent free (see below), high-quality data compressor. It typically compresses files to within 10X to 15X of the best available techniques (the PPM family of statistical compressors), whilst being around twice as fast at compression and six times faster at decompression. http://www.bzip.org</long_description> <archive>package_bzlib_1_0-58a38de96f9b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_eigen_3_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads, compiles and extracts version 3.1.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_3_1-b72b882b30b4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="natural_product_likeness" type="unrestricted" username="bgruening"> <description>Natural product likeness scorer</description> <long_description>The natural product likeness scorer implemented by Peter Ertl was originally devised to screen large compound libraries for natural product likeness in drug designing studies. His work is re-implemented using open source and open data to validate natural product likeness of theoretical metabolite structures in computer assisted structure elucidation.
http://sourceforge.net/p/np-likeness</long_description> <archive>natural_product_likeness-da45aa9141cf.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemical_data_sources" type="unrestricted" username="bgruening"> <description>Data sources for small molecules</description> <archive>chemical_data_sources-c065b806ff8c.tar.gz</archive> <categories> <category>Data Source</category> </categories> </repository>
<repository name="package_openbabel_2_3" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.3.2 of the Open Babel package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 2.3.2 of Open Babel. http://openbabel.org
Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>package_openbabel_2_3-3a993ff17df5.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="osra" type="unrestricted" username="bgruening"> <description>OSRA - Optical Structure Recognition Application</description> <long_description>OSRA is a utility designed to convert graphical representations of chemical structures, as they appear in journal articles, patent documents, textbooks, trade magazines etc., into SMILES.</long_description> <archive>osra-e5e3d723a675.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="confab" type="unrestricted" username="bgruening"> <description>Systematic conformation generator</description> <long_description>Confab is an open source conformation generator whose goal is the systematic coverage of conformational space.</long_description> <archive>confab-df8c4f142ae3.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_boost_1_53" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.53 of the boost C++ libraries</description> <long_description>Boost provides free peer-reviewed portable C++ source libraries. http://www.boost.org/</long_description> <archive>package_boost_1_53-f5b13e56a321.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="molecule_datatypes" type="unrestricted" username="bgruening"> <description>Datatypes and converter for cheminformatics</description> <long_description>Molecule datatypes and converters. Including SDF, MOL, SMILES, InChI, MOL2, PDB ...</long_description> <archive>molecule_datatypes-18ff2d6e1dca.tar.gz</archive> <categories> <category>Computational chemistry</category> <category>Convert Formats</category> </categories> </repository>
<repository name="package_numpy_1_7" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.7 of the the python numpy package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.7 of numpy. www.numpy.org/
NumPy is the fundamental package for scientific computing with Python.</long_description> <archive>package_numpy_1_7-40db34c904df.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="openbabel" type="unrestricted" username="bgruening"> <description>Open Babel tools</description> <long_description>Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>openbabel-b4ba90dcfde0.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_scipy_0_12" type="tool_dependency_definition" username="bgruening"> <description>Scientific Tools for Python</description> <long_description>SciPy is open-source software for mathematics, science, and engineering. The SciPy library is built to work with NumPy arrays, and provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization.</long_description> <archive>package_scipy_0_12-cd2e7872a8eb.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_matplotlib_1_2" type="unrestricted" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.2.x of the the python matplotlib package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.2.x of matplotlib. www.matplotlib.org/
matplotlib is a python 2D plotting library which produces publication quality figures.</long_description> <archive>package_matplotlib_1_2-dd81b71dff1b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_scikit_learn_0_13" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 0.13.x of the the scikit-learn package.</description> <long_description>Easy-to-use and general-purpose machine learning in Python
Scikit-learn integrates machine learning algorithms in the tightly-knit scientific Python world, building upon numpy, scipy, and matplotlib. As a machine-learning module, it provides versatile tools for data mining and analysis in any field of science and engineering. It strives to be simple and efficient, accessible to everybody, and reusable in various contexts.
http://scikit-learn.org/</long_description> <archive>package_scikit_learn_0_13-ad4341b759f2.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="ctb_machine_learning" type="unrestricted" username="bgruening"> <description>machine learning scripts for the chemicaltoolbox</description> <long_description>This repository contains small tools dealing with machine learning techniques inside the chemicaltoolbox.</long_description> <archive>ctb_machine_learning-bf0014ee0871.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="opsin" type="unrestricted" username="bgruening"> <description>Open Parser for Systematic IUPAC nomenclature</description> <long_description>OPSIN is library for IUPAC name-to-structure conversion offering high recall and precision on organic chemical nomenclature. https://bitbucket.org/dan2097/opsin</long_description> <archive>opsin-ee73c8d529b1.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_rdkit_2012_12" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2012-12 of the RDKit cheminformatics and machine-learning package.</description> <long_description>A collection of cheminformatics and machine-learning software written in C++ and Python.
The core algorithms and data structures are written in C++. Wrappers are provided to use the toolkit from either Python or Java.
Additionally, the RDKit distribution includes a PostgreSQL-based cartridge that allows molecules to be stored in relational database and retrieved via substructure and similarity searches. http://www.rdkit.org/</long_description> <archive>package_rdkit_2012_12-4663c03127f7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="rdkit" type="unrestricted" username="bgruening"> <description>RDKit tools.</description> <long_description>Various tools that are using the RDKit library.</long_description> <archive>rdkit-aa46f08461f9.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemfp" type="unrestricted" username="bgruening"> <description>Chemfp tools.</description> <long_description>chemfp is a set of tools for working with cheminformatics fingerprints in the FPS format.</long_description> <archive>chemfp-ee02ddefe21b.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="silicos_it" type="unrestricted" username="bgruening"> <description>silicos</description> <archive>silicos_it-1c218c2a4bd8.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemicaltoolbox" type="unrestricted" username="bgruening"> <description>ChemicalToolBoX Metapackage</description> <long_description>That metapackage will install several cheminformatic tool wrappers (e.g. chemfp, openbabel, osra, confab, rdkit)</long_description> <archive>chemicaltoolbox-4133dbf7ff4d.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="simsearch" type="unrestricted" username="bgruening"> <description>Similarity search with openbabel or chemfp</description> <long_description>Similarity search with openbabel or chemfp</long_description> <archive>simsearch-b4eaf75cac77.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
On Aug 7, 2013, at 10:28 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Ok, time to step up and migrate all the stuff to main.
Cheers, Bjoern
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Greg, I tried the manifest file and saw that the ordering is not correct. You can see that on the last to repositories. Since, chemicaltoolbox is the meta_repository it should be always the last one. Cheers, Bjoern
Hello Bjoern,
I have the export repository ( and optionally all repoository dependencies ) feature working on the test tool shed. The feature allows you to chose a specific installable revision (if in the browser, by being on the Manage repository page for that revision or if using the API by passing the revision on the command line) for export.
An example API call for this is ( from the ~/tool_shed/scripts/api directory):
python ./export.py --url http://testtoolshed.g2.bx.psu.edu --name chemicaltoolbox --owner bgruening --revision 4133dbf7ff4d --export_repository_dependencies True --download_dir /tmp
This will create a file named the following on your local disk:
capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar.gz
If you extract it, you'll have the following:
lily:repo gvk$ tar -xvf capsule_with_dependencies_testtoolshed.g2.bx.psu.edu_chemicaltoolbox_bgruening_4133dbf7ff4d.tar x package_graphicsmagick_1_3-34f511310435.tar.gz x package_eigen_2_0-7293f118efef.tar.gz x package_atlas_3_10-68df9078bf69.tar.gz x package_chemfp_1_1-616ee8e4abf4.tar.gz x package_freetype_2_4-4e54e357ac25.tar.gz x package_lapack_3_4-a6b8f46acca7.tar.gz x package_bzlib_1_0-58a38de96f9b.tar.gz x package_eigen_3_1-b72b882b30b4.tar.gz x natural_product_likeness-da45aa9141cf.tar.gz x chemical_data_sources-c065b806ff8c.tar.gz x package_openbabel_2_3-3a993ff17df5.tar.gz x osra-e5e3d723a675.tar.gz x confab-df8c4f142ae3.tar.gz x package_boost_1_53-f5b13e56a321.tar.gz x molecule_datatypes-18ff2d6e1dca.tar.gz x package_numpy_1_7-40db34c904df.tar.gz x openbabel-b4ba90dcfde0.tar.gz x package_scipy_0_12-cd2e7872a8eb.tar.gz x package_matplotlib_1_2-dd81b71dff1b.tar.gz x package_scikit_learn_0_13-ad4341b759f2.tar.gz x ctb_machine_learning-bf0014ee0871.tar.gz x opsin-ee73c8d529b1.tar.gz x package_rdkit_2012_12-4663c03127f7.tar.gz x rdkit-aa46f08461f9.tar.gz x chemfp-ee02ddefe21b.tar.gz x silicos_it-1c218c2a4bd8.tar.gz x chemicaltoolbox-4133dbf7ff4d.tar.gz x simsearch-b4eaf75cac77.tar.gz x export_info.xml x manifest.xml
The manifest.xml file defines the order in which the various tarballs should be uploaded to the tool shed. It also includes the categories associated with each repository. The dependency definitions within each of these tarballs that contain <repository> tags have had the tool_shed and change set_revision attributes eliminated, so when they are loaded into a tool shed, they should be automatically populated.
I do not yet have the import feature implemented, but I'm working on it. Even though the import is not finished, you may find the export useful in helping you migrate the chemicaltolbox over to the main tool shed. However, I have not tested anything yet with regard to importing or uploading these tarball, so use a local tool shed first if you can!
Let me know if you have any questions or bump into any problems!
Here is the manifest.xml file for the chemicaltolbox repository:
<?xml version="1.0"?> <repository name="package_graphicsmagick_1_3" type="tool_dependency_definition" username="bgruening"> <description>GraphicsMagick Image Processing System</description> <long_description>GraphicsMagick is the swiss army knife of image processing and it provides a robust and efficient collection of tools and libraries which support reading, writing, and manipulating an image in over 88 major formats including important formats like DPX, GIF, JPEG, JPEG-2000, PNG, PDF, PNM, and TIFF. </long_description> <archive>package_graphicsmagick_1_3-34f511310435.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_eigen_2_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and extract version 2.0.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_2_0-7293f118efef.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_atlas_3_10" type="tool_dependency_definition" username="bgruening"> <description>Automatically Tuned Linear Algebra Software</description> <long_description>The ATLAS (Automatically Tuned Linear Algebra Software) project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance. At present, it provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a few routines from LAPACK.
http://math-atlas.sourceforge.net/</long_description> <archive>package_atlas_3_10-68df9078bf69.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_chemfp_1_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.1.x of the the chemfp package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.1.x of chemfp. http://chemfp.com/
chemfp provides toolkit independent generation of fingerprints and high-performance similarity search. The tools support the FPS fingerprint file format.</long_description> <archive>package_chemfp_1_1-616ee8e4abf4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_freetype_2_4" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.4 of the freetype font library.</description> <long_description>FreeType is a freely available software library to render fonts.
It is written in C, designed to be small, efficient, highly customizable, and portable while capable of producing high-quality output (glyph images) of most vector and bitmap font formats. http://www.freetype.org/</long_description> <archive>package_freetype_2_4-4e54e357ac25.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_lapack_3_4" type="tool_dependency_definition" username="bgruening"> <description>Linear Algebra PACKage</description> <long_description>Lapack provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.
You need a Fortran 90 compiler, e.g. gfortran, to build that package.</long_description> <archive>package_lapack_3_4-a6b8f46acca7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_bzlib_1_0" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.0 of the bzlib library.</description> <long_description>bzip2 is a freely available, patent free (see below), high-quality data compressor. It typically compresses files to within 10X to 15X of the best available techniques (the PPM family of statistical compressors), whilst being around twice as fast at compression and six times faster at decompression. http://www.bzip.org</long_description> <archive>package_bzlib_1_0-58a38de96f9b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_eigen_3_1" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads, compiles and extracts version 3.1.x of the the eigen template library.</description> <long_description>Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.</long_description> <archive>package_eigen_3_1-b72b882b30b4.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="natural_product_likeness" type="unrestricted" username="bgruening"> <description>Natural product likeness scorer</description> <long_description>The natural product likeness scorer implemented by Peter Ertl was originally devised to screen large compound libraries for natural product likeness in drug designing studies. His work is re-implemented using open source and open data to validate natural product likeness of theoretical metabolite structures in computer assisted structure elucidation.
http://sourceforge.net/p/np-likeness</long_description> <archive>natural_product_likeness-da45aa9141cf.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemical_data_sources" type="unrestricted" username="bgruening"> <description>Data sources for small molecules</description> <archive>chemical_data_sources-c065b806ff8c.tar.gz</archive> <categories> <category>Data Source</category> </categories> </repository>
<repository name="package_openbabel_2_3" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2.3.2 of the Open Babel package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 2.3.2 of Open Babel. http://openbabel.org
Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>package_openbabel_2_3-3a993ff17df5.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="osra" type="unrestricted" username="bgruening"> <description>OSRA - Optical Structure Recognition Application</description> <long_description>OSRA is a utility designed to convert graphical representations of chemical structures, as they appear in journal articles, patent documents, textbooks, trade magazines etc., into SMILES.</long_description> <archive>osra-e5e3d723a675.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="confab" type="unrestricted" username="bgruening"> <description>Systematic conformation generator</description> <long_description>Confab is an open source conformation generator whose goal is the systematic coverage of conformational space.</long_description> <archive>confab-df8c4f142ae3.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_boost_1_53" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.53 of the boost C++ libraries</description> <long_description>Boost provides free peer-reviewed portable C++ source libraries. http://www.boost.org/</long_description> <archive>package_boost_1_53-f5b13e56a321.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="molecule_datatypes" type="unrestricted" username="bgruening"> <description>Datatypes and converter for cheminformatics</description> <long_description>Molecule datatypes and converters. Including SDF, MOL, SMILES, InChI, MOL2, PDB ...</long_description> <archive>molecule_datatypes-18ff2d6e1dca.tar.gz</archive> <categories> <category>Computational chemistry</category> <category>Convert Formats</category> </categories> </repository>
<repository name="package_numpy_1_7" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.7 of the the python numpy package</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.7 of numpy. www.numpy.org/
NumPy is the fundamental package for scientific computing with Python.</long_description> <archive>package_numpy_1_7-40db34c904df.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="openbabel" type="unrestricted" username="bgruening"> <description>Open Babel tools</description> <long_description>Open Babel is a chemical toolbox designed to speak the many languages of chemical data. It's an open, collaborative project allowing anyone to search, convert, analyze, or store data from molecular modeling, chemistry, solid-state materials, biochemistry, or related areas. </long_description> <archive>openbabel-b4ba90dcfde0.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_scipy_0_12" type="tool_dependency_definition" username="bgruening"> <description>Scientific Tools for Python</description> <long_description>SciPy is open-source software for mathematics, science, and engineering. The SciPy library is built to work with NumPy arrays, and provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization.</long_description> <archive>package_scipy_0_12-cd2e7872a8eb.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_matplotlib_1_2" type="unrestricted" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 1.2.x of the the python matplotlib package.</description> <long_description>This repository is intended to be defined as a complex repository dependency within a separate repository. http://wiki.galaxyproject.org/DefiningRepositoryDependencies#Complex_reposit...
This is the version 1.2.x of matplotlib. www.matplotlib.org/
matplotlib is a python 2D plotting library which produces publication quality figures.</long_description> <archive>package_matplotlib_1_2-dd81b71dff1b.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="package_scikit_learn_0_13" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 0.13.x of the the scikit-learn package.</description> <long_description>Easy-to-use and general-purpose machine learning in Python
Scikit-learn integrates machine learning algorithms in the tightly-knit scientific Python world, building upon numpy, scipy, and matplotlib. As a machine-learning module, it provides versatile tools for data mining and analysis in any field of science and engineering. It strives to be simple and efficient, accessible to everybody, and reusable in various contexts.
http://scikit-learn.org/</long_description> <archive>package_scikit_learn_0_13-ad4341b759f2.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="ctb_machine_learning" type="unrestricted" username="bgruening"> <description>machine learning scripts for the chemicaltoolbox</description> <long_description>This repository contains small tools dealing with machine learning techniques inside the chemicaltoolbox.</long_description> <archive>ctb_machine_learning-bf0014ee0871.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="opsin" type="unrestricted" username="bgruening"> <description>Open Parser for Systematic IUPAC nomenclature</description> <long_description>OPSIN is library for IUPAC name-to-structure conversion offering high recall and precision on organic chemical nomenclature. https://bitbucket.org/dan2097/opsin</long_description> <archive>opsin-ee73c8d529b1.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="package_rdkit_2012_12" type="tool_dependency_definition" username="bgruening"> <description>Contains a tool dependency definition that downloads and compiles version 2012-12 of the RDKit cheminformatics and machine-learning package.</description> <long_description>A collection of cheminformatics and machine-learning software written in C++ and Python.
The core algorithms and data structures are written in C++. Wrappers are provided to use the toolkit from either Python or Java.
Additionally, the RDKit distribution includes a PostgreSQL-based cartridge that allows molecules to be stored in relational database and retrieved via substructure and similarity searches. http://www.rdkit.org/</long_description> <archive>package_rdkit_2012_12-4663c03127f7.tar.gz</archive> <categories> <category>Tool Dependency Packages</category> </categories> </repository>
<repository name="rdkit" type="unrestricted" username="bgruening"> <description>RDKit tools.</description> <long_description>Various tools that are using the RDKit library.</long_description> <archive>rdkit-aa46f08461f9.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemfp" type="unrestricted" username="bgruening"> <description>Chemfp tools.</description> <long_description>chemfp is a set of tools for working with cheminformatics fingerprints in the FPS format.</long_description> <archive>chemfp-ee02ddefe21b.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="silicos_it" type="unrestricted" username="bgruening"> <description>silicos</description> <archive>silicos_it-1c218c2a4bd8.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="chemicaltoolbox" type="unrestricted" username="bgruening"> <description>ChemicalToolBoX Metapackage</description> <long_description>That metapackage will install several cheminformatic tool wrappers (e.g. chemfp, openbabel, osra, confab, rdkit)</long_description> <archive>chemicaltoolbox-4133dbf7ff4d.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
<repository name="simsearch" type="unrestricted" username="bgruening"> <description>Similarity search with openbabel or chemfp</description> <long_description>Similarity search with openbabel or chemfp</long_description> <archive>simsearch-b4eaf75cac77.tar.gz</archive> <categories> <category>Computational chemistry</category> </categories> </repository>
On Aug 7, 2013, at 10:28 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Ok, time to step up and migrate all the stuff to main.
Cheers, Bjoern
Hi Bjoern, Was this reply on the wrong thread - or is there something here related to the BLAST+ binaries? Thanks, Peter On Wed, Aug 14, 2013 at 10:29 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Hi Greg,
I tried the manifest file and saw that the ordering is not correct. You can see that on the last to repositories. Since, chemicaltoolbox is the meta_repository it should be always the last one.
Cheers, Bjoern
Hello Bjoern,
I have the export repository ( and optionally all repoository dependencies ) feature working on the test tool shed. The feature allows you to chose a specific installable revision (if in the browser, by being on the Manage repository page for that revision or if using the API by passing the revision on the command line) for export.
<cut>
Hi Peter, no nothing BLAST+ related. Greg highjacked that thread to inform me about the new export functionality. I will stop posting here. Cheers, Bjoern
Hi Bjoern,
Was this reply on the wrong thread - or is there something here related to the BLAST+ binaries?
Thanks,
Peter
On Wed, Aug 14, 2013 at 10:29 AM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Hi Greg,
I tried the manifest file and saw that the ordering is not correct. You can see that on the last to repositories. Since, chemicaltoolbox is the meta_repository it should be always the last one.
Cheers, Bjoern
Hello Bjoern,
I have the export repository ( and optionally all repoository dependencies ) feature working on the test tool shed. The feature allows you to chose a specific installable revision (if in the browser, by being on the Manage repository page for that revision or if using the API by passing the revision on the command line) for export.
<cut>
Il giorno mer, 07/08/2013 alle 16.28 +0200, Bjoern Gruening ha scritto:
Also, we may wish to add a dependency on BOOST, e.g. http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 once this is on the main Tool Shed.
Ok, time to step up and migrate all the stuff to main.
Thanks Bjoern, I'd make us of it also for another tool I'm preparing (a wrapper for Mugsy multiple aligner). Nicola -- Nicola Soranzo, Ph.D. CRS4 Bioinformatics Program Loc. Piscina Manna 09010 Pula (CA), Italy http://www.bioinformatica.crs4.it/
Hi Nicola!
Il giorno mer, 07/08/2013 alle 16.28 +0200, Bjoern Gruening ha scritto:
Also, we may wish to add a dependency on BOOST, e.g. http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 once this is on the main Tool Shed.
Ok, time to step up and migrate all the stuff to main.
Thanks Bjoern, I'd make us of it also for another tool I'm preparing (a wrapper for Mugsy multiple aligner).
Nicola
Forgot to mention that most of my tool dependency packages are now in the Main Tool Shed under the IUC account. If I forget something to migrate, please ping me. A complete list can be found here: https://github.com/bgruening/galaxytools/tree/master/orphan_tool_dependencie... Cheers, Bjoern
Hi Peter, On Aug 7, 2013, at 10:26 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg et al,
For the NCBI BLAST+ wrappers, Nicola has suggested we split out the binaries themselves into a separate Tool Shed package, e.g. package_blast_plus_2_2_26 and similar for the later releases.
https://github.com/peterjc/galaxy_blast/issues/7
I think this is a good idea.
I agree.
I'd like to check if there are any complications foreseeable from the fact that the NCBI BLAST+ wrappers were migrated out of the Galaxy core and therefore have migration scripts we need to consider.
I don't think this change will affect the Galaxy tool migration process for the ncbi_blast_plus repository since the migration script defines revision d375502056f1 of the repository. SInce that is an installable revision, the migration process will install that revision only, and additional revisions of the repository requires separate installation.
Potentially this could be held under the shared IUC account as planned for other commonly used packages? Or would the migration issues require it to also live under devteam?
I think using hte iuc account would be best - again, this should have no effect on the migration process.
Also, we may wish to add a dependency on BOOST, e.g. http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 once this is on the main Tool Shed.
Sounds great!
Regards,
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Wed, Aug 7, 2013 at 3:36 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Peter,
On Aug 7, 2013, at 10:26 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg et al,
For the NCBI BLAST+ wrappers, Nicola has suggested we split out the binaries themselves into a separate Tool Shed package, e.g. package_blast_plus_2_2_26 and similar for the later releases.
https://github.com/peterjc/galaxy_blast/issues/7
I think this is a good idea.
I agree.
Excellent, we'll start work on that.
I'd like to check if there are any complications foreseeable from the fact that the NCBI BLAST+ wrappers were migrated out of the Galaxy core and therefore have migration scripts we need to consider.
I don't think this change will affect the Galaxy tool migration process for the ncbi_blast_plus repository since the migration script defines revision d375502056f1 of the repository. SInce that is an installable revision, the migration process will install that revision only, and additional revisions of the repository requires separate installation.
OK, good.
Potentially this could be held under the shared IUC account as planned for other commonly used packages? Or would the migration issues require it to also live under devteam?
I think using the iuc account would be best - again, this should have no effect on the migration process.
OK then, I'll setup these using the current tool_dependencies.xml from the ncbi_blast_plus suite (which right now is breaking on the Tool Shed due to its copy of BOOST being updated), first: - http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 And later once that works: - http://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 Does the Tool Shed test the installation of packages like this as part of the nightly tests? If not could that be added please?
Also, we may wish to add a dependency on BOOST, e.g. http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 once this is on the main Tool Shed.
Sounds great!
I'm sure Bjoern will be able to assist here - we'll get this working on the Test Tool Shed first, and then push it to the main Tool Shed (along with the other minor updates to the NCBI+ wrappers being held up by this). Thanks, Peter
On Wed, Aug 7, 2013 at 3:54 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Aug 7, 2013 at 3:36 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Peter,
On Aug 7, 2013, at 10:26 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg et al,
For the NCBI BLAST+ wrappers, Nicola has suggested we split out the binaries themselves into a separate Tool Shed package, e.g. package_blast_plus_2_2_26 and similar for the later releases.
https://github.com/peterjc/galaxy_blast/issues/7
I think this is a good idea.
I agree.
Excellent, we'll start work on that.
...
OK then, I'll setup these using the current tool_dependencies.xml from the ncbi_blast_plus suite (which right now is breaking on the Tool Shed due to its copy of BOOST being updated), first:
- http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26
Done, using just the tool_dependencies.xml file which was previously bundled within the ncbi_blast_plus Tool Shed repo: <?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> .... </readme> </package> </tool_dependency> Now, back in ncbi_blast_plus, I have revised the repository_dependencies.xml to declare this new repository dependency (the revision tag etc is missing to mean the latest version at upload is used): <?xml version="1.0"?> <repositories description="This requires the BLAST datatype definitions (e.g. the BLAST XML format) and the BLAST+ binaries"> <repository name="blast_datatypes" owner="devteam" /> <repository name="package_blast_plus_2_2_26" owner="iuc" /> </repositories> Now do I need to change the individual tools at all? e.g. <tool id="ncbi_blastn_wrapper" name="NCBI BLAST+ blastn" version="0.0.20"> <description>Search nucleotide database with nucleotide query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" shared_inputs="subject,histdb" merge_outputs="output1"></parallelism> <requirements> <requirement type="binary">blastn</requirement> <requirement type="package" version="2.2.26+">blast+</requirement> </requirements> .... The package name and version still match the (relocated) tool_dependencies.xml file, so this should be OK, right? Thanks, Peter
Hi Peter,
On Wed, Aug 7, 2013 at 3:54 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Aug 7, 2013 at 3:36 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Peter,
On Aug 7, 2013, at 10:26 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg et al,
For the NCBI BLAST+ wrappers, Nicola has suggested we split out the binaries themselves into a separate Tool Shed package, e.g. package_blast_plus_2_2_26 and similar for the later releases.
https://github.com/peterjc/galaxy_blast/issues/7
I think this is a good idea.
I agree.
Excellent, we'll start work on that.
...
OK then, I'll setup these using the current tool_dependencies.xml from the ncbi_blast_plus suite (which right now is breaking on the Tool Shed due to its copy of BOOST being updated), first:
- http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26
Done, using just the tool_dependencies.xml file which was previously bundled within the ncbi_blast_plus Tool Shed repo:
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> .... </readme> </package> </tool_dependency>
Now, back in ncbi_blast_plus, I have revised the repository_dependencies.xml to declare this new repository dependency (the revision tag etc is missing to mean the latest version at upload is used):
Mh, I never tried that but I think that will not work. The blast datatypes are ok to be included in repository_dependency.xml but the package_blast_plus needs to be in tool_dependency.xml, imho. <?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <repository name="package_blast_plus_2_2_26" owner="iuc" /> </package> </tool_dependency> If you have that you do not need to change your tools. Hopefully I'm correct. Bjoern
<?xml version="1.0"?> <repositories description="This requires the BLAST datatype definitions (e.g. the BLAST XML format) and the BLAST+ binaries"> <repository name="blast_datatypes" owner="devteam" /> <repository name="package_blast_plus_2_2_26" owner="iuc" /> </repositories>
Now do I need to change the individual tools at all? e.g.
<tool id="ncbi_blastn_wrapper" name="NCBI BLAST+ blastn" version="0.0.20"> <description>Search nucleotide database with nucleotide query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" shared_inputs="subject,histdb" merge_outputs="output1"></parallelism> <requirements> <requirement type="binary">blastn</requirement> <requirement type="package" version="2.2.26+">blast+</requirement> </requirements> ....
The package name and version still match the (relocated) tool_dependencies.xml file, so this should be OK, right?
Thanks,
Peter
On Wed, Aug 7, 2013 at 5:11 PM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Hi Peter,
Now, back in ncbi_blast_plus, I have revised the repository_dependencies.xml to declare this new repository dependency (the revision tag etc is missing to mean the latest version at upload is used):
Mh, I never tried that but I think that will not work. The blast datatypes are ok to be included in repository_dependency.xml but the package_blast_plus needs to be in tool_dependency.xml, imho.
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <repository name="package_blast_plus_2_2_26" owner="iuc" /> </package> </tool_dependency>
If you have that you do not need to change your tools.
Hopefully I'm correct. Bjoern
I think you're right, certainly that is how this example works: http://toolshed.g2.bx.psu.edu/view/jjohnson/samtools_filter So I don't need this in my repository_dependencies.xml after all... Thanks, Peter
On Wed, Aug 7, 2013 at 5:35 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Aug 7, 2013 at 5:11 PM, Bjoern Gruening <bjoern.gruening@gmail.com> wrote:
Hi Peter,
Now, back in ncbi_blast_plus, I have revised the repository_dependencies.xml to declare this new repository dependency (the revision tag etc is missing to mean the latest version at upload is used):
Mh, I never tried that but I think that will not work. The blast datatypes are ok to be included in repository_dependency.xml but the package_blast_plus needs to be in tool_dependency.xml, imho.
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <repository name="package_blast_plus_2_2_26" owner="iuc" /> </package> </tool_dependency>
If you have that you do not need to change your tools.
Hopefully I'm correct. Bjoern
I think you're right, certainly that is how this example works: http://toolshed.g2.bx.psu.edu/view/jjohnson/samtools_filter
So I don't need this in my repository_dependencies.xml after all...
Trying this out, http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/949fa0294c0d Thanks Peter
Hi Greg, I'm hoping you (or Dave) can throw a little light on why the NCBI BLAST+ nightly tests are not working again yet: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/949fa0294c0d e.g. Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found As of that commit, the ncbi_blast_plus repository depends on the new IUC repository package_blast_plus_2_2_26 which now depends on the IUC repository package_boost_1_53, which depends on Bjoern's package_bzlib_1_0, i.e. this chain: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus -> http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 -> http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 -> http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_bzlib_1_0 Sadly the (Test) Tool Shed isn't showing me any install logs, so I am hoping *you* have access to them. I've also requested individual package repositories (i.e. type "Tool dependency definition") be tested nightly for installation problems: https://trello.com/c/HVGrShnC/1042-tool-shed-should-test-installation-of-pac... Meanwhile, I have tried this locally - the dependencies seem to install, but the BLAST+ package says "Installed, missing tool dependencies". It looks like the BLAST+ make fails, but I don't know how/if I can see this from the Galaxy Admin web-interface. I can see the logs at the command line, $ tail -n 30 blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f/INSTALLATION.log make[1]: Leaving directory `/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/build' ############################################# ############################################# cd c++ && ./configure --with-boost=$BOOST_ROOT_DIR --prefix=/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f && make && make install STDERR basename: missing operand Try `basename --help' for more information. configure: WARNING: Untested Boost version; may prove incompatible. configure: WARNING: If so, please re-run this script with the flag --without-boost. make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/db/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/db/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/ctools/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/ctools/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/misc/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/misc/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/gui/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/gui/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/sample/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/sample/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/internal/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/internal/Makefile.in] Error 1 (ignored) make[2]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/corelib/test/Makefile.in] Error 1 (ignored) make[2]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/corelib/test/Makefile.in] Error 1 (ignored) /bin/sh: /usr/bin/touch: No such file or directory make[3]: *** [sources.usr] Error 127 make[2]: *** [sources.usr.locked] Error 2 make[1]: *** [all_r.real] Error 5 make: *** [all] Error 2 ############################################# I just checked the BOOST bit worked by manually compiling BLAST+, $ echo 'BOOST_ROOT_DIR=/opt/galaxy-dist-shed-tools/boost/1.53.0/iuc/package_boost_1_53/a72f8efe9201/boost; export BOOST_ROOT_DIR' $ ./configure --with-boost=$BOOST_ROOT_DIR --prefix=/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f ... ******* CONFIGURATION SUCCESSFUL ******* $ make ... /bin/sh: /usr/bin/touch: No such file or directory ... make: *** [all] Error 2 So sign of the basename error, but the touch problem persists. Fixed with: $ sudo ln -s /bin/touch /usr/bin/touch Having created the touch alias, manual compilation appears to be working (it just takes a long long time). I intend to repeat this via the Galaxy Admin page ... However, this doesn't help with knowing what might be going wrong during the (Test) Tool Shed nightly tests. Peter
On Wed, Aug 14, 2013 at 11:56 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg,
I'm hoping you (or Dave) can throw a little light on why the NCBI BLAST+ nightly tests are not working again yet:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/949fa0294c0d
e.g. Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found
As of that commit, the ncbi_blast_plus repository depends on the new IUC repository package_blast_plus_2_2_26 which now depends on the IUC repository package_boost_1_53, which depends on Bjoern's package_bzlib_1_0, i.e. this chain:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus -> http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 -> http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 -> http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_bzlib_1_0
Sadly the (Test) Tool Shed isn't showing me any install logs, so I am hoping *you* have access to them.
I've also requested individual package repositories (i.e. type "Tool dependency definition") be tested nightly for installation problems: https://trello.com/c/HVGrShnC/1042-tool-shed-should-test-installation-of-pac...
Meanwhile, I have tried this locally - the dependencies seem to install, but the BLAST+ package says "Installed, missing tool dependencies". It looks like the BLAST+ make fails, but I don't know how/if I can see this from the Galaxy Admin web-interface.
I can see the logs at the command line,
$ tail -n 30 blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f/INSTALLATION.log make[1]: Leaving directory `/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/build' #############################################
############################################# cd c++ && ./configure --with-boost=$BOOST_ROOT_DIR --prefix=/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f && make && make install STDERR basename: missing operand Try `basename --help' for more information. configure: WARNING: Untested Boost version; may prove incompatible. configure: WARNING: If so, please re-run this script with the flag --without-boost. make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/db/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/db/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/ctools/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/ctools/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/misc/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/misc/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/gui/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/gui/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/sample/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/sample/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/internal/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/internal/Makefile.in] Error 1 (ignored) make[2]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/corelib/test/Makefile.in] Error 1 (ignored) make[2]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/corelib/test/Makefile.in] Error 1 (ignored) /bin/sh: /usr/bin/touch: No such file or directory make[3]: *** [sources.usr] Error 127 make[2]: *** [sources.usr.locked] Error 2 make[1]: *** [all_r.real] Error 5 make: *** [all] Error 2 #############################################
I just checked the BOOST bit worked by manually compiling BLAST+,
$ echo 'BOOST_ROOT_DIR=/opt/galaxy-dist-shed-tools/boost/1.53.0/iuc/package_boost_1_53/a72f8efe9201/boost; export BOOST_ROOT_DIR' $ ./configure --with-boost=$BOOST_ROOT_DIR --prefix=/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f ... ******* CONFIGURATION SUCCESSFUL ******* $ make ... /bin/sh: /usr/bin/touch: No such file or directory ... make: *** [all] Error 2
So sign of the basename error, but the touch problem persists. Fixed with:
$ sudo ln -s /bin/touch /usr/bin/touch
Having created the touch alias, manual compilation appears to be working (it just takes a long long time). I intend to repeat this via the Galaxy Admin page ...
Failed to compile, ... make[4]: Entering directory `/home/pjcock/Downloads/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/build/app/blast' /usr/bin/g++ -c -Wall -Wno-format-y2k -g -fPIC -D_DEBUG -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -DNCBI_WITHOUT_MT -I/home/pjcock/Downloads/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/inc -I/home/pjcock/Downloads/ncbi-blast-2.2.26+-src/c++/include /home/pjcock/Downloads/ncbi-blast-2.2.26+-src/c++/src/app/blast/blastn_app.cpp -o blastn_app.o /usr/bin/g++ -Wl,--enable-new-dtags -g -Wl,-E blastn_app.o -Wl,-rpath,/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f/lib -Wl,-rpath-link,/home/pjcock/Downloads/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/lib -L/home/pjcock/Downloads/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/lib -lblast_app_util -lblastinput-static -lncbi_xloader_blastdb-static -lncbi_xloader_blastdb_rmt-static -lxblastformat-static -lalign_format-static -ltaxon1-static -lblastdb_format-static -lgene_info-static -lxalnmgr-static -lblastxml-static -lxcgi-static -lxhtml-static -lxblast-static -lxalgoblastdbindex-static -lcomposition_adjustment-static -lxalgodustmask-static -lxalgowinmask-static -lseqmasks_io-static -lseqdb-static -lblast_services-static -lxobjutil-static -lxobjread-static -lcreaders-static -lxnetblastcli-static -lxnetblast-static -lblastdb-static -lscoremat-static -ltables-static -lxalnmgr-static -lncbi_xloader_genbank-static -lncbi_xreader_id1-static -lncbi_xreader_id2-static -lncbi_xreader_cache-static -lxconnext-static -lxconnect-static -ldbapi_driver-static -lncbi_xreader-static -lid1-static -lid2-static -lseqsplit-static -lxcompress-static -lxobjmgr-static -lgenome_collection-static -lseqset-static -lseqedit-static -lseq-static -lseqcode-static -lsequtil-static -lpub-static -lmedline-static -lbiblio-static -lgeneral-static -lxser-static -lxutil-static -lxncbi-static -lz -lbz2 -ldl -lnsl -lrt -lm -o blastn collect2: ld returned 1 exit status make[4]: *** [blastn] Error 1 Presumably this machine is missing one dependency or other?
However, this doesn't help with knowing what might be going wrong during the (Test) Tool Shed nightly tests.
I'm still curious how this is failing on the Test Tool Shed. However, I'm going back to the binary download route... it just seems like a more robust & reproducible solution all round: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26/c85236c... This seems to work OK for 64 bit Linux, but once again, fingers crossed for the next nightly Tool Shed tests... Peter
On Wed, Aug 14, 2013 at 2:25 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Aug 14, 2013 at 11:56 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg,
I'm hoping you (or Dave) can throw a little light on why the NCBI BLAST+ nightly tests are not working again yet:
...
Sadly the (Test) Tool Shed isn't showing me any install logs, so I am hoping *you* have access to them.
...
However, I'm going back to the binary download route... it just seems like a more robust & reproducible solution all round:
http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26/c85236c...
This seems to work OK for 64 bit Linux, but once again, fingers crossed for the next nightly Tool Shed tests...
The new package_blast_plus_2_2_26 binary installer seems to be working for me and Bjoern, but the Test Tool Shed tests for ncbi_blastn_wrapper are still failing, e.g. Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found Could you access the Test Tool Shed logs for this please to help track us down what is going wrong? Thanks, Peter
Peter, I believe this was again due to the installation of an earlier repository hanging until the testing framework canceled the build. I've added the repository in question to the exclude list, and hopefully your blast repositories will be properly tested within the next few hours. --Dave B. On 8/16/13 04:51:34.000, Peter Cock wrote:
On Wed, Aug 14, 2013 at 2:25 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Aug 14, 2013 at 11:56 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi Greg,
I'm hoping you (or Dave) can throw a little light on why the NCBI BLAST+ nightly tests are not working again yet:
...
Sadly the (Test) Tool Shed isn't showing me any install logs, so I am hoping *you* have access to them.
...
However, I'm going back to the binary download route... it just seems like a more robust & reproducible solution all round:
http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26/c85236c...
This seems to work OK for 64 bit Linux, but once again, fingers crossed for the next nightly Tool Shed tests...
The new package_blast_plus_2_2_26 binary installer seems to be working for me and Bjoern, but the Test Tool Shed tests for ncbi_blastn_wrapper are still failing, e.g.
Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found
Could you access the Test Tool Shed logs for this please to help track us down what is going wrong?
Thanks,
Peter
On Tue, Aug 20, 2013 at 9:15 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
I believe this was again due to the installation of an earlier repository hanging until the testing framework canceled the build. I've added the repository in question to the exclude list, and hopefully your blast repositories will be properly tested within the next few hours.
--Dave B.
Thanks Dave, Re: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/946749c8605f The output has changed, but we're back to missing any test results at all now, all I get is: ... Automated tool test results Tool test results Automated test environment Time tested: ~ 10 hours ago System: Linux 3.8.0-25-generic Architecture: x86_64 Python version: 2.7.4 Galaxy revision: 10419:4f4e01316260 Galaxy database version: 115 Tool shed revision: 10413:6e148e87d819 Tool shed database version: 21 Tool shed mercurial version: 2.2.3 Automated tool tests ... Peter
On Wed, Aug 21, 2013 at 10:54 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, Aug 20, 2013 at 9:15 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
I believe this was again due to the installation of an earlier repository hanging until the testing framework canceled the build. I've added the repository in question to the exclude list, and hopefully your blast repositories will be properly tested within the next few hours.
--Dave B.
Thanks Dave,
Re: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/946749c8605f
The output has changed, but we're back to missing any test results at all now, all I get is:
... Automated tool test results
Tool test results Automated test environment Time tested: ~ 10 hours ago System: Linux 3.8.0-25-generic Architecture: x86_64 Python version: 2.7.4 Galaxy revision: 10419:4f4e01316260 Galaxy database version: 115 Tool shed revision: 10413:6e148e87d819 Tool shed database version: 21 Tool shed mercurial version: 2.2.3
Automated tool tests ...
Test results now visible now (odd?), back to the install problem: Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found Any clues from the Test Tool Shed log file would be helpful, Thanks, Peter
Peter, Here is the relevant log output for the installation of ncbi_blast_plus. A quick google informs me that exit code 64 means "command line usage error", for what it's worth. install_and_test_repositories INFO 2013-08-20 23:34:43,270 Installing and testing revision 946749c8605f of repository ncbi_blast_plus owned by peterjc... install_repository_ncbi_blast_plus (install_and_test_tool_shed_repositories.functional.test_install_repositories.TestInstallRepository_ncbi_blast_plus) Install the repository ncbi_blast_plus from http://testtoolshed.g2.bx.psu.edu. ... tool_shed.util.shed_util_common DEBUG 2013-08-20 23:34:47,484 Adding new row (or updating an existing row) for repository 'ncbi_blast_plus' in the tool_shed_repository table, status set to 'New'. tool_shed.util.shed_util_common DEBUG 2013-08-20 23:34:47,674 Adding new row (or updating an existing row) for repository 'blast_datatypes' in the tool_shed_repository table, status set to 'New'. tool_shed.util.shed_util_common DEBUG 2013-08-20 23:34:47,886 Adding new row (or updating an existing row) for repository 'package_blast_plus_2_2_26' in the tool_shed_repository table, status set to 'New'. Warning: local() encountered an error (return code 64) while executing ' if [[ "$(uname)" == "Linux" ]]; then export FILENAME="ncbi-blast-2.2.26+-ia32-linux.tar.gz"; fi && if [[ "$(arch)" == "x86_64" ]]; then export FILENAME="ncbi-blast-2.2.26+-x64-linux.tar.gz"; fi && if [[ "$(uname)" == "Darwin" ]]; then export FILENAME="ncbi-blast-2.2.26+-universal-macosx.tar.gz"; fi && echo Fetching $FILENAME && if [[ "$(uname)" == "Linux" ]]; then wget ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/$FILENAME; fi && if [[ "$(uname)" == "Darwin" ]]; then curl -O ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/$FILENAME; fi && tar -zxvf $FILENAME && echo Downloaded and decompressed ' galaxy.datatypes.registry DEBUG 2013-08-20 23:34:52,010 Loading datatypes from /var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpskjEfq/tmp-toolshed-acalpdoM4u1f galaxy.datatypes.registry DEBUG 2013-08-20 23:34:52,022 Loaded sniffer for datatype 'galaxy.datatypes.blast:BlastXml' tool_shed.util.tool_util DEBUG 2013-08-20 23:34:54,285 The '/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/tool-data/testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/946749c8605f/tool_data_table_conf.xml' data table file was not found, but was expected to be copied from 'tool_data_table_conf.xml.sample' during repository installation. galaxy.tools DEBUG 2013-08-20 23:34:55,359 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_blastx_wrapper/0.0.19, version: 0.0.19 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,435 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_tblastx_wrapper/0.0.20, version: 0.0.20 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,510 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_tblastn_wrapper/0.0.20, version: 0.0.20 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,541 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_blastdbcmd_info/0.0.6, version: 0.0.6 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,618 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_blastp_wrapper/0.0.20, version: 0.0.20 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,693 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_rpstblastn_wrapper/0.0.4, version: 0.0.4 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,722 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_makeblastdb/0.0.5, version: 0.0.5 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,797 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_blastn_wrapper/0.0.20, version: 0.0.20 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,861 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/blastxml_to_tabular/0.0.11, version: 0.0.11 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,938 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_rpsblast_wrapper/0.0.4, version: 0.0.4 into tool panel. galaxy.tools DEBUG 2013-08-20 23:34:55,971 Loaded tool id: testtoolshed.g2.bx.psu.edu/repos/peterjc/ncbi_blast_plus/ncbi_blastdbcmd_wrapper/0.0.6, version: 0.0.6 into tool panel. --Dave B. On 8/21/13 08:08:49.000, Peter Cock wrote:
On Wed, Aug 21, 2013 at 10:54 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, Aug 20, 2013 at 9:15 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
I believe this was again due to the installation of an earlier repository hanging until the testing framework canceled the build. I've added the repository in question to the exclude list, and hopefully your blast repositories will be properly tested within the next few hours.
--Dave B.
Thanks Dave,
Re: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/946749c8605f
The output has changed, but we're back to missing any test results at all now, all I get is:
... Automated tool test results
Tool test results Automated test environment Time tested: ~ 10 hours ago System: Linux 3.8.0-25-generic Architecture: x86_64 Python version: 2.7.4 Galaxy revision: 10419:4f4e01316260 Galaxy database version: 115 Tool shed revision: 10413:6e148e87d819 Tool shed database version: 21 Tool shed mercurial version: 2.2.3
Automated tool tests ...
Test results now visible now (odd?), back to the install problem:
Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found
Any clues from the Test Tool Shed log file would be helpful,
Thanks,
Peter
On Wed, Aug 21, 2013 at 1:45 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
Here is the relevant log output for the installation of ncbi_blast_plus. A quick google informs me that exit code 64 means "command line usage error", for what it's worth.
install_and_test_repositories INFO 2013-08-20 23:34:43,270 Installing and testing revision 946749c8605f of repository ncbi_blast_plus owned by peterjc... install_repository_ncbi_blast_plus (install_and_test_tool_shed_repositories.functional.test_install_repositories.TestInstallRepository_ncbi_blast_plus) Install the repository ncbi_blast_plus from http://testtoolshed.g2.bx.psu.edu. ... tool_shed.util.shed_util_common DEBUG 2013-08-20 23:34:47,484 Adding new row (or updating an existing row) for repository 'ncbi_blast_plus' in the tool_shed_repository table, status set to 'New'. tool_shed.util.shed_util_common DEBUG 2013-08-20 23:34:47,674 Adding new row (or updating an existing row) for repository 'blast_datatypes' in the tool_shed_repository table, status set to 'New'. tool_shed.util.shed_util_common DEBUG 2013-08-20 23:34:47,886 Adding new row (or updating an existing row) for repository 'package_blast_plus_2_2_26' in the tool_shed_repository table, status set to 'New'.
Warning: local() encountered an error (return code 64) while executing ' if [[ "$(uname)" == "Linux" ]]; then export FILENAME="ncbi-blast-2.2.26+-ia32-linux.tar.gz"; fi && if [[ "$(arch)" == "x86_64" ]]; then export FILENAME="ncbi-blast-2.2.26+-x64-linux.tar.gz"; fi && if [[ "$(uname)" == "Darwin" ]]; then export FILENAME="ncbi-blast-2.2.26+-universal-macosx.tar.gz"; fi && echo Fetching $FILENAME && if [[ "$(uname)" == "Linux" ]]; then wget ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/$FILENAME; fi && if [[ "$(uname)" == "Darwin" ]]; then curl -O ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/$FILENAME; fi && tar -zxvf $FILENAME && echo Downloaded and decompressed '
Thanks Dave, I've made a note on this Trello card about exposing the log information of unsuccessful builds on the Tool Shed, https://trello.com/c/HVGrShnC/1042-tool-shed-should-test-installation-of-pac... Can you tell me what version of Linux this is, and which shell? $ uname ... $ arch ... $ echo $0 ... That might give me a clue about the nature of the failure. My hunch is that I am using something bash specific. The simplest solution would be if you could run each of these command by hand to see if it triggers the same error: if [[ "$(uname)" == "Linux" ]]; then export FILENAME="ncbi-blast-2.2.26+-ia32-linux.tar.gz"; fi if [[ "$(arch)" == "x86_64" ]]; then export FILENAME="ncbi-blast-2.2.26+-x64-linux.tar.gz"; fi if [[ "$(uname)" == "Darwin" ]]; then export FILENAME="ncbi-blast-2.2.26+-universal-macosx.tar.gz"; fi echo Fetching $FILENAME if [[ "$(uname)" == "Linux" ]]; then wget ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/$FILENAME; fi if [[ "$(uname)" == "Darwin" ]]; then curl -O ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/$FILENAME; fi tar -zxvf $FILENAME echo Downloaded and decompressed Thanks, Peter
Peter, The problem with testing the installation of package tool dependencies, especially but not limited to BLAST+, is that the time it takes to compile them exceeds the timeout for the automated testing framework, and so it terminates that build step. I am currently working on enhancing the actions tag to support a different install process depending on the target architecture, and hope to have that ready early next week. --Dave B. On 8/14/13 06:56:35.000, Peter Cock wrote:
Hi Greg,
I'm hoping you (or Dave) can throw a little light on why the NCBI BLAST+ nightly tests are not working again yet:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/949fa0294c0d
e.g. Fatal error: Exit code 127 () /bin/sh: 1: blastn: not found /bin/sh: 1: blastn: not found
As of that commit, the ncbi_blast_plus repository depends on the new IUC repository package_blast_plus_2_2_26 which now depends on the IUC repository package_boost_1_53, which depends on Bjoern's package_bzlib_1_0, i.e. this chain:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus -> http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 -> http://testtoolshed.g2.bx.psu.edu/view/iuc/package_boost_1_53 -> http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_bzlib_1_0
Sadly the (Test) Tool Shed isn't showing me any install logs, so I am hoping *you* have access to them.
I've also requested individual package repositories (i.e. type "Tool dependency definition") be tested nightly for installation problems: https://trello.com/c/HVGrShnC/1042-tool-shed-should-test-installation-of-pac...
Meanwhile, I have tried this locally - the dependencies seem to install, but the BLAST+ package says "Installed, missing tool dependencies". It looks like the BLAST+ make fails, but I don't know how/if I can see this from the Galaxy Admin web-interface.
I can see the logs at the command line,
$ tail -n 30 blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f/INSTALLATION.log make[1]: Leaving directory `/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/GCC412-Debug64/build' #############################################
############################################# cd c++ && ./configure --with-boost=$BOOST_ROOT_DIR --prefix=/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f && make && make install STDERR basename: missing operand Try `basename --help' for more information. configure: WARNING: Untested Boost version; may prove incompatible. configure: WARNING: If so, please re-run this script with the flag --without-boost. make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/db/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/db/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/ctools/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/ctools/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/misc/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/misc/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/gui/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/gui/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/sample/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/sample/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/internal/Makefile.in] Error 1 (ignored) make[1]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/internal/Makefile.in] Error 1 (ignored) make[2]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/corelib/test/Makefile.in] Error 1 (ignored) make[2]: [/opt/galaxy-dist/database/tmp/tmp-toolshed-mtdutx4hN/ncbi-blast-2.2.26+-src/c++/src/corelib/test/Makefile.in] Error 1 (ignored) /bin/sh: /usr/bin/touch: No such file or directory make[3]: *** [sources.usr] Error 127 make[2]: *** [sources.usr.locked] Error 2 make[1]: *** [all_r.real] Error 5 make: *** [all] Error 2 #############################################
I just checked the BOOST bit worked by manually compiling BLAST+,
$ echo 'BOOST_ROOT_DIR=/opt/galaxy-dist-shed-tools/boost/1.53.0/iuc/package_boost_1_53/a72f8efe9201/boost; export BOOST_ROOT_DIR' $ ./configure --with-boost=$BOOST_ROOT_DIR --prefix=/opt/galaxy-dist-shed-tools/blast+/2.2.26+/iuc/package_blast_plus_2_2_26/9ff9bfdcd04f ... ******* CONFIGURATION SUCCESSFUL ******* $ make ... /bin/sh: /usr/bin/touch: No such file or directory ... make: *** [all] Error 2
So sign of the basename error, but the touch problem persists. Fixed with:
$ sudo ln -s /bin/touch /usr/bin/touch
Having created the touch alias, manual compilation appears to be working (it just takes a long long time). I intend to repeat this via the Galaxy Admin page ...
However, this doesn't help with knowing what might be going wrong during the (Test) Tool Shed nightly tests.
Peter _______________________________________________ galaxy-iuc mailing list galaxy-iuc@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-iuc
On Wed, Aug 14, 2013 at 2:32 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
The problem with testing the installation of package tool dependencies, especially but not limited to BLAST+, is that the time it takes to compile them exceeds the timeout for the automated testing framework, and so it terminates that build step. I am currently working on enhancing the actions tag to support a different install process depending on the target architecture, and hope to have that ready early next week.
--Dave B.
That's useful (and probably explains quite a few 'silent failures'). Could you make sure this time out failure and other install failures are visible on the Tool Shed as part of the test results? I think that is more urgent than tweaking the install target architecture. Thanks, Peter
Peter, I've created a Trello card for tracking the status of this issue. https://trello.com/c/32l2NZRn/1048-toolshed-investigate-possibility-of-recor... --Dave B. On 8/14/13 09:40:03.000, Peter Cock wrote:
On Wed, Aug 14, 2013 at 2:32 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
The problem with testing the installation of package tool dependencies, especially but not limited to BLAST+, is that the time it takes to compile them exceeds the timeout for the automated testing framework, and so it terminates that build step. I am currently working on enhancing the actions tag to support a different install process depending on the target architecture, and hope to have that ready early next week.
--Dave B.
That's useful (and probably explains quite a few 'silent failures').
Could you make sure this time out failure and other install failures are visible on the Tool Shed as part of the test results? I think that is more urgent than tweaking the install target architecture.
Thanks,
Peter
participants (5)
-
Bjoern Gruening -
Dave Bouvier -
Greg Von Kuster -
Nicola Soranzo -
Peter Cock