How do I add Ray to Galaxy Central in the tool shed ?
Hello, I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy. Ray is GPLv3 [1]. I have read http://wiki.galaxyproject.org/Admin/Tools/Add%20Tool%20Tutorial and I understand that I have to write a Tool Definition File in XML. I also visited the source code here: https://bitbucket.org/galaxy/galaxy-central/src/5de636ce68faededc52d7b0c1616... I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API. But where is stored the executable (in my case, where is sdtored Ray) ? Does Galaxy include the specs to build all the tools available in Galaxy-Central ? --- [1] http://denovoassembler.sourceforge.net/
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page: http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too... Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
On 06/06/13 02:32 PM, James Taylor wrote:
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
So we are at least two !
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page:
http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too...
Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
So Ray needs a C++ 1998 compiler and a MPI library (let's say g++ and openmpi-bin). Are those already installed in Galaxy deployments ? Also, when is tool_dependencies.xml processed (I assume it is procesed when re-loading the content of the shed into Galaxy ? Thanks !
Am Donnerstag, den 06.06.2013, 14:42 -0400 schrieb Sébastien Boisvert:
On 06/06/13 02:32 PM, James Taylor wrote:
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
So we are at least two !
Joining the party! Make three out of it!
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page:
http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too...
Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
So Ray needs a C++ 1998 compiler and a MPI library (let's say g++ and openmpi-bin). Are those already installed in Galaxy deployments ?
You can assume that gcc is installed. Nevertheless, openmpi-bin needs to be installed. I would suggest to create a orphan-tool-dependency for openmpi-bin. For example have a look at: http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_boost_1_53 And than make that orphan-tool-dep a requirement of your repository.
Also, when is tool_dependencies.xml processed (I assume it is procesed when re-loading the content of the shed into Galaxy ?
I don't understand the question, but if you install Ray the tool_dependency file is read and at that time it will be installed and the openmpi dependency. Cheers, Björn
Thanks ! ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
On 06/06/13 02:58 PM, Björn Grüning wrote:
Am Donnerstag, den 06.06.2013, 14:42 -0400 schrieb Sébastien Boisvert:
On 06/06/13 02:32 PM, James Taylor wrote:
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
So we are at least two !
Joining the party! Make three out of it!
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page:
http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too...
Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
So Ray needs a C++ 1998 compiler and a MPI library (let's say g++ and openmpi-bin). Are those already installed in Galaxy deployments ?
You can assume that gcc is installed. Nevertheless, openmpi-bin needs to be installed. I would suggest to create a orphan-tool-dependency for openmpi-bin. For example have a look at: http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_boost_1_53
And than make that orphan-tool-dep a requirement of your repository.
Also, when is tool_dependencies.xml processed (I assume it is procesed when re-loading the content of the shed into Galaxy ?
I don't understand the question,
On some platforms (like DNANexus), when you launch an app, it launches an instance from an image (let's say Debian). Then, the system install the software (let's say MUMmer) on the instance, fetch the data (from S3 or Azure or something else), launch the software (let's say MUMmer). When the software completes, the results are pushed by to cloud storage, and the instance is killed. So in this kind of platform, the installation occurs independently for each job you launch. I don't know if Galaxy works like that -- that was my question (I am reading the 3 important Galaxy papers today !).
but if you install Ray the tool_dependency file is read and at that time it will be installed and the openmpi dependency.
So the question becomes "When Ray get installed ?" Answers may include: A) Ray is installed once by a sysadmin. B) Ray is installed automatically by Galaxy when it starts if Ray is not already installed. C) Ray is installed each time someone uses it because an instance is spawned. D) Ray is not installed because it ships with the operating system, just like ls, pwd and so on. E) the Galaxy user needs to upload a static Ray executable just like he/she uploads their data.
Cheers, Björn
-Sébastien.
Thanks ! ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
Am Donnerstag, den 06.06.2013, 15:19 -0400 schrieb Sébastien Boisvert:
On 06/06/13 02:58 PM, Björn Grüning wrote:
Am Donnerstag, den 06.06.2013, 14:42 -0400 schrieb Sébastien Boisvert:
On 06/06/13 02:32 PM, James Taylor wrote:
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
So we are at least two !
Joining the party! Make three out of it!
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page:
http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too...
Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
So Ray needs a C++ 1998 compiler and a MPI library (let's say g++ and openmpi-bin). Are those already installed in Galaxy deployments ?
You can assume that gcc is installed. Nevertheless, openmpi-bin needs to be installed. I would suggest to create a orphan-tool-dependency for openmpi-bin. For example have a look at: http://testtoolshed.g2.bx.psu.edu/view/bgruening/package_boost_1_53
And than make that orphan-tool-dep a requirement of your repository.
Also, when is tool_dependencies.xml processed (I assume it is procesed when re-loading the content of the shed into Galaxy ?
I don't understand the question,
On some platforms (like DNANexus), when you launch an app, it launches an instance from an image (let's say Debian). Then, the system install the software (let's say MUMmer) on the instance, fetch the data (from S3 or Azure or something else), launch the software (let's say MUMmer). When the software completes, the results are pushed by to cloud storage, and the instance is killed. So in this kind of platform, the installation occurs independently for each job you launch.
I don't know if Galaxy works like that -- that was my question (I am reading the 3 important Galaxy papers today !).
but if you install Ray the tool_dependency file is read and at that time it will be installed and the openmpi dependency.
So the question becomes "When Ray get installed ?" Answers may include:
A) Ray is installed once by a sysadmin.
I think that is the best answer. Replace sysadmin with galaxy-admin. Because Ray will be installed by him. If you have different Ray versions that will be handled by the toolshed to enable reproducibility.
B) Ray is installed automatically by Galaxy when it starts if Ray is not already installed. C) Ray is installed each time someone uses it because an instance is spawned. D) Ray is not installed because it ships with the operating system, just like ls, pwd and so on.
Possible, but you loose reproducibility, but maybe a good starting point to write the wrapper, the installation instruction can be hooked up later.
E) the Galaxy user needs to upload a static Ray executable just like he/she uploads their data.
Cheers, Björn
-Sébastien.
Thanks ! ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
On 06/06/13 02:32 PM, James Taylor wrote:
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page:
http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too...
From what I understand, I need to write 2 XML files in order to allow people to add Ray to their Galaxy instances: * ray.xml * tool_dependencies.xml These two files are self-contained -- that is Galaxy will install everything using tool_dependencies.xml (in my case, Open-MPI and Ray will be installed from source using the GNU compiler). Can I put these files directly in galaxy-dist/tools/sr_assembly/ray/ ? I suppose these 2 XML files will be distributed in galaxy-dist, but like Velvet, Ray won't be enabled by default in tool_conf.xml as assembly is more costly than mapping. Then, I need to add this to galaxy-dist/tool_conf.xml: <section name="NGS: Assembly" id="ngs_assembly"> <label text="Ray" id="ray"/> <tool file="sr_assembly/ray/ray.xml" /> </section> And finally, I just need to restart Galaxy. Is that correct ?
Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
Hi Sébastien, This all looks fine, except for the part about including this in the distribution. As James stated, we are moving to a model where eventually all tools will be sourced from the Tool Shed. That is where you should create your repository. Whether or not to have the repo binaries self-install & config or not will be up to you. Going forward, you probably will want to start up a new thread for new questions and have discussions on the galaxy-dev@bx.psu.edu mailing list. This will give your questions the most exposure to the development community. http://wiki.galaxyproject.org/MailingLists#The_lists Take care, Jen Galaxy team On 6/12/13 8:05 AM, Sébastien Boisvert wrote:
On 06/06/13 02:32 PM, James Taylor wrote:
I want to add Ray (a scalable de novo assembler for genomes and metagenomes) to Galaxy.
And I really want you to do this!
I will also have to write a wrapper for Ray to prepare the command line from the options provided by the Galaxy API.
But where is stored the executable (in my case, where is sdtored Ray) ?
Does Galaxy include the specs to build all the tools available in Galaxy-Central ?
No, we are gradually moving all the tools out of galaxy-central into the Tool Shed. You probably want to look at this page:
http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_too...
From what I understand, I need to write 2 XML files in order to allow people to add Ray to their Galaxy instances:
* ray.xml * tool_dependencies.xml
These two files are self-contained -- that is Galaxy will install everything using tool_dependencies.xml (in my case, Open-MPI and Ray will be installed from source using the GNU compiler).
Can I put these files directly in
galaxy-dist/tools/sr_assembly/ray/
?
I suppose these 2 XML files will be distributed in galaxy-dist, but like Velvet, Ray won't be enabled by default in tool_conf.xml as assembly is more costly than mapping.
Then, I need to add this to galaxy-dist/tool_conf.xml:
<section name="NGS: Assembly" id="ngs_assembly"> <label text="Ray" id="ray"/> <tool file="sr_assembly/ray/ray.xml" /> </section>
And finally, I just need to restart Galaxy.
Is that correct ?
Which describes how you can add a recipe to the toolshed that will install the Ray binaries.
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org
participants (4)
-
Björn Grüning -
James Taylor -
Jennifer Jackson -
Sébastien Boisvert