software installs: PATH vs env.sh
Hello, We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so: <tool id="tophat" name="Tophat for Illumina" version="1.5.0"> <description>Find splice junctions using RNA-seq data</description> <version_command>tophat --version</version_command> <requirements> <requirement type="package">tophat</requirement> <requirement type='package'>bowtie</requirement> <requirement type="package">samtools</requirement> </requirements> Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that. Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh. Thanks, Andrew Warren
All tools need to be in the path of the user running galaxy. Regards, Thon Thon de Boer, Ph.D Bioinformatics Guru T: +1.650.799.6839 | E-mail: thondeboer@me.com http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b "IMPORTANT NOTICE: This email message is legally privileged, confidential and is for the use of the individual or entity to whom it is addressed. If you have received this email message by error, please immediately notify us by email and delete the message. Thank you." On Jan 21, 2012, at 3:50 PM, Andrew Warren <anwarren@vbi.vt.edu> wrote:
Hello,
We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so:
<tool id="tophat" name="Tophat for Illumina" version="1.5.0"> <description>Find splice junctions using RNA-seq data</description> <version_command>tophat --version</version_command> <requirements> <requirement type="package">tophat</requirement> <requirement type='package'>bowtie</requirement> <requirement type="package">samtools</requirement> </requirements>
Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that.
Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh.
Thanks, Andrew Warren
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On 01/22/2012 01:41 AM, Anthonius deBoer wrote:
All tools need to be in the path of the user running galaxy.
but can be simply added to the path by adding them to the 'run.sh' script Regards, Hans
Regards,
Thon
Thon de Boer, Ph.D Bioinformatics Guru
T: +1.650.799.6839 | E-mail: thondeboer@me.com
http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b
"IMPORTANT NOTICE: This email message is legally privileged, confidential and is for the use of the individual or entity to whom it is addressed. If you have received this email message by error, please immediately notify us by email and delete the message. Thank you."
On Jan 21, 2012, at 3:50 PM, Andrew Warren<anwarren@vbi.vt.edu> wrote:
Hello,
We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so:
<tool id="tophat" name="Tophat for Illumina" version="1.5.0"> <description>Find splice junctions using RNA-seq data</description> <version_command>tophat --version</version_command> <requirements> <requirement type="package">tophat</requirement> <requirement type='package'>bowtie</requirement> <requirement type="package">samtools</requirement> </requirements>
Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that.
Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh.
Thanks, Andrew Warren
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Thanks for the replies. One extra element I failed to properly describe is that we are running on a cluster with a similar setup to CloudMan. This means we have a galaxyTools/tools directory with folders for each tool, and subfolders for each version with a "default" symlink to the version currently in use. Each subfolder has a "env.sh" script which is added by the dependency manager to the qsub script to be processed at runtime . I was thinking that one of the benefits of using the "requirements" tag is that it would/could allow for tool and version specific dependency chains. For instance if a version of tophat only runs with a certain version of bowtie but you want to make the newest beta version of bowtie also available to run independently. Right now the (sort of problem) I am having is that in order for the necessary PATH information to be transmitted to the compute nodes I have to have bowtie and samtools listed as requirements in the tool wrapper. The PATH variable of the galaxy user doesn't transmit to the compute nodes (despite having it set before launching the main instance of galaxy). With our current setup I have seen this with several different tools that call other tools and so I am wondering if my cluster/cloud setup has gone wrong somewhere. Is there a "right" way or place to specify the runtime PATH so that it will be transmitted to the compute nodes? Just trying to figure this out in terms of "best practices" since my current setup seems to require modification of the default wrappers to transmit PATH information to the compute nodes. Thanks again, Andrew On Sun, Jan 22, 2012 at 8:00 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
On 01/22/2012 01:41 AM, Anthonius deBoer wrote:
All tools need to be in the path of the user running galaxy.
but can be simply added to the path by adding them to the 'run.sh' script
Regards, Hans
Regards,
Thon
Thon de Boer, Ph.D Bioinformatics Guru
T: +1.650.799.6839 | E-mail: thondeboer@me.com
"IMPORTANT NOTICE: This email message is legally privileged, confidential and is for the use of the individual or entity to whom it is addressed. If you have received this email message by error, please immediately notify us by email and delete the message. Thank you."
On Jan 21, 2012, at 3:50 PM, Andrew Warren<anwarren@vbi.vt.edu> wrote:
Hello,
We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so:
<tool id="tophat" name="Tophat for Illumina" version="1.5.0"> <description>Find splice junctions using RNA-seq data</description> <version_command>tophat --version</version_command> <requirements> <requirement type="package">tophat</**requirement> <requirement type='package'>bowtie</**requirement> <requirement type="package">samtools</**requirement> </requirements>
Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that.
Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh.
Thanks, Andrew Warren
______________________________**_____________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
______________________________**_____________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Without knowing exactly what 'a cluster with a similar setup to CloudMan' really means, if you're using SGE, passing the galaxy process's path to the job is usually accomplished by setting the -V flag in sge_request eg on an ubuntu 11.10 machine with 'default' as the queue (vgalaxy)galaxy@iaas1:$ cat /var/lib/gridengine/default/common/sge_request # system wide defaults -V -cwd works for me. On Wed, Jan 25, 2012 at 12:12 PM, Andrew Warren <anwarren@vbi.vt.edu> wrote:
Thanks for the replies. One extra element I failed to properly describe is that we are running on a cluster with a similar setup to CloudMan. This means we have a galaxyTools/tools directory with folders for each tool, and subfolders for each version with a "default" symlink to the version currently in use. Each subfolder has a "env.sh" script which is added by the dependency manager to the qsub script to be processed at runtime . I was thinking that one of the benefits of using the "requirements" tag is that it would/could allow for tool and version specific dependency chains. For instance if a version of tophat only runs with a certain version of bowtie but you want to make the newest beta version of bowtie also available to run independently.
Right now the (sort of problem) I am having is that in order for the necessary PATH information to be transmitted to the compute nodes I have to have bowtie and samtools listed as requirements in the tool wrapper. The PATH variable of the galaxy user doesn't transmit to the compute nodes (despite having it set before launching the main instance of galaxy). With our current setup I have seen this with several different tools that call other tools and so I am wondering if my cluster/cloud setup has gone wrong somewhere. Is there a "right" way or place to specify the runtime PATH so that it will be transmitted to the compute nodes? Just trying to figure this out in terms of "best practices" since my current setup seems to require modification of the default wrappers to transmit PATH information to the compute nodes.
Thanks again, Andrew
On Sun, Jan 22, 2012 at 8:00 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
On 01/22/2012 01:41 AM, Anthonius deBoer wrote:
All tools need to be in the path of the user running galaxy.
but can be simply added to the path by adding them to the 'run.sh' script
Regards, Hans
Regards,
Thon
Thon de Boer, Ph.D Bioinformatics Guru
T: +1.650.799.6839 | E-mail: thondeboer@me.com
http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b
"IMPORTANT NOTICE: This email message is legally privileged, confidential and is for the use of the individual or entity to whom it is addressed. If you have received this email message by error, please immediately notify us by email and delete the message. Thank you."
On Jan 21, 2012, at 3:50 PM, Andrew Warren<anwarren@vbi.vt.edu> wrote:
Hello,
We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so:
<tool id="tophat" name="Tophat for Illumina" version="1.5.0"> <description>Find splice junctions using RNA-seq data</description> <version_command>tophat --version</version_command> <requirements> <requirement type="package">tophat</requirement> <requirement type='package'>bowtie</requirement> <requirement type="package">samtools</requirement> </requirements>
Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that.
Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh.
Thanks, Andrew Warren
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;
Thanks Ross, the -v option is definitely a good way to go that I had not considered. For anyone else following this thread, the tool dependency environment system was just commented on and documented by Nate. See the following links: http://galaxy-development-list-archive.2308389.n4.nabble.com/Documentation-f... http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies Thanks! -Andrew On Tue, Jan 24, 2012 at 8:27 PM, Ross <ross.lazarus@gmail.com> wrote:
Without knowing exactly what 'a cluster with a similar setup to CloudMan' really means, if you're using SGE, passing the galaxy process's path to the job is usually accomplished by setting the -V flag in sge_request eg on an ubuntu 11.10 machine with 'default' as the queue (vgalaxy)galaxy@iaas1:$ cat /var/lib/gridengine/default/common/sge_request # system wide defaults -V -cwd
works for me.
On Wed, Jan 25, 2012 at 12:12 PM, Andrew Warren <anwarren@vbi.vt.edu> wrote:
Thanks for the replies. One extra element I failed to properly describe is that we are running on a cluster with a similar setup to CloudMan. This means we have a galaxyTools/tools directory with folders for each tool, and subfolders for each version with a "default" symlink to the version currently in use. Each subfolder has a "env.sh" script which is added by the dependency manager to the qsub script to be processed at runtime . I was thinking that one of the benefits of using the "requirements" tag is that it would/could allow for tool and version specific dependency chains. For instance if a version of tophat only runs with a certain version of bowtie but you want to make the newest beta version of bowtie also available to run independently.
Right now the (sort of problem) I am having is that in order for the necessary PATH information to be transmitted to the compute nodes I have to have bowtie and samtools listed as requirements in the tool wrapper. The PATH variable of the galaxy user doesn't transmit to the compute nodes (despite having it set before launching the main instance of galaxy). With our current setup I have seen this with several different tools that call other tools and so I am wondering if my cluster/cloud setup has gone wrong somewhere. Is there a "right" way or place to specify the runtime PATH so that it will be transmitted to the compute nodes? Just trying to figure this out in terms of "best practices" since my current setup seems to require modification of the default wrappers to transmit PATH information to the compute nodes.
Thanks again, Andrew
On Sun, Jan 22, 2012 at 8:00 AM, Hans-Rudolf Hotz <hrh@fmi.ch> wrote:
On 01/22/2012 01:41 AM, Anthonius deBoer wrote:
All tools need to be in the path of the user running galaxy.
but can be simply added to the path by adding them to the 'run.sh' script
Regards, Hans
Regards,
Thon
Thon de Boer, Ph.D Bioinformatics Guru
T: +1.650.799.6839 | E-mail: thondeboer@me.com
http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b
"IMPORTANT NOTICE: This email message is legally privileged, confidential and is for the use of the individual or entity to whom it is addressed. If you have received this email message by error, please immediately notify us by email and delete the message. Thank you."
On Jan 21, 2012, at 3:50 PM, Andrew Warren<anwarren@vbi.vt.edu> wrote:
Hello,
We recently transitioned from a CloudMan instance of galaxy to our own cluster and started having problems with calls to tools from within other tools. For example when Tophat calls bowtie-inspect its not finding the executable. To fix this I listed bowtie in the requirements section of the tophat wrapper like so:
<tool id="tophat" name="Tophat for Illumina" version="1.5.0"> <description>Find splice junctions using RNA-seq data</description> <version_command>tophat --version</version_command> <requirements> <requirement type="package">tophat</requirement> <requirement type='package'>bowtie</requirement> <requirement type="package">samtools</requirement> </requirements>
Now I am wondering, is it generally expected that all tools used by galaxy will have their executables on the user galaxy's PATH? Is the above a good solution? Or is there something else likely amiss with our galaxy setup? I think we recently pulled updates for some major tool_shed release but I haven't been able to determine if any of the tools listed above were affected by that.
Wish I were in Český Krumlov asking this question. Missed the registration deadline...doh.
Thanks, Andrew Warren
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;
participants (4)
-
Andrew Warren -
Anthonius deBoer -
Hans-Rudolf Hotz -
Ross