Creating tool shed repository: installation of dependencies fails.
Hi all, I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails: ************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat ************************* Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error. The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url. ************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action> <action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> ************************* Thanks for helping me further. Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
Joachim, I'm unable to duplicate that error, what revision of Galaxy and the tool shed code are you running? The path mentioned in the error message is a temporary working directory that is created for downloading and processing files needed for a tool dependency. Had the installation succeeded, the required files would have been moved to a path under /mnt/testgalaxydb/database, as expected. --Dave B. On 5/29/13 06:54:39.000, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat
*************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action>
<action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
Hello Joachim, This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind? Greg Von Kuster On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat *************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action> <action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy? Version info of the Tool Shed: ***************************************** [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ ***************************************** Version info of the Galaxy integrating the tool shed repository: ***************************************** parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) ***************************************** Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote:
Hello Joachim,
This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind?
Greg Von Kuster
On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat *************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action> <action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Joachim, I believe the behavior you're experiencing may be due to an issue that was fixed between 9320:47ddf167c9f1 and the current stable branch. I would recommend updating to the most recent stable branch of galaxy-central. --Dave B. On 5/29/13 09:30:37.000, Joachim Jacob | VIB | wrote:
Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy?
Version info of the Tool Shed: ***************************************** [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ *****************************************
Version info of the Galaxy integrating the tool shed repository: ***************************************** parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) *****************************************
Cheers, Joachim
Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
On 05/29/2013 03:08 PM, Greg Von Kuster wrote:
Hello Joachim,
This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind?
Greg Von Kuster
On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat
*************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action>
<action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem. In Trello, this card #584 was posted, mentioning installing compiled binaries is not yet possible, apparently. This is what I experience I believe. (BTW, the search bar at the top of Trello does not work for me - Galaxy board is only pinned to my account, so I cannot apparently search that way) Thanks, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 07:43 PM, Dave Bouvier wrote:
Joachim,
I believe the behavior you're experiencing may be due to an issue that was fixed between 9320:47ddf167c9f1 and the current stable branch. I would recommend updating to the most recent stable branch of galaxy-central.
--Dave B.
On 5/29/13 09:30:37.000, Joachim Jacob | VIB | wrote:
Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy?
Version info of the Tool Shed: ***************************************** [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ *****************************************
Version info of the Galaxy integrating the tool shed repository: ***************************************** parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) *****************************************
Cheers, Joachim
Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
On 05/29/2013 03:08 PM, Greg Von Kuster wrote:
Hello Joachim,
This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind?
Greg Von Kuster
On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat
*************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action>
<action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote:
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
In Trello, this card #584 was posted, mentioning installing compiled binaries is not yet possible, apparently. This is what I experience I believe.
I don't understand why that issue is open, this is possible. https://trello.com/card/-/506338ce32ae458f6d15e4b3/584 e.g. I've successfully tested installing the pre-compiled NCBI BLAST+ binaries as an alternative to local compilation: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus See revision 19:c1a6e5aefee0 where I switched back to compiling: http://testtoolshed.g2.bx.psu.edu/repository/view_changeset?ctx_str=c1a6e5aefee0&id=c1542e8b1988898c
(BTW, the search bar at the top of Trello does not work for me - Galaxy board is only pinned to my account, so I cannot apparently search that way)
Yes - sadly the best we can do as non-members is filter cards (pop out panel on right hand side) but that only looks at the titles. Peter
Hi Peter, Yes, but your NCBI-blast executables were always packed in a tar.gz. The BLAT executable I want to use is just downloadable as such, without extracting needed. That's where it goes wrong. Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On Thu 30 May 2013 11:52:54 AM CEST, Peter Cock wrote:
On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote:
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
In Trello, this card #584 was posted, mentioning installing compiled binaries is not yet possible, apparently. This is what I experience I believe.
I don't understand why that issue is open, this is possible. https://trello.com/card/-/506338ce32ae458f6d15e4b3/584
e.g. I've successfully tested installing the pre-compiled NCBI BLAST+ binaries as an alternative to local compilation: http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus
See revision 19:c1a6e5aefee0 where I switched back to compiling: http://testtoolshed.g2.bx.psu.edu/repository/view_changeset?ctx_str=c1a6e5aefee0&id=c1542e8b1988898c
(BTW, the search bar at the top of Trello does not work for me - Galaxy board is only pinned to my account, so I cannot apparently search that way)
Yes - sadly the best we can do as non-members is filter cards (pop out panel on right hand side) but that only looks at the titles.
Peter
On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote:
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
I may be seeing the same issue on the Test Tool Shed, which suggests a recent regression: This is for the NCBI BLAST+ tools, attempting to compile from source: Installation errors - no functional tests were run for any tools in this changeset revision Type Name Version blast+ package 2.2.26+ Error [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpOASYB_/tmpkEDmTH/ncbi-blast-2.2.26+-src.tar.gz' http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/c1a6e5aefee0 <?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> Downloads and compiles BLAST+ from the NCBI, which assumes you have all the required build dependencies installed. See: http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency> This was based on an older working tool_dependencies.xml file, and other than the comments should match that currently on the main tool shed, where it was working: http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/9dabbfd73c8a <?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> These links provide information for building the NCBI Blast+ package in most environments. System requirements http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency> Peter
Hi Peter, is it also crashing for you locally? I was also able to install the wrapper successfully. I'm just wondering if the testtoolshed is not updated or if I have some nice local modifications left, that are worth a pull request :) Cheers, Bjoern
On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote:
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
I may be seeing the same issue on the Test Tool Shed, which suggests a recent regression:
This is for the NCBI BLAST+ tools, attempting to compile from source:
Installation errors - no functional tests were run for any tools in this changeset revision Type Name Version blast+ package 2.2.26+ Error [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpOASYB_/tmpkEDmTH/ncbi-blast-2.2.26+-src.tar.gz'
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/c1a6e5aefee0
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> Downloads and compiles BLAST+ from the NCBI, which assumes you have all the required build dependencies installed. See: http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency>
This was based on an older working tool_dependencies.xml file, and other than the comments should match that currently on the main tool shed, where it was working: http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/9dabbfd73c8a
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> These links provide information for building the NCBI Blast+ package in most environments.
System requirements http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency>
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
I was also able to install the BLAST+ repository and tool dependency locally, which suggests that the automated testing environment is causing this issue. I'll have a look at that and update you. --Dave B. On 5/30/13 08:45:27.000, Björn Grüning wrote:
Hi Peter,
is it also crashing for you locally? I was also able to install the wrapper successfully. I'm just wondering if the testtoolshed is not updated or if I have some nice local modifications left, that are worth a pull request :)
Cheers, Bjoern
On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote:
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
I may be seeing the same issue on the Test Tool Shed, which suggests a recent regression:
This is for the NCBI BLAST+ tools, attempting to compile from source:
Installation errors - no functional tests were run for any tools in this changeset revision Type Name Version blast+ package 2.2.26+ Error [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpOASYB_/tmpkEDmTH/ncbi-blast-2.2.26+-src.tar.gz'
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/c1a6e5aefee0
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> Downloads and compiles BLAST+ from the NCBI, which assumes you have all the required build dependencies installed. See: http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency>
This was based on an older working tool_dependencies.xml file, and other than the comments should match that currently on the main tool shed, where it was working: http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/9dabbfd73c8a
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> These links provide information for building the NCBI Blast+ package in most environments.
System requirements http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency>
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Peter, It turns out the issue was due to an overzealous firewall rule blocking outgoing FTP connections on the host running the automated tests. I've opened outgoing FTP and started a test for ncbi_blast_plus. --Dave B. On 5/30/13 06:01:53.000, Peter Cock wrote:
On Thu, May 30, 2013 at 10:30 AM, Joachim Jacob | VIB | <joachim.jacob@vib.be> wrote:
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
I may be seeing the same issue on the Test Tool Shed, which suggests a recent regression:
This is for the NCBI BLAST+ tools, attempting to compile from source:
Installation errors - no functional tests were run for any tools in this changeset revision Type Name Version blast+ package 2.2.26+ Error [Errno 2] No such file or directory: '/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpOASYB_/tmpkEDmTH/ncbi-blast-2.2.26+-src.tar.gz'
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/c1a6e5aefee0
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> Downloads and compiles BLAST+ from the NCBI, which assumes you have all the required build dependencies installed. See: http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency>
This was based on an older working tool_dependencies.xml file, and other than the comments should match that currently on the main tool shed, where it was working: http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus/9dabbfd73c8a
<?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.26+"> <install version="1.0"> <actions> <action type="download_by_url">ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.2.26/ncbi-blast-2.2.26+-src.tar.gz</action> <action type="shell_command">cd c++ && ./configure --prefix=$INSTALL_DIR && make && make install</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> These links provide information for building the NCBI Blast+ package in most environments.
System requirements http://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&PAGE_TYPE=BlastDocs&... </readme> </package> </tool_dependency>
Peter
On Thu, May 30, 2013 at 3:46 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Peter,
It turns out the issue was due to an overzealous firewall rule blocking outgoing FTP connections on the host running the automated tests. I've opened outgoing FTP and started a test for ncbi_blast_plus.
--Dave B.
Thanks Dave - the download step has worked this time, but the build failed with a BOOST dependency problem which is probably best as a separate discussion. Regards, Peter
Hi Joachim, your definition is working fine for me (latest galaxy-central 9895:e3455a092882). If have uploaded it to this test repository: http://testtoolshed.g2.bx.psu.edu/view/bgruening/precommit_test ... and was able to install it. blat has no executable flag, so I needed to add chmod +x into the definition. The next Galaxy stable will be out in one week, afaik. At that time it will work :) Cheers, Björn
Unfortunately, updating to the latest code (9320:47ddf167c9f1) does not solve the problem.
In Trello, this card #584 was posted, mentioning installing compiled binaries is not yet possible, apparently. This is what I experience I believe. (BTW, the search bar at the top of Trello does not work for me - Galaxy board is only pinned to my account, so I cannot apparently search that way)
Thanks, Joachim
Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
On 05/29/2013 07:43 PM, Dave Bouvier wrote:
Joachim,
I believe the behavior you're experiencing may be due to an issue that was fixed between 9320:47ddf167c9f1 and the current stable branch. I would recommend updating to the most recent stable branch of galaxy-central.
--Dave B.
On 5/29/13 09:30:37.000, Joachim Jacob | VIB | wrote:
Strange. When I just wget that link in a terminal, do 'chmod +X blat' and ./blat, it just works fine. Could it be that the permissions need to be set by Galaxy?
Version info of the Tool Shed: ***************************************** [galaxy@galaxy galaxy-dist]$ hg summary parent: 9320:47ddf167c9f1 tip Use Galaxy's ErrorMiddleware since Paste's doesn't return start_response. Fixes downloading tarballs from the Tool Shed when use_debug = false. branch: stable commit: 5 modified, 271 unknown update: (current) [galaxy@galaxy galaxy-dist]$ *****************************************
Version info of the Galaxy integrating the tool shed repository: ***************************************** parent: 9292:2cc8d10988e0 security_2013.04.08 Controllers/history: use get_history in switch_to_history branch: stable commit: 4 modified, 22 unknown (new branch head) update: 11 new changesets (update) *****************************************
Cheers, Joachim
Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
On 05/29/2013 03:08 PM, Greg Von Kuster wrote:
Hello Joachim,
This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind?
Greg Von Kuster
On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat
*************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action>
<action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Thu, May 30, 2013 at 11:06 AM, Björn Grüning <bjoern.gruening@pharmazie.uni-freiburg.de> wrote:
Hi Joachim,
your definition is working fine for me (latest galaxy-central 9895:e3455a092882).
If have uploaded it to this test repository: http://testtoolshed.g2.bx.psu.edu/view/bgruening/precommit_test
... and was able to install it. blat has no executable flag, so I needed to add chmod +x into the definition.
The next Galaxy stable will be out in one week, afaik. At that time it will work :)
Cheers, Björn
That makes sense - I've used 'naked' JAR files but they don't seem to need the executable flag set. Peter
The problem is most likely in ~/lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py, line 100. 1. current_dir = os.path.abspath( os.path.join( work_dir, dir ) ) 2. while dir is set a couple of lines before as: dir = workdir (line 78). workdir is the temporary directory. 3. current_dir is then used by common_util.move_file on line 111. This explains the duplication in the path that I observed. It does not yet explain why the temporary directory is created in the wrong location. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 05/29/2013 03:08 PM, Greg Von Kuster wrote:
Hello Joachim,
This URL does not properly download blat. The Firefox browser determines it as a text file, while Safari displays it. This is the cause of the problem. Is there a different URL to download blat that actually downloads it as an archive os some kind?
Greg Von Kuster
On May 29, 2013, at 6:54 AM, Joachim Jacob | VIB | wrote:
Hi all,
I am building a wrapper around BLAT, and have put it locally in a toolshed. However, the automatic installation of dependencies fails:
************************* Tool dependency installation error: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/tmp/tmp2J7K_L/database/tmp/tmp2J7K_L/blat *************************
Note that there is a strange duplication in the path. Furthermore, on that Galaxy instance, the database is on /mnt/testgalaxydb/database instead of path mentioned in the error.
The tool_dependencies.xml matches the wrapper xml. blat is just downloaded from its url.
************************* <?xml version="1.0"?> <tool_dependency> <package name="blat" version="1.0.0"> <install version="1.0"> <actions> <action type="download_by_url">http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/blat/blat</action> <action type="move_file"> <source>blat</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> The BLAT binary for linux x86_64 is retrieved from UCSC. </readme> </package> </tool_dependency> *************************
Thanks for helping me further.
Cheers, Joachim
-- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (5)
-
Björn Grüning -
Dave Bouvier -
Greg Von Kuster -
Joachim Jacob | VIB | -
Peter Cock