Hi All, I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies. The tool I'm considering is xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///) Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine. When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ). Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework. Cheers Ira
Ira, There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini. This issue is documented in https://trello.com/c/eVT3QWJF --Dave B. On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
On Thu, May 9, 2013 at 3:10 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini.
This issue is documented in https://trello.com/c/eVT3QWJF
--Dave B.
Does this issue affect the Tool Shed tests as well, or just when run via run_functional_test.sh? (I'm working though a couple of test failures related to the dependency installation breaking somehow). Thanks, Peter
This only affects tests that are run manually, the automated testing framework correctly resolves tool dependencies. --Dave B. On 5/9/13 10:41:22.000, Peter Cock wrote:
On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
On Thu, May 9, 2013 at 3:10 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini.
This issue is documented in https://trello.com/c/eVT3QWJF
--Dave B.
Does this issue affect the Tool Shed tests as well, or just when run via run_functional_test.sh? (I'm working though a couple of test failures related to the dependency installation breaking somehow).
Thanks,
Peter
Hi Dave, Just following up on this (now quite a bit later sorry). I can't seem to get things to work using your suggested workaround. Using a fresh checkout of galaxy-central 9931:716f29e4c77d I run GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies sh run_functional_tests.sh -installed In my universe_wsgi.ini I have tool_dependency_dir = /home/iracooke/tool_dependencies When running tests I find that the dependencies fail to be resolved (ie ... I get errors like the following). galaxy.jobs.handler INFO 2013-04-28 11:01:17,955 (3) Job dispatched galaxy.tools DEBUG 2013-04-28 11:01:18,594 Building dependency shell command for dependency 'galaxy_protk' galaxy.tools WARNING 2013-04-28 11:01:18,595 Failed to resolve dependency on 'galaxy_protk', ignoring galaxy.tools DEBUG 2013-04-28 11:01:18,596 Building dependency shell command for dependency 'trans_proteomic_pipeline' galaxy.tools WARNING 2013-04-28 11:01:18,596 Failed to resolve dependency on 'trans_proteomic_pipeline', ignoring When I run the tool myself (ie from galaxy ui not from the test script) it resolves the dependencies just fine. Any idea what am I doing wrong? Thanks Ira On 10/05/2013, at 12:10 AM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini.
This issue is documented in https://trello.com/c/eVT3QWJF
--Dave B.
On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Ira, Yes, I should have mentioned that due to the way functional tests are executed, an environment variable needs to be explicitly exported before the functional test framework will recognize it. The syntax is very nearly the same: export GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies ; sh run_functional_tests.sh -installed --Dave B. On 6/3/13 00:25:53.000, Ira Cooke wrote:
Hi Dave,
Just following up on this (now quite a bit later sorry).
I can't seem to get things to work using your suggested workaround.
Using a fresh checkout of galaxy-central 9931:716f29e4c77d
I run
GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies sh run_functional_tests.sh -installed
In my universe_wsgi.ini I have
tool_dependency_dir = /home/iracooke/tool_dependencies
When running tests I find that the dependencies fail to be resolved (ie ... I get errors like the following).
galaxy.jobs.handler INFO 2013-04-28 11:01:17,955 (3) Job dispatched galaxy.tools DEBUG 2013-04-28 11:01:18,594 Building dependency shell command for dependency 'galaxy_protk' galaxy.tools WARNING 2013-04-28 11:01:18,595 Failed to resolve dependency on 'galaxy_protk', ignoring galaxy.tools DEBUG 2013-04-28 11:01:18,596 Building dependency shell command for dependency 'trans_proteomic_pipeline' galaxy.tools WARNING 2013-04-28 11:01:18,596 Failed to resolve dependency on 'trans_proteomic_pipeline', ignoring
When I run the tool myself (ie from galaxy ui not from the test script) it resolves the dependencies just fine.
Any idea what am I doing wrong?
Thanks
Ira
On 10/05/2013, at 12:10 AM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini.
This issue is documented in https://trello.com/c/eVT3QWJF
--Dave B.
On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Oops ... I should have thought of that Thanks .. it works now. Ira On 04/06/2013, at 11:20 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
Yes, I should have mentioned that due to the way functional tests are executed, an environment variable needs to be explicitly exported before the functional test framework will recognize it. The syntax is very nearly the same:
export GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies ; sh run_functional_tests.sh -installed
--Dave B.
On 6/3/13 00:25:53.000, Ira Cooke wrote:
Hi Dave,
Just following up on this (now quite a bit later sorry).
I can't seem to get things to work using your suggested workaround.
Using a fresh checkout of galaxy-central 9931:716f29e4c77d
I run
GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies sh run_functional_tests.sh -installed
In my universe_wsgi.ini I have
tool_dependency_dir = /home/iracooke/tool_dependencies
When running tests I find that the dependencies fail to be resolved (ie ... I get errors like the following).
galaxy.jobs.handler INFO 2013-04-28 11:01:17,955 (3) Job dispatched galaxy.tools DEBUG 2013-04-28 11:01:18,594 Building dependency shell command for dependency 'galaxy_protk' galaxy.tools WARNING 2013-04-28 11:01:18,595 Failed to resolve dependency on 'galaxy_protk', ignoring galaxy.tools DEBUG 2013-04-28 11:01:18,596 Building dependency shell command for dependency 'trans_proteomic_pipeline' galaxy.tools WARNING 2013-04-28 11:01:18,596 Failed to resolve dependency on 'trans_proteomic_pipeline', ignoring
When I run the tool myself (ie from galaxy ui not from the test script) it resolves the dependencies just fine.
Any idea what am I doing wrong?
Thanks
Ira
On 10/05/2013, at 12:10 AM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini.
This issue is documented in https://trello.com/c/eVT3QWJF
--Dave B.
On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Ira, is it still working for you? I installed one repository in a fresh installation and executed: export GALAXY_TOOL_DEPENDENCY_DIR=/home/bag/projects/galaxy/galaxy-central/tool_deps/ ; sh run_functional_tests.sh -installed It failed with: Failed to resolve dependency on 'SCRIPT_PATH', ignoring Any ideas? Bjoern
Oops ... I should have thought of that
Thanks .. it works now.
Ira
On 04/06/2013, at 11:20 PM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
Yes, I should have mentioned that due to the way functional tests are executed, an environment variable needs to be explicitly exported before the functional test framework will recognize it. The syntax is very nearly the same:
export GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies ; sh run_functional_tests.sh -installed
--Dave B.
On 6/3/13 00:25:53.000, Ira Cooke wrote:
Hi Dave,
Just following up on this (now quite a bit later sorry).
I can't seem to get things to work using your suggested workaround.
Using a fresh checkout of galaxy-central 9931:716f29e4c77d
I run
GALAXY_TOOL_DEPENDENCY_DIR=/home/iracooke/tool_dependencies sh run_functional_tests.sh -installed
In my universe_wsgi.ini I have
tool_dependency_dir = /home/iracooke/tool_dependencies
When running tests I find that the dependencies fail to be resolved (ie ... I get errors like the following).
galaxy.jobs.handler INFO 2013-04-28 11:01:17,955 (3) Job dispatched galaxy.tools DEBUG 2013-04-28 11:01:18,594 Building dependency shell command for dependency 'galaxy_protk' galaxy.tools WARNING 2013-04-28 11:01:18,595 Failed to resolve dependency on 'galaxy_protk', ignoring galaxy.tools DEBUG 2013-04-28 11:01:18,596 Building dependency shell command for dependency 'trans_proteomic_pipeline' galaxy.tools WARNING 2013-04-28 11:01:18,596 Failed to resolve dependency on 'trans_proteomic_pipeline', ignoring
When I run the tool myself (ie from galaxy ui not from the test script) it resolves the dependencies just fine.
Any idea what am I doing wrong?
Thanks
Ira
On 10/05/2013, at 12:10 AM, Dave Bouvier <dave@bx.psu.edu> wrote:
Ira,
There is a known issue with resolving tool dependencies when running functional tests. One workaround is to set the GALAXY_TOOL_DEPENDENCY_DIR environment variable to the path configured in your universe_wsgi.ini.
This issue is documented in https://trello.com/c/eVT3QWJF
--Dave B.
On 5/8/13 20:08:08.000, Ira Cooke wrote:
Hi All,
I'm starting to add functional tests to my tools but I've found that I can't get the test framework to work well with tool dependencies.
The tool I'm considering is xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
When I install this tool on my local galaxy instance (on a clean install) the tool runs fine and if I look in the handler logs I see
executing: PACKAGE_BASE=/home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/galaxy_protk/1.2.0/iracooke/xtandem/0940ff5d28ab/env.sh; PACKAGE_BASE=/home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab; export PACKAGE_BASE; . /home/iracooke/tool_dependencies/trans_proteomic_pipeline/4.6.1/iracooke/xtandem/0940ff5d28ab/env.sh; rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /home/iracooke/galaxy-central/database/files/000/dataset_1.dat --var-mods=' 57.021464__at__C ' --fix-mods=' 15.994915__at__M ' /home/iracooke/galaxy-central/database/files/000/dataset_2.dat -o /home/iracooke/galaxy-central/database/files/000/dataset_4.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:53:35,586 (4) Persisting job destination (destination id: local:///)
Note that in this case the tool environments from my tool dependencies are loaded ... and the tool runs fine.
When running functional tests however the tests are executed without first loading the environments from tool dependencies ... and naturally the tool fails. On my local machine I get the following when running the tests (Note that no loading of dependency shel environments occurs). (This seems to be the same issue on the testtoolshed.g2.bx.psu.edu ).
Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,564 (3) executing: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M , ' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files galaxy.jobs DEBUG 2013-04-27 06:55:19,730 (3) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-27 06:55:19,889 execution finished: rvm 1.9.3@protk-1.2.0 do tandem_search.rb -d /tmp/tmp3I8GgN/database/files/000/dataset_1.dat --var-mods=' 15.994915__at__M ,' --fix-mods=' 57.021464__at__C ' /tmp/tmp3I8GgN/database/files/000/dataset_2.dat -o /tmp/tmp3I8GgN/database/files/000/dataset_3.dat -r --enzyme=Trypsin --precursor-ion-tol-units=ppm -v 0 -f 0.65 -p 100.0 --strict-monoisotopic-mass --keep-params-files ... galaxy.jobs DEBUG 2013-04-27 06:55:24,277 setting dataset state to ERROR
Is there something wrong with the way I've defined my tool dependencies for this tool? Or perhaps it's a bug in the testing framework.
Cheers Ira
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (4)
-
Björn Grüning -
Dave Bouvier -
Ira Cooke -
Peter Cock