On Mon, May 13, 2013 at 10:34 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Fri, May 10, 2013 at 8:11 PM, Björn Grüning <bjoern.gruening@pharmazie.uni-freiburg.de> wrote:
Hi Dave,
that was fast.
Thanks a lot! Björn
Thanks Dave,
I see this also should fix handling of JAR files in download_by_url :) https://bitbucket.org/galaxy/galaxy-central/commits/b4733e42a2c98c42401bca5e...
Could I suggest an enhancement to abort on undefined action types, rough patch below (untested)? This will help give a clear error if a new command is used but the Galaxy host is too old to implement it (which is exactly the situation we'd see right now on the main Tool Shed if the new download_file action is used).
Regards,
Peter
Another data point, with my old wget based tool_wrapper.xml file: http://testtoolshed.g2.bx.psu.edu/view/peterjc/effectivet3/b863c4bec840
From the test failure of the previous commit I suspected this, but this revision checks specifically for the 'module' subdirectory and confirms it is not being created:
Effective T3 module folder not found: '/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tmp/tmpAzkY3r/effectiveT3/1.0.1/peterjc/effectivet3/b863c4bec840/module' I infer the 'install script' is failing silently after setting the environment variable but before or when creating the sub-directory $INSTALL_DIR/module, <actions> <!-- Set environment variable so Python script knows where to look --> <action type="set_environment"> <environment_variable name="EFFECTIVET3" action="set_to">$INSTALL_DIR</environment_variable> </action> <!-- Main JAR file --> <action type="shell_command">wget http://effectors.org/download/version/TTSS_GUI-1.0.1.jar</action> <action type="move_file"><source>TTSS_GUI-1.0.1.jar</source><destination>$INSTALL_DIR/</destination></action> <!-- Three model JAR files --> <action type="make_directory">$INSTALL_DIR/module</action> <action type="shell_command">wget http://effectors.org/download/module/TTSS_ANIMAL-1.0.1.jar</action> <action type="move_file"><source>TTSS_ANIMAL-1.0.1.jar</source><destination>$INSTALL_DIR/module/</destination></action> <action type="shell_command">wget http://effectors.org/download/module/TTSS_PLANT-1.0.1.jar</action> <action type="move_file"><source>TTSS_PLANT-1.0.1.jar</source><destination>$INSTALL_DIR/module/</destination></action> <action type="shell_command">wget http://effectors.org/download/module/TTSS_STD-1.0.1.jar</action> <action type="move_file"><source>TTSS_STD-1.0.1.jar</source><destination>$INSTALL_DIR/module/</destination></action> </actions> The first possible cause of failure is the set_environment action, but from the limited information from the failing test that worked. The second possible causes of failure is the wget call, perhaps that command line tool isn't even installed? However, from the limited information reported via the failing test I know the JAR file was downloaded (and moved into place). The third possible failure is the move_file action, and again I know from the limited information reported via the failing test that the TTSS_GUI-1.0.1.jar file was moved into place. My suspicion therefore falls on the make_directory action - my debug output from the failing test indicates the expected directory was not created. However, any exception within fabric_util.install_and_build_package should be caught by install_util.install_and_build_package_via_fabric and recorded in the tool_dependency object. Tracing up the call tree, I found this puzzling bit in the common_install_util.py file: https://bitbucket.org/galaxy/galaxy-central/src/65a81aead95e147f709ee3969d49... def handle_tool_dependencies( app, tool_shed_repository, tool_dependencies_config, tool_dependencies ): ... if tool_dependency.can_install: try: tool_dependency = install_package( app, elem, tool_shed_repository, tool_dependencies=tool_dependencies ) except Exception, e: error_message = "Error installing tool dependency %s version %s: %s" % ( str( package_name ), str( package_version ), str( e ) ) log.debug( error_message ) if tool_dependency: tool_dependency.status = app.model.ToolDependency.installation_status.ERROR tool_dependency.error_message = error_message sa_session.add( tool_dependency ) sa_session.flush() if tool_dependency and tool_dependency.status in [ app.model.ToolDependency.installation_status.INSTALLED, app.model.ToolDependency.installation_status.ERROR ]: installed_tool_dependencies.append( tool_dependency ) It seems even if install_package fails, the tool_dependency is still considered to be installed? If so, why? This would explain the behaviour I am seeing with effectivet3 and other tools on the Tool Shed - despite the tool_dependencies.xml seeming to fail part way though, the tests are run anyway. (I would instead expect the Tool Shed to tell me the install failed, and not bother trying to run the tests, which are probably going to fail anyway). Regards, Peter