- galaxy-dev - lists.galaxyproject.org

Unable to finish job - metadata settings
by Preussner, Jens 25 Jun '14

25 Jun '14

Dear all, I noticed a strange behaviour in our local galaxy installation. First of all, my universe_wsgi.ini contains "retry_metadata_internally = False" and "cleanup_job = always". The tool writes its output simply into the job_working_directory and we move it via && mv static_filename.txt $output in the <command>-tag. This works fine. When restarting the galaxy server and executing the tool after a fresh restart, everything is ok, there are no errors. When executing the same tool a second time, galaxy brings a "tool error" stating that it was unable to finish the job. Nevertheless, the output files are all correct (but marked as red or failed). The error report states: Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 997, in finish if ( not self.external_output_metadata.external_metadata_set_successfully( dataset, self.sa_session ) and self.app.config.retry_metadata_internally ): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py", line 731, in external_metadata_set_successfully rval, rstring = json.load( open( metadata_files.filename_results_code ) ) IOError: [Errno 2] No such file or directory: u'/home/galaxy/galaxy-dist/database/job_working_directory/000/59/metadata_results_HistoryDatasetAssociation_281_oHFjx0' And in the logfile you can find multiple entries like that: galaxy.datatypes.metadata DEBUG 2014-06-25 14:29:35,466 Failed to cleanup external metadata file (filename_results_code) for HistoryDatasetAssociation_281: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/job_working_directory/000/59/metadata_results_HistoryDatasetAssociation_281_oHFjx0' The if-statement in /home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 997 should evaluate to False, since self.app.config.retry_metadata_internally is set to False in the universe_wsgi.ini but it seems it doesn't in this case? Anyone having experienced such a behavior? Any suggestions how to go on and solve the issue? Many thanks! Jens

2 1

Galaxy Tool Wrapper/DTD/Schema Needed
by Jones, Tyler William 24 Jun '14

24 Jun '14

Hi, I am working for USD Bioinformatics and trying to integrate Galaxy tools in an application. I have been looking at this site: https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax? but was wondering if there was anyway to get the actual DTD or Schema for your tool definitions. Is this possible? Thanks, Tyler Jones

2 1

Re: [galaxy-dev] Control on versioning in toolshed tools
by Eric Kuyt 24 Jun '14

24 Jun '14

Thanks! I think http://gregvonkuster.org/galaxy-tool-shed-leveraging-community-contribution… will solve my problem! Because I want to use R which has multiple layers of dependencies rebuilding it locally seemed a bit labor intensive but this capsule thing sounds exactly what I need. I'll take a look into it. Thanks for the fast response. On 24 June 2014 15:00, Greg Von Kuster <greg(a)bx.psu.edu> wrote: > your tool in your local Galaxy environment, using the local Tool Shed as > a source code repository if necessary. > 6. When development is finished, make sure the finished tool is > available in hyour local Tool Shed's repository and execute your local Tool > Shed's Install and Test framework to validate the tool - see the "Using the > Install an > -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

1 0

Control on versioning in toolshed tools
by Eric Kuyt 24 Jun '14

24 Jun '14

Hi All, I am playing around with putting a tool in testtoolshed. Now when changes to dependency versions are detected, the toolshed detects a new version and a dropdown is created. but sometimes I do not want this behavior when the first version was erroneous for example. I tried hg strip on the repository and pushing it back to the testtoolshed but sadly it didn't result in a clean repository but a multi-headered mess. Is there a way to cleanup the remote repository and start over. And what would be a cleaner way to develop tools on a toolshed still making use of repository dependencies? Thanks, Eric

3 2

Difficulties wrapping r script with r_wrapper.sh
by Jeremy Liu 23 Jun '14

23 Jun '14

Hi all, I have been trying to wrap a test r script into my local galaxy install using r_wrapper.sh but have been running with problems. When I run the tool, I get the following error in the side bar. For reference, I've pasted the xml file below. Error: Error in file(file, "rt") : cannot open the connection Calls: read.table -> file Error during wrapup: object 'l' not found Lost warning messages Execution halted The xml file: <tool id="r_script_test" name="Test R Script"> <description>for proof of concept</description> <command interpreter="bash">r_wrapper.sh $script_file</command> <inputs> <param name="in_data" type="data" format="tabular" label="Test source file"/> </inputs> <outputs> <data name="out_data" format="tabular" /> </outputs> <configfiles> <configfile name="script_file"> ## Setup R error handling to go to stderr options(show.error.messages=F, error = function() { cat(geterrmessage(), file=stderr()); q( "no", l, F)}) ## Read in tab file data1 = read.table("{$in_data}", sep="\t") ## Output tab file write.table(data1, file="{$out_data}", quote=FALSE, sep="\t", eol = "\n", row.names=FALSE, col.names=FALSE) ## Close the session dev.off() </configfile> </configfiles> <help> This tools tests a R script to do simple file I/O </help> </tool> I have r_wrapper.sh copied in the same directory as the xml file. Any help would be appreciated! Thanks! Jeremy Liu

2 1

tool_data_table_conf.xml entries for migrated tools
by Lance Parsons 23 Jun '14

23 Jun '14

I've just upgraded a galaxy instance from the Nov 2013 release to the current. I had a few issues, mostly related to tool migration. 1) The entires in the tool_conf.xml file were not deleted as they should have been. This is a pain, but I can fix it (though it will take some time) 2) The repo directories in tool-data/toolshed.g2.bx.psu.edu/repos/devteam/ were never created (e.g. there was no bowtie2 directory there). I did uninstall and reinstall bowtie2 as a test, and the tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2 directory was created. However, it seems that Galaxy now looks for the bowtie2 indicies in tool-data/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/96d2e31a3938/bowtie2… instead of in tool-data/bowtie2_indicies.loc. How should I maintain these indicies going forward? Do I need to manually update the tool_data_table_conf.xml for each migrated tool? Thanks for any help or suggestions. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

1 0

submit() gets blocked when running workflows via API
by Carlos del Ojo 23 Jun '14

23 Jun '14

Hi All, I am using the API (posting to 'http://localhost:8080/api/workflows' ) and running 8 workflows in a row. I use submit function imported from common. First call to submit takes few seconds to return from submit() second takes a few more seconds, but in the fifth submit it gets hanged. However if I start stopping all workflows via the web interface fifth submit() ends goes on with 6th 7th and 8th. I am not sure if there is a limit of workflows running in parallel but if a run them manually through the GUI they work fine (the eight) does anyone know why API calls get blocked? Is it maybe something to do with the Paste HTTP server used by galaxy? Best, C

3 3

deseq_hts problems
by Israel Bravo 22 Jun '14

22 Jun '14

Hi ! I'm trying to execute Galaxy DESeq2 on datasets created in other steps. The script executed is *deseq2-hts.sh* (placed at shed_tools/ toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq…) The first two steps - "Data preparation" and "Read counting" are executed without problems, but the 3-rd step - "Differential testing" reports an error " *Error in file(file, "rt") : cannot open the connectionCalls: read.delim -> read.table -> file"* The error comes from script *difftest_deseq2.R* - it is getting three command arguments (and they are right) and attempts to read files *INFILE_COUNTS.tab* and *INFILE_CONDITIONS.tab* (where INFILE - the second argument.) But the problem that these two files are never created in the database/files/NNN ! Where can be the root of the problem? It is a local installation of Galaxy OS - CentOS 6.5 with all updates. [image: Israel Bravo on about.me] Israel Bravo about.me/bravobih <http://about.me/bravobih>

1 0

interpreter option for version_command tag
by Will Holtz 20 Jun '14

20 Jun '14

The documentation for the version_command tag doesn't mention it can take an interpreter option: https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Cversion_com… I looked at the code in lib/galaxy/tools/__init__.py to see how this was implemented. I was surprised to see that if the interpreter option is passed to version_command, then the interpreter value from the command tag is used in generating the version number response. The version_command interpreter value is never used. For example, I tried the following: <version_command interpreter="foobar">myPythonTool -v</version_command> <command interpreter="python">myPythonTool someCheetahCode</command> And Galaxy successfully got the version number. Is this the expected behavior? -Will

3 2

Announcement: Galaxy Docker builds
by Björn Grüning 20 Jun '14

20 Jun '14

Dear Galaxy Devs and Users, I'm pleased to announce the Galaxy Docker Image project! The Galaxy Docker Image is an easy distributable full-fledged Galaxy installation, that can be used for testing, teaching and presenting new tools and features. One of the main goals is to make the access to entire tool suites as easy as possible. Usually, this includes the setup of a public available webservice that needs to be maintained, or that the Tool-user needs to either setup a Galaxy Server by its own or to have Admin access to a local Galaxy server. With docker, tool developers can create their own Image with all dependencies and the user only needs to run it within docker. Ideally suited to make your reviewers happy :) Currently, I have three test images to play with. The first one, bgruening/galaxy-stable is the main Image that will do all the heavy lifting of setting up PostgreSQL, Apache and Galaxy. bgruening/deepTools and bgruening/chemicaltoolbox are proof of concepts how easy it is to create an own Image build on top of the main Galaxy Image. https://index.docker.io/u/bgruening/galaxy-deeptools/ https://index.docker.io/u/bgruening/galaxy-chemicaltoolbox/ How to use it: docker run -d -p 8080:80 bgruening/galaxy-deeptools docker run -d -p 8080:80 -p 8000:8000 bgruening/galaxy-chemicaltoolbox For a more detailed description please read: https://github.com/bgruening/docker-recipes/tree/master/galaxy I would be very much interested in any kind of feedback. Especially, in two technical questions: I'm not sure if a "trusted build" from the docker index is the best way to go. Trusted Builds in docker are super convenient. It is a post-commit hook from github. Every time something is pushed, the image will be rebuild. The problem is that I can't tag my images with "trusted builds" and unless I have missed something, that forces me to create new repositories every time a new Galaxy release is out. A better way would be, one repository with different tags: galaxy-stable, galaxy-feb-2014 and so on. Every Galaxy release will be just a diff between the old and the new. But this is not possible with "trusted builds". One other trick I used is to remove the mercurial history, that saves 180MB. I'm not sure if this is wise, because it probably kills a few use cases. Currently, the aim of this project is a little bit different from being a developer environment. So I decided to remove it. Please ping me if this was a bad decision :) Thanks for reading! Bjoern

2 2