September 2013 - galaxy-dev - lists.galaxyproject.org

Conditional and validator: refresh_on_change without validation
by Kahlke Tim 14 Sep '13

14 Sep '13

Hi I'm using the <validator> tag for simple sanity checks of a tool input. But I realized that any conditional in the form will automatically call refresh_on_change which also validates all fields and prints errror messages. So questioin: is there a more galaxy-like way of checking input parameters? I am aware of the possibility to do sanity checks through custom code (http://wiki.galaxyproject.org/Admin/Tools/Custom%20Code) but the page states it should only be done if there is no other way. Regards, Tim

2 1

A proposed modules extension for toolshed wrappers
by Guest, Simon 13 Sep '13

13 Sep '13

> > Another vote for the excellent modules system from us at AgResearch > > Galaxy's dependency injection works in a similar way, without requiring > all the external dependencies. We don't need unload since we compose a > unique environment for every tool execution. This is much cleaner that > loading everything into the environment before running Galaxy, we want > tools to be executed with as clean an environment as possible. Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. So far, I have done this in the tool XML file by prefixing the command with a module load command, e.g. <command>. /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence [snipped] </command> The ugly ". /etc/profile" is needed for technical reasons. I don't like this, as it's a bit hacky, but what it achieves is useful and flexible. I envisage an extension to the tool XML syntax which would let me write it like this, say: <requirements><requirement type="module" version="5.0.0">EMBOSS</requirement></requirements> <command>infoseq -sequence [snipped] </command> This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? Here's where I'm coming from. I already have most of the applications I want installed on my system. I am working on updating the RPMs to support side-by-side installs of multiple versions, with version selection by the environment modules facility. This is a small change (and improvement) on what I have today. It's a work in progress, but it's a manageable amount of work. It's looking likely that this multi-version packaging approach may be adopted by an official CentOS Scientific Repo, and this could change the landscape in terms of packaging of scientific applications for major Linux distributions. Who knows, that's all in the future. In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) The big question is, will the Galaxy team accept contributions to Galaxy to support different ways of solving this packaging problem? In particular, allowing for environment modules to be loaded as part of running a tool, and having this functionality in the toolshed? cheers, Simon ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================

3 2

Circumvent "Show all" button
by Gromobir 13 Sep '13

13 Sep '13

Hey guys, Just one short question: Is there any way to circumvent the "Show all" dialog within galaxy, if the data I would like to display is too large? I would like to display some xhtml files, which can really have a lot of lines and I don't want the users of my tool to click on this button every time before accessing the page. Any hints are highly appreciated. Best regards, Gromobir

2 2

toolshed tool versions and tool panel
by Lukasse, Pieter 13 Sep '13

13 Sep '13

Hi Greg, When updating the version number of a tool I noticed that, as expected, both the old version and the new version of the tool become available in the Galaxy environment. What I did not expect is that the new version resulted in a new entry in the menu bar (see screenshot): [cid:image001.png@01CEB07A.3ECB90B0] If I remember correctly, you showed in Oslo that the tool version becomes a choice in the tool itself, something like the screenshot below: [cid:image002.png@01CEB07A.9929B870] Is this a setting somewhere or are we facing a bug? Best regards, Pieter. Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

2 1

Viewing old nightly test results on the (Test) Tool Shed
by Peter Cock 13 Sep '13

13 Sep '13

Hi all, I was wondering if the web interface for the Tool Shed could allow us to look back at old test results? This would be useful in connection with explicit revisions to the tool concerned, but also when something changed which was external to the tool (e.g. a change in Galaxy itself, or the system running the tests, e.g. a library version change). Right now I'm wondering what happened to this tool, which from memory was working, but which failed last night following me updating it yesterday: http://testtoolshed.g2.bx.psu.edu/view/peterjc/clinod Previously this was installing correctly (and I remember debugging the tests themselves due to the number of threads available), but now I seem to have a failed installation: Tests that failed Tool id: clinod Tool version: clinod Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj…) Stderr: Fatal error: Exit code 1 () Error: Unable to access jarfile /ToolDependencies/clinod/1.3/peterjc/clinod/1d09534d6dcd/clinod-1.3.jar Traceback: Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 102, in do_it self.verify_dataset_correctness( outfile, hid=elem_hid, maxseconds=testdef.maxseconds, attributes=attributes, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 782, in verify_dataset_correctness self._assert_dataset_state( elem, 'ok' ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 606, in _assert_dataset_state raise AssertionError( errmsg ) AssertionError: Expecting dataset state 'ok', but state is 'error'. Dataset blurb: error Regards, Peter

1 1

Creating Galaxy Admin / Local Install Mailing List - Updates?
by Adam Brenner 12 Sep '13

12 Sep '13

Howdy, I know this has been discussed before, but I can not find the email threads on this...but what was the status on creating a galaxy admin / local install mailing list? The Devel list is very chatty and while my filters do a good job, I think it will be easier for others to find help on their own if it was on a separate mailing list.... It would be easier to search and look back at previous threads. However, the added benefit of having the devel list is that we get feedback from Galaxy Developers.... As the saying godes: Just food for thought, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

3 2

(Test) Tool Shed rending bug with non-ASCII characters in README.rst
by Peter Cock 12 Sep '13

12 Sep '13

Hello all, Another day, another bug report :( Yesterday I updated this repository, including the README file which now uses reStructuredText markup: http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go/fd0c284a245c After that change, I couldn't view this repository on the Test Tool Shed: >> return render_render_readme(context,readme,pad,parent,row_counter) Module _webapps_tool_shed_repository_common_mako:988 in render_render_readme >> __M_writer(unicode( rst_to_html( readme.text ) )) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1009: ordinal not in range(128) A unicode error. Sigh. As can be seen here, there was a obvious non-ASCII character in the references (Götz) listed in this README.rst file, which renders nicely on BitBucket: https://bitbucket.org/peterjc/galaxy-central/src/7f03d0b7eaa9d002c7389a94a0… In the short term I have replaced "Götz" with "Goetz" - which I was able to do by entering the upload URL by hand, giving: http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go/0f2fc13ce5fc >> return render_render_readme(context,readme,pad,parent,row_counter) Module _webapps_tool_shed_repository_common_mako:988 in render_render_readme >> __M_writer(unicode( rst_to_html( readme.text ) )) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1138: ordinal not in range(128) Progress - the error is now later in the file, which turned out to be a long-dash in a reference, also edited, and now it works: http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go/9b6482bd1222 The commit in my repository for 'fixing' the tool: https://bitbucket.org/peterjc/galaxy-central/commits/0d329a0e28171754c2d17d… My hunch is this is related to this recent commit in Galaxy, https://bitbucket.org/galaxy/galaxy-central/commits/535d8ed52369a8bf02ce5e3… In the medium term I would like to revert back to "Götz", and also replace "Bjoern Gruening" with "Björn Grüning" too (both in my README.rst files and in the RST markup within the tool XML files). Regards, Peter

2 4

Re: [galaxy-dev] HOWTO for making new toolshed package
by Greg Von Kuster 12 Sep '13

12 Sep '13

Hello Pieter, Please send questions like this to the galaxy-dev(a)bx.psu.edu or galaxy-dev(a)lists.bx.psu.edu mailing list rather than individual email addresses to ensure optimal, timely responses. The Tool Shed wiki contains all of the information you need: http://wiki.galaxyproject.org/Tool%20Shed This page provides a nice overview of repository features: http://wiki.galaxyproject.org/ToolShedRepositoryFeatures See additional specific pages below for each item you've listed: On Sep 11, 2013, at 11:52 AM, "Lukasse, Pieter" <pieter.lukasse(a)wur.nl> wrote: > Hi Greg, > > I want to make a toolshed package with the following contents: The above "package" is actually a mercurial repository in the tool shed. > · A number of tools The following pages will provide all of the information you'll need for building Galaxy tools for the Tool Shed. http://wiki.galaxyproject.org/ToolShedToolFeatures > · Some workflows The following pages provide information for enabling workflow sharing via the Tool Shed. http://wiki.galaxyproject.org/ToolShedWorkflowSharing > · Datatypes Datatype features in the Tool Shed are described in the following pages: http://wiki.galaxyproject.org/ToolShedDatatypesFeatures > · Installation configurations for installing external tools like R, Gnuplot and X!Tandem The above information is document in the "ToolShedToolFeatures" section of the wiki, specifically in the following pages: http://wiki.galaxyproject.org/ToolShedToolFeatures#Automatic_third-party_to… > · Dependencies configurations to other packages If by "packages", you are referring to other tool shed repositories, then the documentation describing defining these relationships is located in the following pages: http://wiki.galaxyproject.org/DefiningRepositoryDependencies > > Is there a HOWTO wiki page where I can find exactly which files I should add to the .zip file and how they should be named? Or maybe an example package which I could use? > > Regards, > > Pieter Lukasse > > Wageningen UR, Plant Research International > > Departments of Bioscience and Bioinformatics > > Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, > Wageningen, the Netherlands > > +31-317480891; skype: pieter.lukasse.wur > > http://www.pri.wur.nl > >

3 9

Scaling and hardware requirements
by Nikos Sidiropoulos 12 Sep '13

12 Sep '13

Hi all, I have a couple of questions regarding a server setup dedicated on Galaxy. The idea is to buy a 64 core 256GB RAM server. From my experience I believe that Galaxy will be able to scale up to 64 cpu's but I would like some more feedback on this. Also, is 4GB RAM per CPU core enough for NGS data? (including de-novo assembly) Bests, Nikos

4 7

PATH and local jobs
by Ryan Davis 12 Sep '13

12 Sep '13

Hi, We have a local server running Galaxy and a majority of the tools work correctly (bowtie, BWA, samtools and etc.). I am having issues with some of the FASTX-Toolkit tools. I am starting Galaxy with an init script on Centos 6 and the tools are in my $PATH and work at the command line, just not in Galaxy. The wiki suggest to "set $PATH in the startup file or use." It leaves me hanging and don't know what the other option is. Can someone share with me their workaround? Here is the wiki page: http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies Here is the section I am hoping will help: Local Jobs Changes to $PATH can be persisted by setting them in your shell's startup file(s). This typically means ~/.bash_profile for bash, but please see the <http://www.gnu.org/software/bash/manual/bashref.html#Bash-Startup-Files> bash documentation on startup files or the INVOCATION section of the bash(1) man page to understand the intricacies of how that file is read. Of particular importance, if you are starting Galaxy in a method other than manual invocation from a shell prompt (with sh run.sh) such as with an init script, it is likely that your startup file will not be read. In this instance, you should set $PATH in the startup file or use. Thanks! Ryan

4 3