June 2013 - galaxy-dev - lists.galaxyproject.org

Messed up a tool installation from a local toolshed
by Joachim Jacob 30 Sep '13

30 Sep '13

Hi all, I have developed a tool in our local Galaxy, and imported it in a local toolshed. I next removed the 'dev' version of the tool in Galaxy, and installed it now via het Tool Shed Manager. I got an error that the dependencies could not be installed. If figured it out that it (needed to add an extra shell_command in the tool_dependencies) (and the mount point on my system needs to have exec permissions of course). So I updated the tool in the toolshed. BUT, then I uninstalled the tool from Galaxy, and reinstalled it from the Tool Shed ... instead of just updating. Anyway, it has happened. Now I have the tool listed in my 'Install Tool Shed Repositories', and when I click on it, it says 'this tool is not installed'. The only action I can apply on this installed tool shed repository is 'Install' (...). But clicking on that 'install' button, leads me to an error page (http error 500). ****************************** URL: http://localhost/admin_toolshed/prepare_for_install?tool_shed_url=http://lo… Module weberror.evalexception.middleware:364 in respond view >> app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request view >> body = method( trans, **kwargs ) Module galaxy.web.framework:221 in decorator view >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:825 in prepare_for_install view >> raw_text = common_util.tool_shed_get( trans.app, tool_shed_url, url ) Module tool_shed.util.common_util:110 in tool_shed_get view >> response = urlopener.open( uri ) Module urllib2:397 in open view >> response = meth(req, response) Module urllib2:510 in http_response view >> 'http', request, response, code, msg, hdrs) Module urllib2:435 in error view >> return self._call_chain(*args) Module urllib2:369 in _call_chain view >> result = func(*args) Module urllib2:518 in http_error_default view >> raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error ************************************ The same happens when I go to the Tool Shed, and try to install the tool from there: Galaxy detects that the tool has been installed before, and points me to that 500 error page. Any help is heartily appreciated, Thanks, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team -- @joachimjacob

4 6

cuffdiff wrapper not synchronized with cuffdiff version
by Karen Chait 26 Sep '13

26 Sep '13

Hello, After the latest update of Galaxy I saw that the cuffdiff wrapper is for version 2.0. I updated the $PATH to cuffdiff v. 2.0.2 but when I run cuffdiff in Galaxy I get the following error: Error running cuffdiff. cuffdiff: /lib64/libz.so.1: no version information available (required by cuffdiff) cuffdiff: unrecognized option '--library-norm-method' cuffdiff v1.3.0 (3022) What else do I need to define inorder to run the latest version of cuffdiff. Thank you, Karen

3 10

Reserved variables in param tags
by Kohler Manuel 12 Sep '13

12 Sep '13

Hi, I have a question regarding the param tag. I would like to pass on the user email to a external python script. I tried to use it like this: <param name="email" type="hidden" value=$__user_email__ /> <param name="experiment" type="select" label="Experiment" help="select Experiment" refresh_on_change="true" dynamic_options="getExperiments(email)"/> This does not work. Ideally I would like to have something like this: dynamic_options="getExperiments($__user_email__)" Has someone done this before? Cheers Manuel -- Manuel Kohler Center for Information Sciences and Databases (C-ISD) Department of Biosystems Science & Engineering (D-BSSE) ETH Zurich, Maulbeerstrasse (1078, 1.02), CH-4058 Basel, +41 61 387 3132

2 1

Galaxy CloudMan release
by Enis Afgan 28 Aug '13

28 Aug '13

*Last night, we released an update to Galaxy CloudMan.* CloudMan offers an easy way to get a personal and completely functional instance of Galaxy in the cloud in just a few minutes, without any manual configuration. *IMPORTANT - please read* Any new cluster will automatically start using this version of CloudMan. Existing clusters will be given an option to do an automatic update once the main interface page is refreshed. Note that this upgrade is a major version upgrade and thus the migration is rather complicated. The migration process has been automated but will take a little while to complete. If you have made customizations to your cluster in terms of adding file systems, upgrading the database, or similar, we do not recommend you perform the upgrade. Note that this upgrade comes with (and requires) a new AMI (ami-118bfc78), which will be automatically used if starting an instance via CloudLaunch <http://usegalaxy.org/cloudlaunch>. *This update brings a large number of updates and new features, the most prominent ones being:* - Unification of *galaxyTools* and *galaxyData* file systems into a single * galaxy* filesystem. This change makes it possible to utilize the Galaxy Tool Shed when installing tools into Galaxy. - Added initial support for Hadoop-type workloads - Added initial support for cluster federation via HTCondor - Added a new file system service for instance's transient storage, allowing it to be used across the cluster over NFS - Added a service for Galaxy Reports webapp - Added optional Loggly (loggly.com) based off-site logging support - Added tags to all resources utilized by CloudMan For more details on the new features, see the the CHANGELOG<https://bitbucket.org/galaxy/cloudman/src/tip/CHANGELOG.md?at=default>and for even more details see, all of 291 commit messages<https://bitbucket.org/galaxy/cloudman/commits/all?search=35baec1%3A8bbae3f> from 7 contributors. Enjoy and please let us know what you think, Enis P.S. We also now have a logo for CloudMan [image: Inline image 2]

3 5

Embedding a help text in a Workflow?
by Peter Cock 22 Aug '13

22 Aug '13

Hello all, I've recently started sharing some sample workflows on the Galaxy Tool Shed, http://toolshed.g2.bx.psu.edu/view/peterjc/rxlr_venn_workflow http://toolshed.g2.bx.psu.edu/view/peterjc/secreted_protein_workflow When writing a Galaxy tool we have a lot of flexibility within the <help> tag to describe the tool, its output, and relevant references using some nice markup language for things like bold fonts and tables. I would very much like something similar for embedding README or help text into a workflow - which again should be shown to the user, both on the 'View workflow action' but also under the "Run workflow" / "Execute" button when actually running a workflow. Am I overlooking something or is the only workflow level description the plain text 'Annotation / Notes' field? Thanks, Peter P.S. I'm not talking about the repository level README text which I can use for a workflow repository - that is only really seen by the Galaxy administrator (not by the users within the Galaxy web-interface) and is not specific to each workflow.

4 8

error checking active jobs : KeyError: 'destination'
by Geert Vandeweyer 19 Aug '13

19 Aug '13

Hi, I'm getting errors in the handler logs after upgrading galaxy stable to today's release. I specified the "dynamic" job runner as default, which returns a job for pbs. These errors appear as soon as the server is up and running. What could be the issue here? galaxy.jobs.runners ERROR 2013-04-02 17:24:23,411 Unhandled exception checking active jobs Traceback (most recent call last): File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 336, in monitor self.check_watched_items() File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 382, in check_watched_items ( failures, statuses ) = self.check_all_jobs() File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 463, in check_all_jobs pbs_server_name = self.__get_pbs_server(pbs_job_state.job_destination.params) File "/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 221, in __get_pbs_server return job_destination_params['destination'].split('@')[-1] KeyError: 'destination' -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726

3 2

Setting torque queues
by Scott Hazelhurst 15 Aug '13

15 Aug '13

[Apologies for my previous email : finger slippage meant I sent before completing the email] Dear dev I would be grateful for some assistance in setting up our galaxy server to submit jobs to our torque server.. We are using release_2013.04.01 I have tried going through the install information but clearly I¹ve misunderstood something, and unfortunately most examples come from earlier versions Here is my job_conf.xml file ‹ this is just a temporary file while I set things up. My intention is to run clustalw on our cluster using the torque queue WitsLong, assuming our torque server is galaxy.bioinf.wits.ac.za Below that is the output I get in the log. From the torque log it complains that there is no default queue specified. Any help would be appreciated. Thanks Scott <?xml version="1.0"?>  <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner"/> <plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner"/> </plugins> <handlers default="handlers"> <handler id="main" tags="handlers,pbs"/> </handlers> <destinations default="WitsLong"> <destination id="local" runner="local"/> <destination id="WitsLong" runner="pbs"> <param id="Resource_List">walltime=72:00:00,nodes=1:ppn=8</param> <param id="-p">128</param> </destination> </destinations> <tools> <tool id="foo" handler="trackster_handler"> <param id="source">trackster</param> </tool> <tool id="clustalw" destination="WitsLong"/> </tools> </job_conf> galaxy.jobs DEBUG 2013-04-30 15:28:13,481 (33) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/33 galaxy.jobs.handler DEBUG 2013-04-30 15:28:13,527 (33) Dispatching to pbs runner 146.141.75.218, 146.141.1.81 - - [30/Apr/2013:15:28:12 +0200] "POST /galaxy/tool_runner/index HTTP/1.1" 200 - "http://galaxy.bioinf.wits.ac.za/galaxy/tool_runner/rerun?id=9752b387803d3e1 e" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:20.0) Gecko/20100101 Firefox/20.0" galaxy.jobs DEBUG 2013-04-30 15:28:14,213 (33) Persisting job destination (destination id: WitsLong) Exception AssertionError: AssertionError('State <sqlalchemy.orm.state.InstanceState object at 0x7f79682e5150> is not present in this identity map',) in <bound method InstanceState._cleanup of <sqlalchemy.orm.state.InstanceState object at 0x7f79682e5150>> ignored galaxy.jobs.handler INFO 2013-04-30 15:28:14,522 (33) Job dispatched galaxy.tools DEBUG 2013-04-30 15:28:15,189 Building dependency shell command for dependency 'clustalw2' galaxy.tools WARNING 2013-04-30 15:28:15,192 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:15,957 (33) submitting file /home/galaxy/galaxy-dist/database/pbs/33.sh galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:15,958 (33) command is: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/000/dataset_2.dat" -o "/home/galaxy/galaxy-dist/database/files/000/dataset_60.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/000/dataset_61.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:15,963 (33) pbs_submit failed (try 1/5), PBS error 15039: Route rejected by all destinations galaxy.jobs DEBUG 2013-04-30 15:28:16,400 (34) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/34 galaxy.jobs.handler DEBUG 2013-04-30 15:28:16,416 (34) Dispatching to pbs runner 146.141.75.218, 146.141.1.81 - - [30/Apr/2013:15:28:13 +0200] "POST /galaxy/tool_runner/index HTTP/1.1" 200 - "http://galaxy.bioinf.wits.ac.za/galaxy/tool_runner/rerun?id=9752b387803d3e1 e" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:20.0) Gecko/20100101 Firefox/20.0" galaxy.jobs DEBUG 2013-04-30 15:28:16,668 (34) Persisting job destination (destination id: WitsLong) galaxy.jobs.handler INFO 2013-04-30 15:28:16,689 (34) Job dispatched galaxy.tools DEBUG 2013-04-30 15:28:16,937 Building dependency shell command for dependency 'clustalw2' galaxy.tools WARNING 2013-04-30 15:28:16,938 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:17,160 (34) submitting file /home/galaxy/galaxy-dist/database/pbs/34.sh galaxy.jobs.runners.pbs DEBUG 2013-04-30 15:28:17,161 (34) command is: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i "/home/galaxy/galaxy-dist/database/files/000/dataset_2.dat" -o "/home/galaxy/galaxy-dist/database/files/000/dataset_62.dat" -s "ALIGNED" -l "/home/galaxy/galaxy-dist/database/files/000/dataset_63.dat" -t "Clustal_run" -d "DNA" -f "CLUSTAL" galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:17,162 (34) pbs_submit failed (try 1/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:17,971 (33) pbs_submit failed (try 2/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:19,166 (34) pbs_submit failed (try 2/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:19,975 (33) pbs_submit failed (try 3/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:21,170 (34) pbs_submit failed (try 3/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:21,978 (33) pbs_submit failed (try 4/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:23,173 (34) pbs_submit failed (try 4/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:23,982 (33) pbs_submit failed (try 5/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs WARNING 2013-04-30 15:28:25,176 (34) pbs_submit failed (try 5/5), PBS error 15039: Route rejected by all destinations galaxy.jobs.runners.pbs ERROR 2013-04-30 15:28:25,984 (33) All attempts to submit job failed galaxy.jobs.runners.pbs ERROR 2013-04-30 15:28:27,179 (34) All attempts to submit job failed <table width="100%" border="0" cellspacing="0" cellpadding="0" style="width:100%;"> <tr> <td align="left" style="text-align:justify;"><font face="arial,sans-serif" size="1" color="#999999"><span style="font-size:11px;">This communication is intended for the addressee only. It is confidential. If you have received this communication in error, please notify us immediately and destroy the original message. You may not copy or disseminate this communication without the permission of the University. Only authorised signatories are competent to enter into agreements on behalf of the University and recipients are thus advised that the content of this message may not be legally binding on the University and may contain the personal views and opinions of the author, which are not necessarily the views and opinions of The University of the Witwatersrand, Johannesburg. All agreements between the University and outsiders are subject to South African Law unless the University agrees in writing to the contrary. </span></font></td> </tr> </table

2 1

Adding GATK tools to Galaxy Cloudman
by Iry Witham 12 Aug '13

12 Aug '13

Hi Team, I am trying to get GATK tools added to my cloud instance and keep getting the following error: [Errno 2] No such file or directory: '/mnt/galaxyData/tmp/tmpszWRLP/GenomeAnalysisTKLite-2.3-9-gdcdccbb/GenomeAnalysisTK.jar'. Then when I try to install it again I get the following: All selected tool dependencies are already installed. I also get the following error: Initializing repository installation failed. Thanks, Iry The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

3 3

Invalid Galaxy URL: None - Installing Tools Shed
by Adam Brenner 07 Aug '13

07 Aug '13

Howdy, I am trying to install a tools shed item, Emboss, but when I do this via the admin interface, I get the following text: Repository installation is not possible due to an invalid Galaxy URL: None. You may need to enable cookies in your browser. I have searched the universe_wsgi.ini file and could not find anything that looks like Galaxy URL. Any ideas on how to set the Galaxy URL? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

4 9

Tool Shed test framework breaking where run_functional_tests.sh works
by Peter Cock 29 Jul '13

29 Jul '13

On Thu, May 9, 2013 at 10:49 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > > Hi Dave, > > As you hoped, these test now shows up on the tool page (as expected > they are test failures - apparently my install script isn't quite right yet): > > ... > > This one is more interesting, and appears to be a problem with the > test framework (both tests pass locally via run_functional_tests.sh): > > http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename/16a1a5ae98e9 > Functional test results for changeset revision 16a1a5ae98e9 of seq_rename > 0 tests passed, 2 tests failed, 0 tests missing test components. > > I'll raise that on a new thread, Here's the output from the (Test) Tool Shed where the tests fail: Tests that failed *Tool id:* seq_rename *Tool version:* seq_rename *Test:*test_tool_000000 ( functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj… ) *Stderr:* *Traceback:* Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 78, in do_it self.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1320, in run_tool self.submit_form( **kwd ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1270, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'new_column' to value '['2']' in form 'tool_form' threw exception: cannot find value/label "2" in list control control: <SelectControl(new_column=[1])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. *Tool id:* seq_rename *Tool version:* seq_rename *Test:*test_tool_000001 ( functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj… ) *Stderr:* *Traceback:* Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 78, in do_it self.run_tool( testdef.tool.id, repeat_name=repeat_name, **page_inputs ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1320, in run_tool self.submit_form( **kwd ) File "/var/opt/buildslaves/buildslave-ec2-1/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1270, in submit_form raise AssertionError( errmsg ) AssertionError: Attempting to set field 'new_column' to value '['c2']' in form 'tool_form' threw exception: cannot find value/label "c2" in list control control: <SelectControl(new_column=[1])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Both tests fail with the following assertion: AssertionError: Attempting to set field 'new_column' to value '['c2']' in form 'tool_form' threw exception: cannot find value/label "c2" in list control control: <SelectControl(new_column=[1])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Both tests pass on my development machine run via: $ ./run_functional_tests.sh -id seq_rename The two tests are defined as follows - identical bar the style used for the column arguments - I was wondering if the Tool Shed testing was stricter about numeric column names, but that does not seem to be relevant: <tests> <test> <param name="input_file" value="four_human_proteins.fasta" ftype="fasta" /> <param name="input_tabular" value="four_human_proteins.rename.tabular" ftype="tabular" /> <param name="old_column" value="1" /> <param name="new_column" value="2" /> <output name="output_file" file="four_human_proteins.rename.fasta" ftype="fasta" /> </test> <test> <param name="input_file" value="four_human_proteins.fasta" ftype="fasta" /> <param name="input_tabular" value="four_human_proteins.rename.tabular" ftype="tabular" /> <param name="old_column" value="c1" /> <param name="new_column" value="c2" /> <output name="output_file" file="four_human_proteins.rename.fasta" ftype="fasta" /> </test> </tests> I can retest with the latest galaxy-dist once this issue is resolved: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-May/014557.html Peter

5 12