- galaxy-dev - lists.galaxyproject.org

value_from_basic - KeyError: 'base_name', was: Tests not being run on toolsheds?
by Peter Cock 02 Apr '15

02 Apr '15

Hi Dave, The following looks like a regression on the Test Tool Shed, the tests pass locally (using a recent revision), and on TravisCI using the current galaxy dev branch on GitHub: https://travis-ci.org/peterjc/pico_galaxy/builds/54870500 Problem tool: https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs This may be unrelated, but by chance this is the first time I have uploaded a tool to the Tool Shed which uses the new functionality to test the stdout/stderr strings, and more importantly it includes a test expected to fail via <test expect_failure="true" expect_exit_code="1"> Revision on my development repository: https://github.com/peterjc/pico_galaxy/commit/55ebb308b911b4acef912cc3b03f4… Test Tool Shed output from last night: Automated test environment Time tested: 2015-03-18 02:46:55 System: Linux 3.13.0-36-generic Architecture: x86_64 Python version: 2.7.6 Galaxy revision: 17050:6395e7035143 Galaxy database version: 128 Tool shed revision: 16867:0468d285f89c Tool shed database version: 25 Tool shed mercurial version: 3.2.4 Tests that failed Tool id: sample_seqs Tool version: sample_seqs Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 268, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 37, in do_it stage_data_in_history( galaxy_interactor, testdef.test_data(), test_history, shed_tool_id ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 38, in stage_data_in_history upload_wait() File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 279, in wait while not self.__history_ready( history_id ): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 297, in __history_ready return self._state_ready( state, error_msg="History in error state." ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 356, in _state_ready raise Exception( error_msg ) Exception: History in error state. Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/runners/__init__.py", line 158, in prepare_job job_wrapper.prepare() File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/jobs/__init__.py", line 828, in prepare tool_evaluator.set_compute_environment( compute_environment, get_special=get_special ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/evaluation.py", line 53, in set_compute_environment incoming = self.tool.params_from_strings( incoming, self.app ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/__init__.py", line 1873, in params_from_strings return params_from_strings( self.inputs, params, app, ignore_errors ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/parameters/__init__.py", line 103, in params_from_strings value = params[key].value_from_basic( value, app, ignore_errors ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/lib/galaxy/tools/parameters/grouping.py", line 512, in value_from_basic raise e KeyError: 'base_name' requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 galaxy.datatypes.metadata: DEBUG: Cleaning up external metadata files requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c/contents?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c/contents/d80a923b055ee8e9?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1 requests.packages.urllib3.connectionpool: DEBUG: "GET /api/histories/c33ffb67c9176a6c/contents/d80a923b055ee8e9/provenance?key=ae52a59b996b793c4c06a11fc3c33481 HTTP/1.1" 200 None (All the tests for this tool seem to have failed the same way) Regards, Peter On Wed, Mar 18, 2015 at 1:46 PM, Dave Bouvier <dave(a)bx.psu.edu> wrote: > Gentlemen, > > The issue with the nightly testing was due to a stalled test run blocking > subsequent tests. I've cleared out that blockage and a manual test run > appears to have completed successfully, as should future automated test > runs. As always, feel free to let us know if you encounter any additional > inexplicable behavior. > > --Dave B. > > On 03/18/2015 07:00 AM, Peter Cock wrote: >> >> ... I can also confirm that the Test Tool Shed example I >> gave was tested overnight, although it looks like a novel failure: >> https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs >>

2 4

wrapper accepts fastqcssanger when it shouldn't
by Nikhil Joshi 02 Apr '15

02 Apr '15

Hi all, So in my wrapper for my trimming tool, sickle, I only allow fastq, fastqsanger, fastqillumina, or fastqsolexa input data. However, if a fastqsanger file's datatype has been erroneously changed to fastqcssanger, my tool will still show that file in the dropdown for the possible inputs. Why does this happen? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 1

dat quota
by Williams, Charles Houston 01 Apr '15

01 Apr '15

After reading threads. I have deleted and purged all but 33gb of my datasets and a single history. However, my quota is still 99%. What can I do to fix this issue. My user address is Charles.h.williams@vanderbilt. Edu Thanks, Chuck Charles H Williams III Graduate Student Hong Lab Department of Cell and Developmental Biology Vanderbilt University Light Hall 1155a

2 1

Trouble maintaining Galaxy config on cloud instance
by David Kovalic 01 Apr '15

01 Apr '15

Hi, I am trying to change the ftp dir (and some other modifications) vs. the standard Galaxy Cloudman config. When I modify the appropriate line in /mnt/galaxy/galaxy-app/universe_wsgi.ini on my running cloud instance: my changes disappear when I terminate/restart the cluster. When I modify universe_wsgi.ini.cloud and upload to the S3 bucket the file gets overwritten by the default when I restart the cluster... I can't find anywhere on the net which file I need to modify in order to have config changes persist. Any help would be greatly appreciated. Thanks, David

2 2

storing "set max value" in trackster for wig file display
by Hans-Rudolf Hotz 01 Apr '15

01 Apr '15

Hi We have been using trackster much more in recent times - It really has improved a lot (especially its stability) over the last few years - thank you very much. We are running into a little annoying issue: when we display wiggle files, bedgraph files, bam files in coverage mode, etc, the max value gets somehow calculated. Very often, we want to change that, ie in order to see the full heights of the peaks instead of the capped peaks. This can be easily done by changing the "set max value" at the right hand side of the track. Unfortunately, the new value does not get stored. No matter, whether we click on safe or not. So next time we wanna a look at the same visualization again, we have to set the 'max value' again. Is this a known missing feature (bug) ? I just wanna check before I create a Trello card - so far, I couldn't find in in Trello. Thanks for your help Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland

2 1

strange issue on TEST toolshed
by Lukasse, Pieter 31 Mar '15

31 Mar '15

Hi, We have a strange issue on the test tool shed. Some repositories do not install anymore. For example: https://kuyt002@testtoolshed.g2.bx.psu.edu/repos/kuyt002/vennerable_r Also when just trying to clone this repository we get the error: [cid:image001.png@01D06B9A.58228FA0] Is this a known issue? What is going on with tool shed? I also reported some problems in my previous emails for which I got no response. Should I be reporting this somewhere else? Please let me know! Thanks, Pieter Pieter Lukasse Wageningen UR, Plant Research International Department of Bioinformatics (Bioscience) Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands T: +31-317481122; M: +31-628189540; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

2 1

problem with uploading SRA file to our custom galaxy
by Nikhil Joshi 31 Mar '15

31 Mar '15

Hi all, So it is time again for our next bioinformatics course using galaxy and I am trying to add some new tools to our custom galaxy running in the Amazon Cloud. I've installed the ncbi sra toolkit interface from the toolshed and the sra toolkit is installed on our AMI. However, when I upload an SRA file into our galaxy, the file is empty and I get this error: The uploaded binary file contains inappropriate content I am able to extract the fastq from the archive just fine on the command line, so I know it's not a problem with the file itself. Is there a solution to this? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 4

GalaxyAdmins Online Meetup on 19 February
by Dave Clements 31 Mar '15

31 Mar '15

Hello all, The first GalaxyAdmins <https://wiki.galaxyproject.org/Community/GalaxyAdmins> meetup of 2015 <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_02_19> will happen online on Thursday, 19 February <http://bit.ly/1yZJSkN>. GalaxyAdmins <https://wiki.galaxyproject.org/Community/GalaxyAdmins> is a special interest group for Galaxy community members who are responsible for Galaxy installations. A summary of both the user and admin/developer 2014 Galaxy Community Questionnaires <https://wiki.galaxyproject.org/News/2014Questionnaire> will be presented by Dave Clements <https://wiki.galaxyproject.org/DaveClements>, a *Galaxy Project Update* will be offered by a Galaxy Team member <https://wiki.galaxyproject.org/GalaxyTeam>, and Hans-Rudolf Hotz <https://wiki.galaxyproject.org/HansrudolfHotz> will lead a short discussion on *GalaxyAdmins direction*. Thanks to everyone for letting us know what dates and times worked best for you. This time and day of the week worked for a remarkable 91% of respondents. We are looking forward to starting this group up again, Hans-Rudolf Hotz and Dave Clements -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

3 4

Trouble maintaining Galaxy config on cloud
by David Kovalic 30 Mar '15

30 Mar '15

Hi, I am trying to change the ftp dir (and some other modifications) to the standard Galaxy Cloudman config. When I modify the appropriate line in /mnt/galaxy/galaxy-app/universe_wsgi.ini on my ru8nning cloud instance: my changes disappear when I terminate/restart the cluster. When I modify universe_wsgi.ini.cloud and upload to the S3 bucket the file gets overwritten by the default when I restart the cluster... I cant find anywhere on the net which file I need to modify in order to have config changes persist. Any help would be greatly appreciated. Thanks, David

1 0

Re: [galaxy-dev] data collections - workflow - bug?
by John Chilton 30 Mar '15

30 Mar '15

Sorry again about this mixup (bringing conversation back on channel). Does this happen everytime you run a workflow with collections? Do you know how many job handlers Galaxy is configured with? -John On Mon, Jan 26, 2015 at 11:24 AM, John Chilton <jmchilton(a)gmail.com> wrote: > Ugh.... misread sqlalchemy as sqlite. I will work on this. > > -John > > On Mon, Jan 26, 2015 at 11:16 AM, Torsten Houwaart > <houwaart(a)informatik.uni-freiburg.de> wrote: >> Hi John, >> >> i ran this on the Galaxy server in Freiburg: galaxy.uni-freiburg.de >> Björn (who sits immediately opposite of my desk :) ) told me it's a Postres >> database and that shouldn't be the problem. >> >> Best, >> Torsten >> >> >> On 26.01.2015 16:48, John Chilton wrote: >>> >>> Hey, >>> >>> Really intensive database operations including datasest collections >>> but other things too like multi-running tools or workflows over many >>> individual datasets for instance - can very easily overwhelm the >>> default sqlite database. This is frustrating and shouldn't happen - >>> but it does unfortunately. I would recommend using a postgres database >>> when testing out dataset collections. The good news is that it is >>> easier than ever to get a fully fledged production-quality server >>> thanks to Bjoern's docker server >>> (https://github.com/bgruening/docker-galaxy-stable) - it comes bundled >>> with Postgres and Slurm so it should be able to handle the collection >>> operations. If you need to run Galaxy on a non-containerized server >>> (for instance because that is where the software is) more information >>> on setting up Galaxy can be found here >>> https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer. >>> >>> Here is a Trello card to track progress on the database optimization >>> efforts if you are interested https://trello.com/c/UPLsMKQI. >>> >>> Very sorry. >>> >>> -John >>> >>> >>> >>> -John >>> >>> On Mon, Jan 26, 2015 at 9:35 AM, Torsten Houwaart >>> <houwaart(a)informatik.uni-freiburg.de> wrote: >>>> >>>> Hello Galaxy Devs, >>>> >>>> I was using data collections (for the first time) for a new workflow of >>>> ours >>>> and I ran into this problem. There was no complaint by the >>>> workflow-editor >>>> and I could start the workflow but then <see below> happened. >>>> If you need more information about the workflow or otherwise let me know. >>>> >>>> Best, >>>> Torsten H. >>>> >>>> >>>> job traceback: >>>> Traceback (most recent call last): >>>> File >>>> "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", >>>> line 565, in finish_job >>>> job_state.job_wrapper.finish( stdout, stderr, exit_code ) >>>> File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line >>>> 1250, in finish >>>> self.sa_session.flush() >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/scoping.py", >>>> line 114, in do >>>> return getattr(self.registry(), name)(*args, **kwargs) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py", >>>> line 1718, in flush >>>> self._flush(objects) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py", >>>> line 1789, in _flush >>>> flush_context.execute() >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/unitofwork.py", >>>> line 331, in execute >>>> rec.execute(self) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/unitofwork.py", >>>> line 475, in execute >>>> uow >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/persistence.py", >>>> line 59, in save_obj >>>> mapper, table, update) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/persistence.py", >>>> line 485, in _emit_update_statements >>>> execute(statement, params) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", >>>> line 1449, in execute >>>> params) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", >>>> line 1584, in _execute_clauseelement >>>> compiled_sql, distilled_params >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", >>>> line 1698, in _execute_context >>>> context) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", >>>> line 1691, in _execute_context >>>> context) >>>> File >>>> >>>> "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/default.py", >>>> line 331, in do_execute >>>> cursor.execute(statement, parameters) >>>> DBAPIError: (TransactionRollbackError) deadlock detected >>>> DETAIL: Process 3144 waits for ShareLock on transaction 2517124; blocked >>>> by >>>> process 3143. >>>> Process 3143 waits for ShareLock on transaction 2517123; blocked by >>>> process >>>> 3144. >>>> HINT: See server log for query details. >>>> 'UPDATE workflow_invocation SET update_time=%(update_time)s WHERE >>>> workflow_invocation.id = %(workflow_invocation_id)s' {'update_time': >>>> datetime.datetime(2015, 1, 26, 14, 20, 4, 155440), >>>> 'workflow_invocation_id': >>>> 5454} >>>> >>>> >>>> ___________________________________________________________ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> https://lists.galaxyproject.org/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>> http://galaxyproject.org/search/mailinglists/ >> >>

2 2