April 2015 - galaxy-dev - lists.galaxyproject.org

anybody seen this class of error before?
by Langhorst, Brad 20 Apr '15

20 Apr '15

InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Original exception was: Class 'kombu.transport.sqlalchemy.Message' is not mapped galaxy.jobs.handler ERROR 2015-04-19 14:46:00,794 Exception in monitor_step Traceback (most recent call last): File "/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/jobs/handler.py", line 177, in __monitor self.__monitor_step() File "/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/jobs/handler.py", line 229, in __monitor_step jobs_to_check = self.sa_session.query(model.Job).enable_eagerloads(False) \ File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/scoping.py", line 150, in do return getattr(self.registry(), name)(*args, **kwargs) File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py", line 1165, in query return self._query_cls(entities, self, **kwargs) File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 108, in __init__ self._set_entities(entities) File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 118, in _set_entities self._set_entity_selectables(self._entities) File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 151, in _set_entity_selectables ent.setup_entity(*d[entity]) File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 3036, in setup_entity self._with_polymorphic = ext_info.with_polymorphic_mappers File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/util/langhelpers.py", line 725, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/mapper.py", line 1877, in _with_polymorphic_mappers configure_mappers() File "/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/mapper.py", line 2586, in configure_mappers raise e Once this error occurs, no more jobs execute on the cluster. Restarting the handler seems to get things moving again, but I’d like to find the source of the failure. Any ideas? Brad -- Brad Langhorst, Ph.D. Applications and Product Development Scientist

3 2

Error generating Picard indexes when using data_manager_gatk_picard_index_builder from the main tool shed
by Paulo Silva 17 Apr '15

17 Apr '15

Hello all Galaxy devs, I’ve come across (what seems to be) a Galaxy bug (or rather, tool-shed bug) when I run the data manager data_manager_gatk_picard_index_builder (rev. a15709ad26ea) from the Galaxy main tool shed repo on a clean Galaxy install. Installation goes fine - when I install this data manager, all dependencies are correctly installed: package_picard_1_56_0 (rev. 99a28567c3a3) and package_samtools_0_1_18 (rev. 171cd8bc208d). After installation, I go to Admin -> Manage local data (beta) -> Run Data Manager Tools, click “Generate GATK-sorted Picard indexes” and the data manager form is displayed. I select the reference genome (either hg19 or hg38 - installed with data_manager_fetch_genome_dbkeys_all_fasta from the main tool shed, rev. bca4c608408c - working fine with other tools including bowtie2 and trackster), enter “hg19” both “Name of sequence” and “ID for sequence” fields, and click “Execute”. The indexing job is added to the queue, but when I go to view data manager jobs for this data manager, I see it stopped with the following error: Traceback (most recent call last): File "/home/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea/data_manager_gatk_picard_index_builder/data_manager/data_manager_gatk_picard_index_builder.py", line 179, in <module> if __name__ == "__main__": main() File "/home/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea/data_manager_gatk_picard_index_builder/data_manager/data_manager_gatk_picard_index_builder.py", line 162, in main raise Exception( '"%s" is not a valid dbkey. You must specify a valid dbkey.' % ( options.fasta_dbkey ) ) Exception: "None" is not a valid dbkey. You must specify a valid dbkey. Inspecting galaxy.log doesn’t reveal anything relevant (at least for me - which is why I need your help!): galaxy.jobs DEBUG 2015-04-17 11:13:23,031 (63) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/63 galaxy.jobs.handler DEBUG 2015-04-17 11:13:23,044 (63) Dispatching to local runner 192.168.2.74 - - [17/Apr/2015:11:13:22 +0100] "POST /api/tools HTTP/1.1" 200 - "http://192.168.2.221:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu%2Frepo…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/600.4.10 (KHTML, like Gecko) Version/8.0.4 Safari/600.4.10" galaxy.jobs DEBUG 2015-04-17 11:13:23,168 (63) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2015-04-17 11:13:23,212 (63) Job dispatched galaxy.tools.deps DEBUG 2015-04-17 11:13:23,393 Building dependency shell command for dependency 'samtools' galaxy.tools.deps DEBUG 2015-04-17 11:13:23,395 Building dependency shell command for dependency 'picard' galaxy.jobs.runners DEBUG 2015-04-17 11:13:23,475 (63) command is: PACKAGE_BASE=/home/galaxy/tool-deps/samtools/0.1.18/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea; export PACKAGE_BASE; . /home/galaxy/tool-deps/samtools/0.1.18/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea/env.sh; PACKAGE_BASE=/home/galaxy/tool-deps/picard/1.56.0/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea; export PACKAGE_BASE; . /home/galaxy/tool-deps/picard/1.56.0/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea/env.sh; python /home/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_gatk_picard_index_builder/a15709ad26ea/data_manager_gatk_picard_index_builder/data_manager/data_manager_gatk_picard_index_builder.py "/home/galaxy/galaxy-dist/database/files/000/dataset_70.dat" \ --jar "$JAVA_JAR_PATH/CreateSequenceDictionary.jar" \ --fasta_filename "/home/galaxy/galaxy-dist/tool-data/hg19/seq/hg19.fa" \ --fasta_dbkey "hg19" \ --fasta_description "Human, Feb. 2009, GRCh37/hg19" \ --data_table_name "gatk_picard_indexes"; return_code=$?; python /home/galaxy/galaxy-dist/database/job_working_directory/000/63/set_metadata_Ts4xSd.py /home/galaxy/galaxy-dist/database/tmp/tmpfomc7m /home/galaxy/galaxy-dist/database/job_working_directory/000/63/galaxy.json /home/galaxy/galaxy-dist/database/job_working_directory/000/63/metadata_in_HistoryDatasetAssociation_70_Lkhynl,/home/galaxy/galaxy-dist/database/job_working_directory/000/63/metadata_kwds_HistoryDatasetAssociation_70_IuPBQs,/home/galaxy/galaxy-dist/database/job_working_directory/000/63/metadata_out_HistoryDatasetAssociation_70_uuRnIn,/home/galaxy/galaxy-dist/database/job_working_directory/000/63/metadata_results_HistoryDatasetAssociation_70_aVp4XH,/home/galaxy/galaxy-dist/database/files/000/dataset_70.dat,/home/galaxy/galaxy-dist/database/job_working_directory/000/63/metadata_override_HistoryDatasetAssociation_70_hXaBQB; sh -c "exit $return_code" galaxy.jobs.runners.local DEBUG 2015-04-17 11:13:23,476 (63) executing job script: /home/galaxy/galaxy-dist/database/job_working_directory/000/63/galaxy_63.sh galaxy.jobs DEBUG 2015-04-17 11:13:23,526 (63) Persisting job destination (destination id: local) galaxy.jobs.runners.local DEBUG 2015-04-17 11:13:24,976 execution finished: /home/galaxy/galaxy-dist/database/job_working_directory/000/63/galaxy_63.sh galaxy.jobs DEBUG 2015-04-17 11:13:25,054 setting dataset state to ERROR galaxy.jobs DEBUG 2015-04-17 11:13:25,122 job 63 ended galaxy.datatypes.metadata DEBUG 2015-04-17 11:13:25,123 Cleaning up external metadata files Some more system details: Galaxy version: latest_15.03, running on Ubuntu 14.04.2 LTS Java: 1.8.0_45-b14 from the webupd8 Ubuntu repo. Using Postresql locally, everything else local too. Cloned from the Bitbucket repo (in case this matters). Can anyone shed some light on this? In the traceback log above it seems options.fasta_dbkey is being set to “None” rather than “hg19” but I am not sure how to solve this, or if this is indeed the problem. Thank you in advance! All the best, -- Paulo ____________________________________________________________ Paulo Filipe Silva | PhD, Biology | MSc, Bioinformatics Bioinformatician @ CGPP - Centre for Predictive and Preventive Genetics http://www.cgpp.eu <http://www.cgpp.eu/> IBMC - Institute for Molecular and Cell Biology Rua do Campo Alegre, 823 4150-180 Porto, Portugal E-mail: paulo.silva(a)ibmc.up.pt <mailto:paulo.silva@ibmc.up.pt> Telephone: +351 22 607 49 42 ext 2811 Fax: +351 22 600 29 23

1 0

metadata in parallelization
by Roberto Alonso CIPF 17 Apr '15

17 Apr '15

Hello, I am writing some code to enable parallelization for some tool wrappers. First, I did it for simple bwa wrapper, but now I am modifying toolshed.g2.bx.psu.edu/repos/devteam/bwa/c71dd035971e/bwa/bwa-mem.xml to check if the code would work with this wrapper. So, I wrote some code that I thing was necessary in order to merge some bam and I added the parallelism tag (in bold) to the config file: <tool id="bwa_mem" name="BWA-MEM" version="0.1"> <macros> <import>bwa_macros.xml</import> </macros> <requirements> <requirement type="package" version="0.7.10.039ea20639">bwa</requirement> <requirement type="package" version="1.1">samtools</requirement> </requirements> <description>- map medium and long reads (> 100 bp) against reference genome</description> <parallelism method="multi" split_size="3" shared_inputs="ref_file" split_mode="number_of_parts" merge_outputs="bam_output" split_inputs="fastq_input1,fastq_input2" ></parallelism> <command> ... So, everything works well, and the resulting bam from parallelization mode and without the parallelization mode is the same but the Galaxy log throws an error regarding metadata, it says something like this: galaxy.jobs.splitters.multi DEBUG 2015-04-17 09:54:58,335 merge finished: /home/ralonso/galaxy/database/files/000/dataset_198.dat galaxy.jobs.runners.tasks DEBUG 2015-04-17 09:54:58,473 executing external set_meta script for job 200: python /home/ralonso/galaxy/database/tmp/set_metadata_E5fGIE.py /home/ralonso/galaxy/database/tmp/tmpHS8Byo /home/ralonso/galaxy/database/job_working_directory/000/200/galaxy.json /home/ralonso/galaxy/database/tmp/metadata_in_HistoryDatasetAssociation_198_yOGiQG,/home/ralonso/galaxy/database/tmp/metadata_kwds_HistoryDatasetAssociation_198_nAsQoq,/home/ralonso/galaxy/database/tmp/metadata_out_HistoryDatasetAssociation_198_I_cLs4,/home/ralonso/galaxy/database/tmp/metadata_results_HistoryDatasetAssociation_198_qhjzoV,/home/ralonso/galaxy/database/files/000/dataset_198.dat,/home/ralonso/galaxy/database/tmp/metadata_override_HistoryDatasetAssociation_198_ScKLqH Traceback (most recent call last): File "/home/ralonso/galaxy/database/tmp/set_metadata_E5fGIE.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners.tasks DEBUG 2015-04-17 09:54:58,624 execution of external set_meta finished for job 200 galaxy.datatypes.metadata DEBUG 2015-04-17 09:54:58,714 setting metadata externally failed for HistoryDatasetAssociation 198: External set_meta() not called When using no parallelization mode, there is no problem, also because Galaxy doesn't go through this part of code, I mean it doesn't execute this. I see that Galaxy have to do something with metada attributes, but what is t trying to do? is there any way to solve this? Thank you very much Regards, -- Roberto Alonso Functional Genomics Unit Bioinformatics and Genomics Department Prince Felipe Research Center (CIPF) C./Eduardo Primo Yúfera (Científic), nº 3 (junto Oceanografico) 46012 Valencia, Spain Tel: +34 963289680 Ext. 1021 Fax: +34 963289574 E-Mail: ralonso(a)cipf.es

1 0

Difference between GitHub and Mercurial Galaxy download...
by Asma Riyaz 16 Apr '15

16 Apr '15

Hi, Is there a difference between the github and mercurial galaxy downloads? Is there one preferred over another? The galaxy website asks to do a Github download and many other places including the GMOD tutorial held last summer on Galaxy suggested a mercurial one? Kindly advice. Thank you, Asma

3 2

substitution of special characters in arguments of galaxy tools
by Wang, Yu 16 Apr '15

16 Apr '15

Dear galaxy developers, Galaxy seems to substitute special characters when it passes arguments to its tools (e.g. the content of input fields of type 'text'). In general, this is useful. However, '%' is substituted by 'X', i.e. the substitution is not invertible which can be a real show stopper. Is there some way to change this substitution or switch it of completely? Cheers, Yu di29her wang(a)lrz.de<mailto:wang@lrz.de>

2 1

Older galaxy I'm keeping alive by drip feed, has problem restarting when I turn debug off.
by Dooley, Damion 16 Apr '15

16 Apr '15

I have an OLDER Galaxy install I was going to fade out. In meantime I saw that its debug flag should be turned off, so I did. But when it restarted it complained in the history tab with a server error. I put debug back into config and it runs fine. I haven't changed anything recently on it. Wondering if there is a simple remedy? Details below. Thanks, Damion ********************************************************* Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) And indeed, after the startup log message... serving on http://127.0.0.1:8080 There is an error that seems to be triggered when A GET /galaxy/history command is sent. ................ - - [15/Apr/2015:13:03:43 -0700] "GET /galaxy/history HTTP/1.1" 200 - "http://salk.bccdc.med.ubc.ca/galaxy/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36" ................ - - [15/Apr/2015:13:03:43 -0700] "GET /galaxy/history HTTP/1.1" 500 - "http://salk.bccdc.med.ubc.ca/galaxy/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36" Error - <type 'exceptions.TypeError'>: 'NoneType' object is not callable ................ - - [15/Apr/2015:13:03:43 -0700] "GET /galaxy/root/tool_menu HTTP/1.1" 200 - "http://salk.bccdc.med.ubc.ca/galaxy/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36" URL: http://salk.bccdc.med.ubc.ca/galaxy/history File '/usr/lib/python2.6/site-packages/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/lib/python2.6/site-packages/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/usr/lib/python2.6/site-packages/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/production/galaxy-dist/lib/galaxy/web/framework/base.py', line 169 in __call__ return body( environ, start_response ) File '/usr/local/galaxy/production/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 759 in render template.render_context( context ) File '/usr/local/galaxy/production/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg/mako/template.py', line 319 in render_context **kwargs) File '/usr/local/galaxy/production/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/usr/local/galaxy/production/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/usr/local/galaxy/production/galaxy-dist/database/compiled_templates/root/history.mako.py', line 61 in render_body __M_writer(u'\n\n') File '/usr/local/galaxy/production/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 756 in write response_write( d.encode( 'utf-8' ) ) TypeError: 'NoneType' object is not callable Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

3 6

Using synchronous method to upload multiple files
by Daniele P Colobraro 16 Apr '15

16 Apr '15

Hi all, I've set up a simple tool using the synchronous method to upload a file on galaxy. I'm able to visit the external link, select and upload a file by the list. The script that I use is data_source.py and the 'command' part in the xml configuration is the same as the ucsc xml config (ucsc_tablebrowser.xml). But my purpose is to import multiple files. Have you any idea how I can set a way to import/upload multiple files by using the synchronous method? Thank in advance, Best regards Daniele

3 5

April GalaxyAdmins Meetup: April 16
by Dave Clements 16 Apr '15

16 Apr '15

Hello all, The April 2015 GalaxyAdmins meetup <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_04_16> will be held online on Thursday, 16 April (a week from today). We'll meet at 5pm Central European Summer Time (see your local time <http://bit.ly/1EFUf0M>). Carrie Ganote <http://www.researchgate.net/profile/Carrie_Ganote> from the National Center for Genome Analysis Support (NCGAS) <http://ncgas.org/> and Pervasive Technology Institute <http://pti.iu.edu/> at Indiana University will talk about her Galaxy work with Trinity, IU Galaxy, and the Open Science Grid. And Sam Guerler <https://wiki.galaxyproject.org/guerler> of the Galaxy Team will also discuss recent work to update Galaxy's tool forms. GalaxyAdmins <https://wiki.galaxyproject.org/Community/GalaxyAdmins> is a discussion group for Galaxy community members who are responsible for Galaxy installations. Please see the meetup page ( http://bit.ly/gxyadm201504) for more details. Hope to see (hear?) you there, Dave C (channeling Hans-Rudolf Hotz) -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 2

Geting integer value from InputValueWrapper in plugin xml file
by Christian Brenninkmeijer 16 Apr '15

16 Apr '15

Hi, I am a new galaxy developer playing with writing my first plugins In my plugin xml configaration file: I want to be able to increase the value of one parameter based on another using <command interpreter="python"> #if $rows.option =="header_ignore" #set $data_start = $rows.ignore + 1 but I get: unsupported operand type(s) for +: 'InputValueWrapper' and 'int' A hack workaround I found was: #set $data_start = int(str($rows.ignore)) + 1 I just wondered if there was a cleaner way to get the integer value from a InputValueWrapper object. Which is set to integer by: <param name="ignore" size="4" type="integer" value="1" .... If documentation exist somwere please send me an RTFM link. (I didn't find it in https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax) Thanks in advance Christian Brenninkmeijer University of Manchester Ps. If you want to see exactly what I am playing with see https://github.com/Christian-B/galaxy_tools/tree/master/summary (Note that is subject to change as I learn)

3 2

pushing toolshed directories to our own repo
by Nikhil Joshi 16 Apr '15

16 Apr '15

Hi all, So I am trying to push our toolshed directories to our own repo and I am running into problems. Basically, since the toolshed tools are subrepos, I can't figure out how to push those changes into *our* repo. Also, I have had to change some of the toolshed tools because they don't work properly with our galaxy. So, when I try to do the push, it aborts giving me this error: abort: push creates new remote head b9bd905700fe! (in subrepo toolshed.g2.bx.psu.edu/repos/devteam/tophat2/da1f39fe14bc/tophat2) And, of course, I can't push to that repo, nor do I want to. All I want to do is to push our shed tools directory to our repo, but at the same time keep the mercurial information so that when we clone our repo into a new galaxy install, the toolshed will still work from galaxy. How can I do this? Am I even thinking about this the right way? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

2 1