August 2014 - galaxy-dev - lists.galaxyproject.org

Providing BLAST db in a data library
by Ulf Schaefer 14 Aug '14

14 Aug '14

Dear all I have several smallish BLAST databases that I would like to provide in a data library. I create them in a history with the makeblastdb tool and them try to add them to the library. I see that for each blast db there is an empty file created (like /path/dataset_12345.dat) and a folder with the same name (/path/dataset_12345_files/) that contains the actual db files (blastdb.n*). In my library the blastdb shows up empty and I cannot import it back to another history. I does not seem to be aware of the _files folder, despite it being the right data type (blastdbn). Any ideas what I am doing wrong? Thanks a lot for your help Ulf ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

4 11

unable to stob jobs
by Michael Mason 13 Aug '14

13 Aug '14

Hello all, I am running a Galaxy instance on slurm. I am unable to stop jobs via the admin "Manage Jobs" window. Via postgres, there are no jobs in the job table. Any thoughts? I realize this may not be dev but I am at a loss of what to do. Thanks Mike ________________________________ --CONFIDENTIALITY NOTICE--: The information contained in this email is intended for the exclusive use of the addressee and may contain confidential information. If you are not the intended recipient, you are hereby notified that any form of dissemination of this communication is strictly prohibited. www.benaroyaresearch.org

2 1

Galaxy server broken with install of latest changes
by Tony Kusalik 13 Aug '14

13 Aug '14

Hi, I installed the latest set of changes to Galaxy, and now the Galaxy server will not start up. I am hoping someone can point me to the source of the problem and/or a fix. This morning my Galaxy server was running just fine. I did a hg incoming and was told that a fair number of changes were waiting to be applied, changeset 13772 through changeset 14309. Fine. I did hg pull -u and got searching for changes adding changesets adding manifests adding file changes added 538 changesets with 1738 changes to 542 files 473 files updated, 0 files merged, 152 files removed, 0 files unresolved Fine. However, now my Galaxy server will not start up and run. On sh run.sh --daemon the server dies with the following produced in paster.log where /birl2/Galaxy/galaxy-dist is the name of the installation directory. -- begin paster.log -- Traceback (most recent call last): File "./scripts/paster.py", line 33, in <module> serve.run() File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 10 49, in run invoke(command, command_name, options, args[1:]) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 10 55, in invoke exit_code = runner.run(args) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 22 0, in run result = self.command() File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 64 3, in command app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 350, in loadapp return loadobj(APP, uri, name=name, **kw) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 374, in loadobj global_conf=global_conf) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 399, in loadcontext global_conf=global_conf) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 423, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 561, in get_context section) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 620, in _context_from_explicit value = import_string(found_expr) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 125, in import_string return pkg_resources.EntryPoint.parse("x=" + s).load(False) File "/birl2/Galaxy/galaxy-dist/lib/pkg_resources.py", line 1954, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/web/__init__.py", line 4, in <modul e> from framework import expose File "/birl2/Galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 48 , in <module> eggs.require( "amqp" ) File "/birl2/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 414, in req uire raise EggNotFetchable( str( [ egg.name for egg in e.eggs ] ) ) galaxy.eggs.EggNotFetchable: ['amqp'] Removing PID file paster.pid -- end paster.log -- The above is the complete content of paster.log. Anyone have any idea what the problem is? More importantly, how do I fix the problem and get Galaxy running again?

2 3

Problems uploading files to a data library
by Juan Vladimir de la Rosa Medina 12 Aug '14

12 Aug '14

Hi everyone,I recently have installed a local instance of galaxy in my computer, everything worked fine until I changed the path for library_import_dir in the universe_wsgi.ini file, to upload files to data libraries.I followed the Galaxy documentation to setup this feature:This is the path in the universe_wsgi.ini filelibrary_import_dir = /media/New/Vladimir/My_RNA-seq/The steps I followed are: Admin > Data Library > Create new data library > Add datasets > Upload directory of files file format was set to auto-detect server Directory was /media/New/Vladimir/My_RNA-seq/ and we chose the option to link to files instead of copying themWhen I try to upload files, galaxy displays the following error:Error Traceback:⇝ AttributeError: 'NoneType' object has no attribute 'new_state'<?xml version="1.0" ?><traceback> <sysinfo> <language version="2.7.3">Python</language> </sysinfo> <stack> <frame> <module>weberror.evalexception.middleware</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py</filename> <line>364</line> <function>respond</function> <operation>app_iter = self.application(environ, detect_start_response)</operation> <operation_context> try: __traceback_supplement__ = errormiddleware.Supplement, self, environ app_iter = self.application(environ, detect_start_response) try: return_iter = list(app_iter)</operation_context> </frame> <frame> <module>paste.recursive</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py</filename> <line>84</line> <function>__call__</function> <operation>return self.application(environ, start_response)</operation> <operation_context> environ['paste.recursive.script_name'] = my_script_name try: return self.application(environ, start_response) except ForwardRequestException, e: middleware = CheckForRecursionMiddleware(</operation_context> </frame> <frame> <module>paste.httpexceptions</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py</filename> <line>633</line> <function>__call__</function> <operation>return self.application(environ, start_response)</operation> <operation_context> []).append(HTTPException) try: return self.application(environ, start_response) except HTTPException, exc: return exc(environ, start_response)</operation_context> </frame> <frame> <module>galaxy.web.framework.base</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/lib/galaxy/web/framework/base.py</filename> <line>132</line> <function>__call__</function> <operation>return self.handle_request( environ, start_response )</operation> <operation_context> self.trace( message="Starting request" ) try: return self.handle_request( environ, start_response ) finally: self.trace( message="Handle request finished" )</operation_context> </frame> <frame> <module>galaxy.web.framework.base</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/lib/galaxy/web/framework/base.py</filename> <line>190</line> <function>handle_request</function> <operation>body = method( trans, **kwargs )</operation> <operation_context> kwargs.pop( '_', None ) try: body = method( trans, **kwargs ) except Exception, e: body = self.handle_controller_exception( e, trans, **kwargs )</operation_context> </frame> <frame> <module>galaxy.webapps.galaxy.controllers.library_common</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/library_common.py</filename> <line>927</line> <function>upload_library_dataset</function> <operation>**kwd )</operation> <operation_context> widgets=widgets, replace_dataset=replace_dataset, **kwd ) if created_outputs_dict: if cntrller == 'api':</operation_context> </frame> <frame> <module>galaxy.webapps.galaxy.controllers.library_common</module> <filename>/home/labinmunologiaulpgc/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/library_common.py</filename> <line>1049</line> <function>upload_dataset</function> <operation>state = tool.new_state( trans )</operation> <operation_context> tool_id = 'upload1' tool = trans.app.toolbox.get_tool( tool_id ) state = tool.new_state( trans ) errors = tool.update_state( trans, tool.inputs_by_page[0], state.inputs, kwd ) tool_params = state.inputs</operation_context> </frame> </stack> <exception> <type>AttributeError</type> <value>'NoneType' object has no attribute 'new_state'</value> </exception></traceback> I have been looking for a solution to fix this problem on the web without success. thank you very much

3 4

Installed tool invisible in Galaxy
by Peter van Heusden 12 Aug '14

12 Aug '14

Hi there I updated a tool of mine recently, committed to the local toolshed, did the update from within the Admin panel and then got this error: Error - <type 'exceptions.AttributeError'>: 'Tool' object has no attribute 'citations' A restart of Galaxy seemed to fix the problem, but since then the tool could not be found - it is not in any of the Tool categories. Deleting and re-installing the tool does not fix this problem - and on installation time I never get asked what group to add it to. It seems as if it is somehow "stuck" in an incorrect state. Any ideas where to go poking? Presumably there is somewhere in the database to go poking to "reset" things. Thanks, Peter P.S. this is with the latest Galaxy from galaxy-central.

2 4

Determining datatype inheritance in tool XML Cheetah
by Peter Cock 12 Aug '14

12 Aug '14

Hi all, I've just uploaded a simple sequence composition tool to the Test Tool Shed: https://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_composition https://github.com/peterjc/pico_galaxy/commit/45669446f5a14fd90a8a0d9d74304… This accepts multiple input in FASTA, FASTQ, or SFF format - and allows a mixture of these: <inputs> <param name="input_file" type="data" format="fasta,fastq,sff" multiple="true" label="Sequence file" help="FASTA, FASTQ, or SFF format." /> </inputs> In order to build the command line string, I am currently using this for loop: <command interpreter="python"> seq_composition.py -o "$output_file" ##For loop over inputs #for i in $input_file --$i.ext "${i}" #end for </command> This results in things like this being run: seq_composition.py -o XXX.dat --fastqsanger XXX.dat --sff XXX.dat This works, but means my Python script has to know about not just the core data types that I specified in my input parameter XML (fasta,fastq,sff) but also any subclasses (e.g. fastqsanger). It seems what I want/need would be something along these lines in pseudo-code to map any datatype which is a subclass for fastq to use a single command line option: <command interpreter="python"> seq_composition.py -o "$output_file" ##For loop over inputs #for i in $input_file #if isinstance($i.datatype, fastq): --fastq "${i}" #else --$i.ext "${i}" #end if #end for </command> This mock example borrows from the Python isinstance function, but of course some Galaxy datatypes are defined as subclasses at the XML level rather than literally at the Python class level. This should result in getting the following regardless of which flavour of FASTQ the input dataset had assigned: seq_composition.py -o XXX.dat --fastq XXX.dat --sff XXX.dat Does anyone have any Tool XML examples probing an input file's datatype in this way? Peter

2 2

bug?
by Langhorst, Brad 12 Aug '14

12 Aug '14

I see this in my database server logs… SELECT history_dataset_association.id AS history_dataset_association_id, history_dataset_association.history_id AS history_dataset_association_history_id, history_dataset_association.dataset_id AS history_dataset_association_dataset_id, history_dataset_association.create_time AS history_dataset_association_create_time, history_dataset_association.update_time AS history_dataset_association_update_time, history_dataset_association.state AS history_dataset_association_state, history_dataset_association.copied_from_history_dataset_association_id AS history_dataset_association_copied_from_history_dataset_a_1, history_dataset_association.copied_from_library_dataset_dataset_association_id AS history_dataset_association_copied_from_library_dataset_d_2, history_dataset_association.hid AS history_dataset_association_hid, history_dataset_association.name AS history_dataset_association_name, history_dataset_association.info AS history_dataset_association_info, history_dataset_association.blurb AS history_dataset_association_blurb, history_dataset_association.peek AS history_dataset_association_peek, history_dataset_association.tool_version AS history_dataset_association_tool_version, history_dataset_association.extension AS history_dataset_association_extension, history_dataset_association.metadata AS history_dataset_association_metadata, history_dataset_association.parent_id AS history_dataset_association_parent_id, history_dataset_association.designation AS history_dataset_association_designation, history_dataset_association.deleted AS history_dataset_association_deleted, history_dataset_association.purged AS history_dataset_association_purged, history_dataset_association.visible AS history_dataset_association_visible, history_dataset_association.hidden_beneath_collection_instance_id AS history_dataset_association_hidden_beneath_collection_ins_3, history_dataset_association.extended_metadata_id AS history_dataset_association_extended_metadata_id, dataset_1.id AS dataset_1_id, dataset_1.create_time AS dataset_1_create_time, dataset_1.update_time AS dataset_1_update_time, dataset_1.state AS dataset_1_state, dataset_1.deleted AS dataset_1_deleted, dataset_1.purged AS dataset_1_purged, dataset_1.purgable AS dataset_1_purgable, dataset_1.object_store_id AS dataset_1_object_store_id, dataset_1.external_filename AS dataset_1_external_filename, dataset_1._extra_files_path AS dataset_1__extra_files_path, dataset_1.file_size AS dataset_1_file_size, dataset_1.total_size AS dataset_1_total_size, dataset_1.uuid AS dataset_1_uuid FROM history_dataset_association LEFT OUTER JOIN dataset AS dataset_1 ON dataset_1.id = history_dataset_association.dataset_id WHERE history_dataset_association.id = ‘49b19822485c7897' looks like a simple bug to me. But I don’t know an easy way to track it down. Any ideas? Brad -- Bradley W. Langhorst, Ph.D. Applications and Product Development Scientist

3 6

can't install numpy from toolshed
by Sajdak, Doris 12 Aug '14

12 Aug '14

I'm running Galaxy in a virtual environment as described here (OS is Debian 3.2.54-2): https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer#Us… When I go to install package_numpy_1_7 from the tool shed, I get the following errors: Running from numpy source directory. /ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/config.py:413: DeprecationWarning: +++++++++++++++++++++++++++++++++++++++++++++++++ Usage of get_output is deprecated: please do not use it anymore, and avoid configuration checks involving running executable on the target machine. +++++++++++++++++++++++++++++++++++++++++++++++++ DeprecationWarning) Traceback (most recent call last): File "setup.py", line 214, in <module> setup_package() File "setup.py", line 207, in setup_package configuration=configuration ) File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/core.py", line 186, in setup return old_setup(**new_attr) File "/usr/lib/python2.6/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib/python2.6/distutils/dist.py", line 975, in run_commands self.run_command(cmd) File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command cmd_obj.run() File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/install.py", line 55, in run r = old_install.run(self) File "/usr/lib/python2.6/distutils/command/install.py", line 615, in run self.run_command('build') File "/usr/lib/python2.6/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command cmd_obj.run() File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/build.py", line 37, in run old_build.run(self) File "/usr/lib/python2.6/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/usr/lib/python2.6/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command cmd_obj.run() File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/build_src.py", line 152, in run self.build_sources() File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/build_src.py", line 169, in build_sources self.build_extension_sources(ext) File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/build_src.py", line 328, in build_extension_sources sources = self.generate_sources(sources, ext) File "/ifs/projects/galaxy/database/tmp/tmp-toolshed-mtdaP2Rhs/numpy-1.7.1/numpy/distutils/command/build_src.py", line 385, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 410, in generate_config_h moredefs, ignored = cocache.check_types(config_cmd, ext, build_dir) File "numpy/core/setup.py", line 41, in check_types out = check_types(*a, **kw) File "numpy/core/setup.py", line 271, in check_types "Cannot compile 'Python.h'. Perhaps you need to "\ SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel. python-dev is installed on the machine, though I don't know if that matters in the virtual environment. Any idea what's causing this? THANKS! Dori ********************************** Dori Sajdak Senior Systems Administrator State University of NY at Buffalo Center for Computational Research 701 Ellicott St Buffalo, NY 14203 Phone: (716) 881-8934 Fax: (716) 849-6656 Web: http://ccr.buffalo.edu **********************************

2 3

Re: [galaxy-dev] Galaxy error
by John Chilton 12 Aug '14

12 Aug '14

Hello, Sorry for the delay - please keep all correspondences on galaxy-dev to maximize the likelihood someone will respond and respond promptly. I do not know what would cause this error - does anyone else have any ideas? -John On Mon, Aug 4, 2014 at 2:07 PM, Beck, Emily A <emily-beck(a)uiowa.edu> wrote: > Sorry for my delayed response. The error is on the main public server, > and happens when I try to map with Bowtie for Illumina. Thank you. > > ~Emily > > > > > *Emily Beck *PhD Candidate > Llopart Lab > 469 Biology Building > Iowa City, IA 52242 > Lab: (319)335-3430 > ------------------------------ > *From:* John Chilton [jmchilton(a)gmail.com] > *Sent:* Tuesday, July 22, 2014 4:06 PM > *To:* Beck, Emily A > *Cc:* galaxy-dev(a)bx.psu.edu > *Subject:* Re: [galaxy-dev] Galaxy error > > Hello, > > Thanks for the bug report. Is this on the main public server ( > usegalaxy.org) or a local instance at the University of Iowa? > > -John > > > On Wed, Jul 16, 2014 at 10:39 AM, Beck, Emily A <emily-beck(a)uiowa.edu> > wrote: > >> >> Hello, >> >> I have repeatedly gotten the following error message when attempting to >> use both the mapping and pileup functions in Galaxy. I am using files that >> I have previously used successfully with these functions, and cannot fix it. >> >> Thank you. >> >> ~Emily Beck >> >> >> user username emilybeck quota_percent 19 total_disk_usage 52086193970 >> nice_total_disk_usage 48.5 GB email emily-beck(a)uiowa.edu is_admin false >> tags_used model_class User id 03f0554e2314759e source >> HDACollection(f313d1d65c4ee57e,451) xhr readyState 4 responseText {"err_msg": >> "Uncaught exception in exposed API method:", "err_code": 0} responseJSON >> err_msg Uncaught exception in exposed API method: err_code 0 status >> 500 statusText Internal Server Error responseHeaders Server nginx/1.4.7 >> Date Wed, 16 Jul 2014 15:19:52 GMT Content-Type application/json >> Transfer-Encoding chunked Connection keep-alive Cache-Control max-age=0,no-cache,no-store >> options data parse true emulateHTTP false emulateJSON false >> >> >> >> *Emily Beck *PhD Candidate >> Llopart Lab >> 469 Biology Building >> Iowa City, IA 52242 >> Lab: (319)335-3430 >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > >

2 1

Bug report: <page> within <help>
by Hans-Philipp Brachvogel 12 Aug '14

12 Aug '14

Hello, I found that it is possible for tool-xmls defined with <page> tags to also have multipage helps, but this feature is not really documented and also does not work because of a tiny bug. You can have multipage helps by writing something like this (if your tool has two <page>s): <help> <page> Helptext for page 1 </page> <page> Helptext for page 2 </page> </help> But for it to work one has to fix the following: rev 13771:7a4d321c0e38 file lib/galaxy/tools/__init__.py expression starting at line 1439: self.help_by_page = [ Template( rst_to_html( help_header + x + help_footer, input_encoding='utf-8', output_encoding='utf-8', default_filters=[ 'decode.utf8' ], encoding_errors='replace' ) ) for x in self.help_by_page ] The bracket is misplaced and the whole thing should instead be: self.help_by_page = [ Template( rst_to_html( help_header + x + help_footer), input_encoding='utf-8', output_encoding='utf-8', default_filters=[ 'decode.utf8' ], encoding_errors='replace' ) for x in self.help_by_page ] One question: <page> is deprecated, does this mean there will be no multipage tools in the future? Or will there be (or is already?) an alternative system? Best, Philipp

3 3