- galaxy-dev - lists.galaxyproject.org

Intermittent workflow errors with local galaxy running postgresql 9.3
by Christensen, Todd 05 Oct '13

05 Oct '13

Hi All, We have been having issues with workflows generating intermittent errors using sqlite. We recently converted to postgresql 9.3 in an attempt to resolve the issue as suggested in prior posts. Despite switching to postgresql we continue to have these errors. Does anyone have any additional suggestions that may resolve this issue? Thanks, Todd Traceback (most recent call last): File "/opt/paml/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 122, in prepare_job job_wrapper.runner_command_line = self.build_command_line( job_wrapper, include_metadata=include_metadata, include_work_dir_outputs=include_work_dir_outputs ) File "/opt/paml/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 172, in build_command_line work_dir_outputs = self.get_work_dir_outputs( job_wrapper ) File "/opt/paml/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 222, in get_work_dir_outputs for dataset in dataset_assoc.dataset.dataset.history_associations + dataset_assoc.dataset.dataset.library_associations: File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 168, in __get__ return self.impl.get(instance_state(instance),dict_) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 453, in get value = self.callable_(state, passive) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py", line 508, in _load_for_state return self._emit_lazyload(session, state, ident_key) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py", line 574, in _emit_lazyload result = q.all() File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py", line 2115, in all return list(self) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py", line 2227, in __iter__ return self._execute_and_instances(context) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py", line 2242, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "/opt/paml/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) database is locked u'SELECT library_dataset_dataset_association.id AS library_dataset_dataset_association_id, PAML EMAIL DISCLAIMER: Information contained in this message may be privileged and confidential. If the reader of this message is not the intended recipient, be notified that any dissemination, distribution or copying of this communication is strictly prohibited. If this communication is received in error, please notify the sender immediately by replying to the message and deleting from your computer. Thank you

2 1

multiple input files
by Ulf Schaefer 04 Oct '13

04 Oct '13

Dear all We frequently find ourselves in situations where a tool needs to be run with a lot of input files. For example, run the GATK UnifiedGenotyper with easily dozens of bam files. Using the <repeat> in this case requires quite a bit of clicking. Is there a more conventient way fo doing this? Maybe similar to the multi-file-select that is possible for workflow inputs? I saw some older discussions on this or similar issues, but I am a bit lost what the current official stable proposed solution for this is. Thanks for your help Ulf ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

4 8

Re: [galaxy-dev] [galaxy-user] slow Galaxy server
by Peter Cock 03 Oct '13

03 Oct '13

Hello all, One of the users on our local galaxy has reported this error, trying IE9, Firefox and Chrome on Windows: "An error occurred while getting updates from the server. Please contact a Galaxy administrator if the problem persists." Meanwhile the server works fine for me. Are there any known causes of this error (other than heavy load which does not seem to be the case here)? Thanks, Peter On Fri, Aug 2, 2013 at 5:49 PM, Politz, Samuel M. <spolitz(a)wpi.edu> wrote: > This is a followup to my previous post on this topic. The exact message (in > red) that I see after a long delay in running a tool is “An error occurred > while getting updates from the server. Please contact a Galaxy administrator > if the problem persists.” I have seen this message twice while waiting for > two unrelated tools to run. There is no green’bug’ icon so I could not > report this to an administrator. Another main server user reported this > error message to the user mailing list as well. The response to their post > was to wait a few minutes. Well, in this case I have been waiting > overnight. > > > > Best regards, > > Sam Politz > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/

1 0

Functional tests broken, NameError: global name 'db_path' is not defined
by Peter Cock 03 Oct '13

03 Oct '13

Hi all, Something has broken on the functional test framework. Attempting to run the tests fails, as does just listing the IDs: $ ./run_functional_tests.sh --list /mnt/galaxy/galaxy-central/eggs/twill-0.9-py2.6.egg/twill/other_packages/_mechanize_dist/_auth.py:14: DeprecationWarning: the md5 module is deprecated; use hashlib instead import re, base64, urlparse, posixpath, md5, sha, sys, copy /mnt/galaxy/galaxy-central/eggs/twill-0.9-py2.6.egg/twill/other_packages/_mechanize_dist/_auth.py:14: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import re, base64, urlparse, posixpath, md5, sha, sys, copy /mnt/galaxy/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility from csamtools import * Traceback (most recent call last): File "./scripts/functional_tests.py", line 457, in <module> sys.exit( main() ) File "./scripts/functional_tests.py", line 246, in main database_connection = 'sqlite:///' + os.path.join( db_path, 'universe.sqlite' ) NameError: global name 'db_path' is not defined 'run_functional_tests.sh help' for help $ hg branch stable [galaxy@ppserver galaxy-central]$ hg heads | more changeset: 11805:a3c018044538 tag: tip user: greg date: Wed Oct 02 16:44:48 2013 -0400 summary: Eliminate the problematic "Select one" option from select fields. changeset: 11761:a477486bf18e branch: stable parent: 11689:4b1c7ad464ea user: Nate Coraor <nate(a)bx.psu.edu> date: Thu Sep 26 11:02:58 2013 -0400 summary: Bugfix for tool-to-destination mapping, tool ids are lowercased but the mapping id was not lowercased. ... Regards, Peter

2 3

Please save these dates! GCC2014: June 30 - July 2
by Dave Clements 03 Oct '13

03 Oct '13

Hello all, The 2014 Galaxy Community Conference (GCC2014)<http://wiki.galaxyproject.org/Events/GCC2014> has been scheduled for June 30 through July 2, at the Homewood Campus<http://webapps.jhu.edu/jhuniverse/information_about_hopkins/campuses/homewo…> of Johns Hopkins University <http://jhu.edu/>, in Baltimore, Maryland<http://visitors.baltimorecity.gov/>, United States. Galaxy Community Conferences are an opportunity to participate in presentations, discussions, poster sessions, lightning talks and breakouts, all about high-throughput biology and the tools that support it. The conference will also includes a*Training Day* offering in-depth topic coverage, across several concurrent sessions. See the GCC2013 web site<http://wiki.galaxyproject.org/Events/GCC2013> for an idea of what happens at a Galaxy Community Conference. See you next summer, GCC2014 Organizing Committee<http://wiki.galaxyproject.org/Events/GCC2014/Organizers> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0

URL problem
by Jennifer Jackson 03 Oct '13

03 Oct '13

Alper, Did you problem get resolved already? Maybe on the galaxy-dev(a)bx.psu.edu mailing list? I am posting it there in case it is still open. Please reply to let us know your current status. Please drop galaxy-user(a)bx.psu.edu from the to/cc on reply and future posts to this thread. Thanks and sorry if this one got missed, Jen Galaxy team On 8/26/13 10:43 AM, Kucukural, Alper wrote: > Hi, > I have a problem in galaxy to get host/domain name in two different > pages. > First one is in the tool installation from toolshed, I got the error > below, > > ##### > > > Not Found > > The requested URL /admin_toolshed/prepare_for_install was not found on > this server. > > ##### > > The second one is in the saved histories. When I click the buttons of > the saved histories. I got the similar error like below. > ##### > > > Not Found > > The requested URL /history/list was not found on this server. > > ##### > > I haven't seen these any other pages yet. > > My installation is working on LDAP authentication with Proxy. So, I > could not find a place to set the domain or host name in these two > places that they can actually find the requested URLs. > > In the paster.log file. I don't get any error when I install a tool > or go to another history. It doesn't report any error. > > Thanks for your help, > > Alper Kucukural > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ -- Jennifer Hillman-Jackson http://galaxyproject.org

2 1

Toolshed Collaboration Issues
by Lance Parsons 02 Oct '13

02 Oct '13

I'm having some difficulty collaborating on a tool development in the toolshed. It seems to happen on the production toolshed as well as the testtoolshed. When I try to grant another user the ability to update a repository (or when the other person tries to add me) we get the following error: URL: http://testtoolshed.g2.bx.psu.edu/repository/manage_repository?id=f49c56ded… File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/framework/__init__.py', line 98 in decorator return func( self, trans, *args, **kwargs ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/webapps/tool_shed/controllers/repository.py', line 2124 in manage_repository user = trans.sa_session.query( trans.model.User ).get( trans.security.decode_id( user_id ) ) File '/var/opt/galaxy/g2cmnty/galaxy_testtoolshed/lib/galaxy/web/security/__init__.py', line 60 in decode_id return int( self.id_cipher.decrypt( obj_id.decode( 'hex' ) ).lstrip( "!" ) ) File '/usr/lib/python2.7/encodings/hex_codec.py', line 42 in hex_decode output = binascii.a2b_hex(input) TypeError: Non-hexadecimal digit found Can anyone else successfully grant a user update permission to a repository? I've submitted an issue on Trello (https://trello.com/c/XYkqW3Lr) -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

2 3

Slice bam
by jean_jasinski＠agilent.com 02 Oct '13

02 Oct '13

It seems like Slice_bam should be pretty straight-forward to use, but when I run it and give it a bed file that contains the human chromosomes with their lengths like this: chr1 1 249250621 chr2 1 243199373 for all chromosomes 1-22, X, Y. I don't get the reads on chr1 and 2; instead, all I am left with are the reads on the nonstandard chromosomes (the unmapped scaffolds). Any idea what I am doing wrong? Jean Jasinski

1 0

Galaxy STARTUP ERROR!
by Giovanni Pascarella 02 Oct '13

02 Oct '13

Hi! I am trying to launch Galaxy locally but I'm getting the following error: tkx417:galaxy pascarellagiovanni$ sh run.sh Traceback (most recent call last): File "./scripts/paster.py", line 33, in <module> serve.run() File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/serve.py", line 1049, in run invoke(command, command_name, options, args[1:]) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/serve.py", line 1055, in invoke exit_code = runner.run(args) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/serve.py", line 220, in run result = self.command() File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/serve.py", line 643, in command app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 350, in loadapp return loadobj(APP, uri, name=name, **kw) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 374, in loadobj global_conf=global_conf) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 399, in loadcontext global_conf=global_conf) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 423, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 561, in get_context section) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 620, in _context_from_explicit value = import_string(found_expr) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 125, in import_string return pkg_resources.EntryPoint.parse("x=" + s).load(False) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/pkg_resources.py", line 1954, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/web/__init__.py", line 4, in <module> from framework import expose File "/Users/pascarellagiovanni/Desktop/galaxy/lib/galaxy/web/framework/__init__.py", line 40, in <module> from babel.support import Translations File "/Users/pascarellagiovanni/Desktop/galaxy/eggs/Babel-0.9.4-py2.7.egg/babel/support.py", line 29, in <module> from babel.dates import format_date, format_datetime, format_time, LC_TIME File "/Users/pascarellagiovanni/Desktop/galaxy/eggs/Babel-0.9.4-py2.7.egg/babel/dates.py", line 34, in <module> LC_TIME = default_locale('LC_TIME') File "/Users/pascarellagiovanni/Desktop/galaxy/eggs/Babel-0.9.4-py2.7.egg/babel/core.py", line 642, in default_locale return '_'.join(filter(None, parse_locale(locale))) File "/Users/pascarellagiovanni/Desktop/galaxy/eggs/Babel-0.9.4-py2.7.egg/babel/core.py", line 763, in parse_locale raise ValueError('expected only letters, got %r' % lang) ValueError: expected only letters, got 'utf-8' Anybody knows what this is about? Thanks, Giovanni

2 1

Re: [galaxy-dev] Dynamic data library
by Hans-Rudolf Hotz 02 Oct '13

02 Oct '13

On 10/01/2013 03:53 PM, Cole, Nathan (NIH/NCI) [C] wrote: > Thank you both for your responses. I will be looking into both of these. > > With regard to the from_file option to add the sample selection into the tool: I assume this means that the metadata and everything is loaded into galaxy at the time the tool is run. This depends on how you write your tool. Do you just wanna read the ie fastq file or do you also wanna read the meta data. Also, how is the meta data accessible? eg. is it stored in a txt file at the same location as the fastq file? > Does this create a copy of the loaded file or simply read it in place? Also are there any efficiency issues created using this method, outside of the tool run time increase due to the load of the data taking place in-tool? It should just read it in place Hans-Rudolf > > Thanks, > Nathan > > -----Original Message----- > From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] > Sent: Tuesday, October 01, 2013 4:07 AM > To: Cole, Nathan (NIH/NCI) [C] > Cc: Martin Čech; galaxy-dev(a)lists.bx.psu.edu > Subject: Re: [galaxy-dev] Dynamic data library > > Hi Nathan > > > Do you have many tools working with those samples or just a few? If you only have a limited, predefined set of tools you might wanna consider adding the sample selection into the tool. > > You can use the from_file, or from_data_table options to dynamically create sample selection list. You can even drill down a hierarchical list. Have a look at ~/tools/annotation_profiler/annotation_profiler.xml > which uses the file > ~/tool-data/annotation_profiler_options.xml > > All you need to do is keeping the file in sync with the directory structure of your samples directory > > > Regards, Hans-Rudolf > > > > > > > On 09/30/2013 09:48 PM, Martin Čech wrote: >> Hi Nathan, >> >> Dannon answered similar question few days ago: >> >> There's an import mechanism in libraries that'll allow you to simply >> link to the file on disk without copy/upload. I believe the >> "example_watch_folder.py" sample script (in the distribution) does >> just this via the API, if you want an example. >> >> >> This might be what you are looking for. >> >> Martin >> >> >> On Mon, Sep 30, 2013 at 2:43 PM, Cole, Nathan (NIH/NCI) [C] >> <nathan.cole(a)nih.gov <mailto:nathan.cole@nih.gov>> wrote: >> >> Hello, we’ve set up a local Galaxy instance in our genotyping and >> next-gen sequencing lab with local Apache LDAP (AD) integration, NFS >> mounts to a large NAS, and cluster integration coming. Do to the >> high volume of samples and staff that will be using the system, I >> want to set up data libraries (without copying to Galaxy). This is >> obviously no problem the first time, however I was wondering if >> there was a way to make a library, added from a system path, be >> dynamic so that it would stay synchronized with the underlying file >> structure?____ >> >> __ __ >> >> If a try dynamic library is not possible, is there a method for >> adding files to an existing library via that same system path that >> would not duplicate all of the original files in the data library?____ >> >> __ __ >> >> I did some scouring of the list and found some old unanswered >> questions and some things tangentially related topics, but I was >> unable to find a true answer or solution to my problem. Any >> information on how to do the tasks above or other solutions to >> provide the same functionality would be greatly appreciated.____ >> >> __ __ >> >> Thanks,____ >> >> Nathan____ >> >> __ __ >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >>

2 2