August 2012 - galaxy-dev - lists.galaxyproject.org

Restarting Galaxy
by Kenneth R. Auerbach 11 Sep '12

11 Sep '12

Hello, Can someone tell me if stopping and then restarting the Galaxy process would prevent currently-running jobs from completing after the restart? Will all the jobs remain in the queue after the restart? Also, what is the best way to restart the Galaxy server? Users are complaining that jobs are extremely slow to start running. Is there a way to speed things up? Any info or suggestions would be greatly appreciated. Thank you.

2 1

Shell script to start Galaxy in multi-server environment
by Karger, Amir 11 Sep '12

11 Sep '12

We're upgrading to a late June Galaxy from a last-year Galaxy. We noticed that the docs say you no longer need 2 different .ini files. Great! Unfortunately, the multiprocess.sh in contrib/ still assumes you have multiple .ini files. So the question is, assuming we correctly set up the different web servers, job managers, and job handlers servers in universe_wsgi.ini, what's the command line we should be giving to run Galaxy on each type of server? The wiki Admin/Config pages for Performance and Scaling and Cluster and that sort of thing had some info on editing the .ini, but I didn't see what my .sh should look like there. Pointers to websites, emails, or existing .sh files appreciated. Thanks, -Amir Karger Senior Research Computing Consultant Harvard Medical School Research Computing amir_karger(a)hms.harvard.edu

3 7

Server Error in history framer when debug = False
by David Roquis 11 Sep '12

11 Sep '12

Dear Galaxy community, I have noticed on forums that some users are experiencing this problem too, but no clues on how to fix it. When I set debug = false in universe.wsgi.ini, I get a server error in the history frame (history works perfectly when debug = true). Trying to create a new history, or to upload a saved one, doesn't work either. I use postgresql and nginx, but I this problem also appears on a freshly cloned instance of galaxy-dist. I tried on two machines with Fedora 13 + python 2.6.4 and Fedora 17 + python 2.7.3. As my goal is to use Galaxy in a production environment (with 2 webservers, 1 manager and 2 handlers), I would really like to be able to turn off debug. Here is what I can find in my paster.log when I turn off debug. Thanks a lot if anyone has a solution! David Starting server in PID 4664. serving on 0.0.0.0:8081 view at http://127.0.0.1:8081 127.0.0.1 - - [03/Aug/2012:16:26:49 +0200] "GET / HTTP/1.0" 200 - "-" "Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1" 127.0.0.1 - - [03/Aug/2012:16:26:49 +0200] "GET /root/tool_menu HTTP/1.0" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1" 127.0.0.1 - - [03/Aug/2012:16:26:49 +0200] "GET /history HTTP/1.0" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1" 127.0.0.1 - - [03/Aug/2012:16:26:49 +0200] "GET /history HTTP/1.0" 500 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1" Error - <type 'exceptions.TypeError'>: 'NoneType' object is not callable URL: http://127.0.0.1/history File '/usr/lib/python2.7/site-packages/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/lib/python2.7/site-packages/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/lib/python2.7/site-packages/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxy2ei/galaxy-dist/lib/galaxy/web/framework/base.py', line 169 in __call__ return body( environ, start_response ) File '/home/galaxy2ei/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 772 in render template.render_context( context ) File '/home/galaxy2ei/galaxy-dist/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 319 in render_context **kwargs) File '/home/galaxy2ei/galaxy-dist/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/home/galaxy2ei/galaxy-dist/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/home/galaxy2ei/galaxy-dist/database/compiled_templates/root/history.mako.py', line 62 in render_body __M_writer(u'\n\n') File '/home/galaxy2ei/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 769 in write response_write( d.encode( 'utf-8' ) ) TypeError: 'NoneType' object is not callable CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-us,en;q=0.5' HTTP_CONNECTION: 'close' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a1a924b5a883eb84f93f9e2c6424cabed485f00c49dc51dd6632e1e679e865175' HTTP_HOST: '127.0.0.1' HTTP_REFERER: 'http://127.0.0.1:8080/' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1' HTTP_X_FORWARDED_FOR: '127.0.0.1' HTTP_X_FORWARDED_HOST: '127.0.0.1' ORGINAL_HTTP_HOST: 'galaxy_app' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/history' REMOTE_ADDR: '127.0.0.1' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8081' SERVER_PROTOCOL: 'HTTP/1.0' Configuration ------------- __file__: '/home/galaxy2ei/galaxy-dist/universe_wsgi.ini' admin_users: 'david.roquis(a)yahoo.com, galaxytest(a)2ei.com' allow_library_path_paste: 'True' allow_user_dataset_purge: 'True' allow_user_deletion: 'True' allow_user_impersonation: 'True' database_connection: 'postgres://galaxy2ei:galaxy2ei@localhost:5432/galaxyprod' database_engine_option_server_side_cursors: 'True' database_engine_option_strategy: 'threadlocal' datatypes_config_file: 'datatypes_conf.xml' debug: 'False' default_job_handlers: 'handler0' enable_quotas: 'True' file_path: 'database/files' here: '/home/galaxy2ei/galaxy-dist' job_handlers: 'handler0' job_manager: 'manager' job_working_directory: 'database/job_working_directory' library_import_dir: '/home/galaxy2ei/Downloads' new_file_path: 'database/tmp' nginx_upload_path: '/_upload' nginx_upload_store: 'database/tmp/upload_store' nginx_x_accel_redirect_base: '/_x_accel_redirect' nglims_config_file: 'tool-data/nglims.yaml' set_metadata_externally: 'True' static_cache_time: '360' static_dir: '/home/galaxy2ei/galaxy-dist/static/' static_enabled: 'True' static_favicon_dir: '/home/galaxy2ei/galaxy-dist/static/favicon.ico' static_images_dir: '/home/galaxy2ei/galaxy-dist/static/images' static_robots_txt: '/home/galaxy2ei/galaxy-dist/static/robots.txt' static_scripts_dir: '/home/galaxy2ei/galaxy-dist/static/scripts/' static_style_dir: '/home/galaxy2ei/galaxy-dist/static/june_2007_style/blue' tool_config_file: 'tool_conf.xml,shed_tool_conf.xml' tool_data_path: 'tool-data' tool_path: 'tools' upstream_gzip: 'True' use_interactive: 'False' use_nglims: 'False' user_library_import_dir: '/home/galaxy2ei/Downloads' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x8077450> paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a1a924b5a883eb84f93f9e2c6424cabed485f00c49dc51dd6632e1e679e865175'>, 'galaxysession=c6ca0ddb55be603a1a924b5a883eb84f93f9e2c6424cabed485f00c49dc51dd6632e1e679e865175') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x8077350> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x7f34ec7f5950> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True webob._parsed_query_vars: (MultiDict([]), '') wsgi process: 'Multithreaded' ------------------------------------------------------------

2 1

get wsgi headers
by Mutlu Dogruel 06 Sep '12

06 Sep '12

Hi folk, Do you know any method for reading (the extended) WSGI HTTP headers from a custom tool? If I put some debugging lines in $galaxy/lib/galaxy/web/framework/middleware/xforwardedhost.py, I can read the HTTP headers I want. But I need to access them under tools/my_script.py. Is there a way to reach the "webapp"s http headers from there? Thanks. Mutlu

2 3

using data_ref to refer to column names?
by Dan Tenenbaum 04 Sep '12

04 Sep '12

Hi, I have some code that makes a plot and creates a legend from a given column name in the user's input data, which may have arbitrary column names. I'd like to create a dropdown list that has all the column names in it, but I don't know the names beforehand; I won't know them until the user uploads their data file. So all I can do is have a freeform text box where I ask the user to type a column name, which seems error-prone and ugly. I notice that the XY plotting tool has something sort of like this, where it can tell how many columns are in your dataset and lets you choose them, but it calls the columns "c1", "c2", etc. My datasets have a header line with column names and I want to show the actual column name, not reference them by column number. Is this possible? Is there a solution to this? I have a number of use cases like this, so it would be great if there was. Thanks, Dan

2 3

Getting syntax error on Join + environment_setup_file not read
by Peter Schmitt 04 Sep '12

04 Sep '12

New galaxy install. Running the tutorial Galaxy 101 and trying to do the join. Get the following error in the stderr of the job: Traceback (most recent call last): File "/opt/galaxy/galaxy-dist/tools/new_operations/gops_join.py", line 20, in ? from galaxy.tools.util.galaxyops import * File "/opt/galaxy/galaxy-dist/lib/galaxy/tools/__init__.py", line 15, in ? from galaxy import util, jobs, model File "/opt/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 15, in ? from galaxy import util, model File "/opt/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ? import galaxy.datatypes.registry File "/opt/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 146 finally: ^ SyntaxError: invalid syntax Traceback (most recent call last): File "./scripts/set_metadata.py", line 24, in ? import galaxy.model.mapping #need to load this before we unpickle, in order to setup properties assigned by the mappers File "/opt/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ? import galaxy.datatypes.registry File "/opt/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 146 finally: ^ SyntaxError: invalid syntax Also I believe the line, environment_setup_file = scripts/galaxy.sh, is not being used any longer. When I put the line: export DRMAA_LIBRARY_PATH=/opt/drmaa/1.0.12/lib/libdrmaa.so.1.0.12, in there the server didn't see it. When I added that line to the top of run.sh, it does. -- * Pete Schmitt Technical Director: Discovery Cluster NH INBRE Grid Computational Genetics Lab Dartmouth College, HB 6223 L24 Berry/Baker Library Hanover, NH 03755 Phone: 603-646-8109 http://discovery.dartmouth.edu http://columbia.dartmouth.edu/grid http://www.epistasis.org *

2 1

Tool-integration: annovar
by Xinyi Chua 03 Sep '12

03 Sep '12

Hi all, I have a new project that requires setting up a local Galaxy instance for a client including integrating new tools and some reference data sources. I'm new to the Galaxy community so am also learning along the way. There's been a request to integrate the ANNOVAR tool for their system. I've searched the mailing-list archive and found this question was posed earlier in the year but there was no reply. I couldn't find any tool wrappers for it on Toolshed, but considering it is such a widely-used tool, I'm wondering whether anyone has any knowledge if this tool has been wrapped for Galaxy before? Many thanks, Xin-Yi _____________________________________ Xin-Yi Chua Senior Bioinformatician Queensland Facility for Advanced Bioinformatics Level 6, QBP University of Queensland, Q4072 T: +61 (0)7 3346 2125 M: +61 (0) 439 783 953 E: <mailto:xinyi.chua@qfab.org> xinyi.chua(a)qfab.org <http://www.qfab.org/> www.qfab.org

2 2

Python issue on cluster
by michael xu 31 Aug '12

31 Aug '12

Hi, Recently, I tried to run galaxy on our cluster and got the following error messages when I was trying to run samtools. Everything is running well when I run it on a local machine. Once I switch it to cluster, the conversion could not be successfully executed. I checked the version of python, they are all v2.6.6 on both head and nodes machines. Anybody can help me on this issue? Thanks a lot in advance! ================================================== Traceback (most recent call last): File "/home/gxu/galaxy_dist/tools/samtools/bam_to_sam.py", line 10, in <module> from galaxy import eggs File "/home/gxu/galaxy_dist/lib/galaxy/__init__.py", line 5, in <module> __import__( "pkg_resources" ).declare_namespace( __name__ ) File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 698, in <module> class Environment(object): File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 701, in Environment def __init__(self, search_path=None, platform=get_supported_platform(), python=PY_MAJOR): File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 96, in get_supported_platform plat = get_build_platform(); m = macosVersionString.match(plat) File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 221, in get_build_platform from distutils.util import get_platform ImportError: No module named distutils.util ==================================================

1 0

GTF, GFF and gff_filter_by_attribute
by Björn Grüning 31 Aug '12

31 Aug '12

Hi, i have found a small problem in gff_filter_by_attribute and not sure how to fix it properly. gff_filter_by_attribute.xml accepts gff data types but crashes if you run it with gff. Because metadata.attribute_types is not defined for gff and the dictionary is empty. For gtf files it works, because gtf has metadata.attribute_types defined. Should we extend the gff data type to also set metadata.attribute_types or is it for some reason better to create a gff<->gtf converter and chance gff_filter_by_attribute.xml to only accept gtf files? Thanks! Bjoern -- Björn Grüning Albert-Ludwigs-Universität Freiburg Institute of Pharmaceutical Sciences Pharmaceutical Bioinformatics Hermann-Herder-Strasse 9 D-79104 Freiburg i. Br. Tel.: +49 761 203-4872 Fax.: +49 761 203-97769 E-Mail: bjoern.gruening(a)pharmazie.uni-freiburg.de Web: http://www.pharmaceutical-bioinformatics.org/

2 1

Socket error starting galaxy
by Gaueko Erge 31 Aug '12

31 Aug '12

Hi, On a fresh galaxy installation the server start fine when it looks at port 8080, but as soon as I change to port 80 I get the following error I think that is due to permissions since I have my galaxy inside a non-root galaxy user directory Any ideas on the best way to solve this would be appreciated Thanks --G Starting server in PID 8799. Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run result = self.command() File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 303, in command serve() File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 287, in serve server(app) File "/usr/local/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 151, in server_wrapper **context.local_conf) File "/usr/local/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1314, in server_runner serve(wsgi_app, **kwargs) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1264, in serve threadpool_options=threadpool_options) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1114, in __init__ RequestHandlerClass, ssl_context) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1094, in __init__ RequestHandlerClass, ssl_context) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 328, in __init__ HTTPServer.__init__(self, server_address, RequestHandlerClass) File "/usr/lib/python2.7/SocketServer.py", line 408, in __init__ self.server_bind() File "/usr/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind SocketServer.TCPServer.server_bind(self) File "/usr/lib/python2.7/SocketServer.py", line 419, in server_bind self.socket.bind(self.server_address) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 13] Permission denied

3 2