- galaxy-dev - lists.galaxyproject.org

LSF DRMAA/FedStage config-make errors
by Douglas W Cloud 30 Aug '12

30 Aug '12

Hello all, I am trying to get a Galaxy server configured to submit jobs to an LSF cluster running version 7 update 2, generally CentOS/Redhat 5 systems. I have the Galaxy host as a client in LSF, but when I tried to configure/make/install the FedStage drmaa code, it errors out in the make stage (see stdout below). Don't know what to fix, if it's possible. If anyone has experience configuring/troubleshooting this code to get Galaxy conversant with LSF, I'd appreciate any feedback. Thanks! Regards, Doug _________ util.c: In function 'lsfdrmaa_map_lsberrno': util.c:730: error: 'LSBE_ADRSV_ID_UNIQUE' undeclared (first use in this function) util.c:730: error: (Each undeclared identifier is reported only once util.c:730: error: for each function it appears in.) util.c:731: error: 'LSBE_BAD_RSVNAME' undeclared (first use in this function) util.c:732: error: 'LSBE_BAD_CHKPNTDIR' undeclared (first use in this function) util.c:734: error: 'LSBE_JOB_REQUEUE_BADEXCLUDE' undeclared (first use in this function) util.c:737: error: 'LSBE_ADVRSV_ACTIVESTART' undeclared (first use in this function) util.c:739: error: 'LSBE_ADRSV_ID_USED' undeclared (first use in this function) util.c:740: error: 'LSBE_ADRSV_PREVDISABLED' undeclared (first use in this function) util.c:742: error: 'LSBE_ADRSV_DISABLECURR' undeclared (first use in this function) util.c:744: error: 'LSBE_ADRSV_NOT_RSV_HOST' undeclared (first use in this function) util.c:749: error: 'LSBE_ADRSV_MOD_REMOTE' undeclared (first use in this function) util.c:751: error: 'LSBE_JOB_RUSAGE_EXCEED_LIMIT' undeclared (first use in this function) util.c:752: error: 'LSBE_APP_RUSAGE_EXCEED_LIMIT' undeclared (first use in this function) util.c:755: error: 'LSBE_ADRSV_DISABLE_DATE' undeclared (first use in this function) util.c:759: error: 'LSBE_ADRSV_DETACH_MIX' undeclared (first use in this function) util.c:761: error: 'LSBE_ADRSV_DETACH_ACTIVE' undeclared (first use in this function) util.c:765: error: 'LSBE_MISSING_START_END_TIME' undeclared (first use in this function) make[3]: *** [libdrmaa_la-util.lo] Error 1 make[3]: Leaving directory `/root/installers/lsf_drmaa-1.0.4/lsf_drmaa' make[2]: *** [all] Error 2 make[2]: Leaving directory `/root/installers/lsf_drmaa-1.0.4/lsf_drmaa' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/installers/lsf_drmaa-1.0.4' (Same output with version 1.0.3.) This communication is for use by the intended recipient and contains information that may be Privileged, confidential or copyrighted under applicable law. If you are not the intended recipient, you are hereby formally notified that any use, copying or distribution of this e-mail, in whole or in part, is strictly prohibited. Please notify the sender by return e-mail and delete this e-mail from your system. Unless explicitly and conspicuously designated as "E-Contract Intended", this e-mail does not constitute a contract offer, a contract amendment, or an acceptance of a contract offer. This e-mail does not constitute a consent to the use of sender's contact information for direct marketing purposes or for transfers of data to third parties. Francais Deutsch Italiano Espanol Portugues Japanese Chinese Korean http://www.DuPont.com/corp/email_disclaimer.html

2 1

run_reports.sh not starting
by Joachim Jacob 30 Aug '12

30 Aug '12

Hi all, When running galaxy-dist/run_reports.sh I get errors. It has been awhile since I run it, and it worked perfect then however. python path is: /home/galaxy/galaxy-dist/eggs/pycrypto-2.5-py2.6-linux-x86_64-ucs4.egg, /home/galaxy/galaxy-dist/eggs/python_lzo-1.08_2.03_static-py2.6-linux-x86_64-ucs4.egg, /home/galaxy/galaxy-dist/eggs/bx_python-0.7.1_7b95ff194725-py2.6-linux-x86_64-ucs4.egg, /home/galaxy/galaxy-dist/eggs/amqplib-0.6.1-py2.6.egg, /home/galaxy/galaxy-dist/eggs/pexpect-2.4-py2.6.egg, /home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg, /home/galaxy/galaxy-dist/eggs/Babel-0.9.4-py2.6.egg, /home/galaxy/galaxy-dist/eggs/MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg, /home/galaxy/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg, /home/galaxy/galaxy-dist/eggs/WebHelpers-0.2-py2.6.egg, /home/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg, /home/galaxy/galaxy-dist/eggs/wchartype-0.1-py2.6.egg, /home/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.6.egg, /home/galaxy/galaxy-dist/eggs/docutils-0.7-py2.6.egg, /home/galaxy/galaxy-dist/eggs/WebOb-0.8.5-py2.6.egg, /home/galaxy/galaxy-dist/eggs/Routes-1.12.3-py2.6.egg, /home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg, /home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg, /home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg, /home/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg, /home/galaxy/galaxy-dist/lib, /usr/local/lib64/python2.6/site-packages, /opt/MACS/1.4.2/lib/python2.6/site-packages, /opt/MACS/1.3.7/lib/python2.6/site-packages, /usr/lib64/python26.zip, /usr/lib64/python2.6, /usr/lib64/python2.6/plat-linux2, /usr/lib64/python2.6/lib-tk, /usr/lib64/python2.6/lib-old, /usr/lib64/python2.6/lib-dynload, /usr/lib64/python2.6/site-packages/PIL, /usr/lib64/python2.6/site-packages/gtk-2.0, /usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info, /usr/lib/python2.6/site-packages, /usr/lib64/python2.6/site-packages galaxy.model.custom_types DEBUG 2012-08-30 09:06:58,051 psycopg2 egg successfully loaded for postgres dialect Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/command.py", line 218, in run result = self.command() File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.6.egg/paste/script/serve.py", line 313, in loadapp **kw) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 225, in loadobj return context.create() File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 625, in create return self.object_type.invoke(self) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/loadwsgi.py", line 110, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/reports/buildapp.py", line 52, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/reports/app.py", line 22, in __init__ create_tables = True ) File "/home/galaxy/galaxy-dist/lib/galaxy/model/mapping.py", line 1892, in init metadata.create_all() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/schema.py", line 1796, in create_all bind.create(self, checkfirst=checkfirst, tables=tables) File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1129, in create self._run_visitor(self.dialect.schemagenerator, entity, connection=connection, **kwargs) File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1154, in _run_visitor conn = self.contextual_connect(close_with_result=False) File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1229, in contextual_connect return self.Connection(self, self.pool.connect(), close_with_result=close_with_result, **kwargs) File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 142, in connect return _ConnectionFairy(self).checkout() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 304, in __init__ rec = self._connection_record = pool.get() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 161, in get return self.do_get() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 639, in do_get con = self.create_connection() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 122, in create_connection return _ConnectionRecord(self) File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 198, in __init__ self.connection = self.__connect() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/pool.py", line 261, in __connect connection = self.__pool._creator() File "/home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/strategies.py", line 80, in connect raise exc.DBAPIError.instance(None, None, e) sqlalchemy.exc.OperationalError: (OperationalError) could not connect to server: Connection refused Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"? None None Removing PID file reports_webapp.pid The postgres database is running, user credentials are correct. I tried also running as root, as opposed to user galaxy, but no avail. Thanks for your consideration, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

2 2

September 2012 Galaxy Update
by Dave Clements 29 Aug '12

29 Aug '12

1 0

Modules
by Fabien Mareuil 29 Aug '12

29 Aug '12

Dear Colleagues, Many teams use module, a user interface for the Modules package. The Modules package enable to manage easily several versions of a program with a dynamic modification of the user's environment via modulefiles. I would like to know if it is planned to integrate the ability to load modules via galaxy? For example, we could imagine a configuration file that would link the name and version of a tool with the module name and allowing to load the module before running the tool locally or on a cluster. Thank you for your answer, Best regards, Fabien Mareuil

2 1

Error when downloading files through API
by Richard Park 29 Aug '12

29 Aug '12

hey galaxy, Has anyone seen this error when downloading files through the API? "TypeError: display_data() got an unexpected keyword argument 'key'" It works for the majority of my files, but fails on "wig" files. This problem recently occurred when I updated my galaxy code about 2-3 weeks ago. The weird thing is that I can see the file if I remove the API key from the url. Any ideas would be greatly appreciated. My url string: http://127.0.0.1:8080/datasets/86889e19bba829b0?key=35a24ae2643785ff3d046c9… ### Error Traceback ### URL: http://127.0.0.1:8080/datasets/86889e19bba829b0?key=35a24ae2643785ff3d046c9… File '/Users/richardpark/Projects/galaxy-central/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/Users/richardpark/Projects/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/Users/richardpark/Projects/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/Users/richardpark/Projects/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/Users/richardpark/Projects/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/Users/richardpark/Projects/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/Users/richardpark/Projects/galaxy-central/lib/galaxy/web/controllers/dataset.py', line 288 in display return data.datatype.display_data(trans, data, preview, filename, to_ext, chunk, **kwd) TypeError: display_data() got an unexpected keyword argument 'key' thanks, Richard Park

1 0

Blastxml error on Galaxy Cloudman
by Mohammad Heydarian 29 Aug '12

29 Aug '12

Hello, I recently updated my instance of Galaxy Cloudman (using Amazon AWS) and I have since not been able to run any jobs. I get the following error: WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' Any help would greatly be appreciated. Cheers, Mo Heydarian PhD candidate The Johns Hopkins School of Medicine Department of Biological Chemistry 725 Wolfe Street 414 Hunterian Baltimore, MD 21205

2 1

Tool test question
by christopher fletez-brant 29 Aug '12

29 Aug '12

Dear Dev List, I am currently writing tests for a tool I am developing and am coming up short in understanding why it's failing. Using the test data, I can generate output identical to the output provided for testing every time I run the tool from my local Galaxy server. However, when I run the tool test through run_functional_tests.sh, it always fails, and apparently fails on every line. Because I consistently see the same output from the Galaxy server-based tool, I'm sure I'm missing something simple and want to look at the test-generated output. Is there a way I can set run_functional_tests to print the generated tool output file in its entirety? Thank you, Kipper Fletez-Brant

2 6

xsendfile permissions
by Julian Pradinuk 29 Aug '12

29 Aug '12

Hi all, I'm wondering if someone who has downloads working through apache (using xsendfile) has encountered this error when downloading HTML files: (13)Permission denied: xsendfile: cannot open file: /galaxy_data/production/tmp/tmpN0ydYt/library_download.zip The output from ls -lZ /galaxy_data/production/tmp is: ... drwx------. galaxy galaxy unconfined_u:object_r:httpd_sys_content_t:s0 tmpN0ydYt ... The output from ls-lZ /galaxy_data/production/tmp/tmpN0ydYt/ is: -rw-rw-r--. galaxy galaxy unconfined_u:object_r:httpd_sys_content_t:s0 library_download.zip It's not a SElinux issue (if I sudo setenforce 0, the problem persists).. It seems it's a regular permissions issue (user apache is running httpd and does not permission to enter the tmpN0ydYt). Since galaxy creates these files with these permissions I don't know how to fix this issue. I'm also wondering why the HTML datasets are in the tmp folder not the files folder? Here's my apache config: <VirtualHost *:80> ServerName ________________ RewriteEngine on #RewriteOptions Inherit #RewriteLog "/etc/httpd/logs/rewrite_log" #RewriteLogLevel 5 RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /usr/local/galaxy/production/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /usr/local/galaxy/production/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /usr/local/galaxy/production/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /usr/local/galaxy/production/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /usr/local/galaxy/production/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] <Location "/galaxy"> order allow,deny allow from all # Compress all uncompressed content SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary # Send files through Apache XSendFile on XSendFilePath /galaxy_data </Location> <Location "/galaxy/static"> # Allow browsers to cache everything from /static for 6 hours ExpiresActive On ExpiresDefault "access plus 6 hours" </Location> </VirtualHost> Is there something wrong with it? Or have I missed something in universe.wsgi? Cheers, Julian

2 1

Issues with database / metadata after upgrade
by Iry Witham 29 Aug '12

29 Aug '12

Hi Team, I had performed an upgrade of my Galaxy server and had experienced an issue with some of the database / metadata files have been corrupted. I have found instances where the dataset exists, but cannot be detected or read by galaxy when attempting to analysis data from a history. In some cases I have been able to locate another copy and replace the damaged file. What might have been the cause and is there a way to resolve it? Is this a possible Postgres issue or since the metadata is contained in the /database/files/ directory an artifact of the upgrade? Thanks, Iry The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

1 0

So I think I fixed a bug.
by James 29 Aug '12

29 Aug '12

Hi All, Today I found a bug relating to the file permissions in composite datatypes. When the extrafiles directory was created and the server was running through apache, the permission did not allow a galaxy user to download by following the HTML link. Found it was related to meta data files not using the os.chmod function to set the permissions of the user data in upload.py. Added that line of code and it works and the file permissions match that of other files in the database. Somehelp with submitting a patch would be welcome ( although one line patch it would be helpful to me.) Cheers James Boocock

5 7