November 2010 - galaxy-dev - lists.galaxyproject.org

Safe to have Apache serve datasets?
by Branden Timm 19 Nov '10

19 Nov '10

Hi, The notes for deploying Galaxy in production using Apache with mod_proxy advise serving static content from Apache. Is there a good reason not to do the same for <galaxy_home>/database/files? -- Branden Timm System Administrator Great Lakes Bioenergy Research Center btimm(a)glbrc.wisc.edu

2 1

Error: Unable to cleanup job
by Peter Cock 19 Nov '10

19 Nov '10

Hi all, In some workflow testing I was getting errors "Unable to cleanup job" which was due to the cleanup method trying to remove non-existent directories. The following patch appears to solve this particular error (although I am still looking into why my workflow was failing). Peter Example error (before the patch): galaxy.jobs ERROR 2010-11-19 13:58:04,806 Unable to cleanup job 150 Traceback (most recent call last): File "/home/pjcock/repositories/galaxy-central/lib/galaxy/jobs/__init__.py", line 614, in cleanup shutil.rmtree( self.working_directory ) File "/usr/local/lib/python2.6/shutil.py", line 204, in rmtree onerror(os.listdir, path, sys.exc_info()) File "/usr/local/lib/python2.6/shutil.py", line 202, in rmtree names = os.listdir(path) OSError: [Errno 2] No such file or directory: './database/job_working_directory/150' Proposed patch: diff -r 730b89c4da26 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Wed Nov 17 18:26:33 2010 +0000 +++ b/lib/galaxy/jobs/__init__.py Fri Nov 19 14:13:45 2010 +0000 @@ -610,7 +610,7 @@ try: for fname in self.extra_filenames: os.remove( fname ) - if self.working_directory is not None: + if self.working_directory and os.path.isdir( self.working_directory ): shutil.rmtree( self.working_directory ) if self.app.config.set_metadata_externally: self.external_output_metadata.cleanup_external_metadata( self.sa_session )

3 4

Reports error
by SHAUN WEBB 18 Nov '10

18 Nov '10

Hi, I am having a few issues when developing reports. If I click on jobs per month I get an error OperationalError: (OperationalError) (1305, 'FUNCTION galaxy_test.date_trunc does not exist') Full traceback is below. There are similar errors when I click on links for a specific user or tool. Any idea what the issue is. Thanks in advance for any help. Shaun Webb URL: http://bifx3.bio.ed.ac.uk:9001/jobs/per_month_all File '/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy_dist/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/home/galaxy/galaxy_dist/lib/galaxy/webapps/reports/controllers/jobs.py', line 251 in per_month_all for row in q.execute(): File '/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/sql/expression.py', line 1121 in execute return e._execute_clauseelement(self, multiparams, params) File '/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py', line 1209 in _execute_clauseelement return connection._execute_clauseelement(elem, multiparams, params) File '/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py', line 874 in _execute_clauseelement return self.__execute_context(context) File '/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py', line 896 in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File '/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py', line 950 in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File '/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py', line 931 in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) OperationalError: (OperationalError) (1305, 'FUNCTION galaxy_test.date_trunc does not exist') u'SELECT date_trunc(%s, date(job.create_time)) AS date, sum(CASE WHEN (galaxy_user.email = %s) THEN %s ELSE %s END) AS monitor_jobs, count(job.id) AS total_jobs \nFROM job LEFT OUTER JOIN galaxy_user ON galaxy_user.id = job.user_id GROUP BY date_trunc(%s, date(job.create_time)) ORDER BY date DESC' ['month', 'monitor(a)bx.psu.edu', 1, 0, 'month'] -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

3 3

missing file? (hapmapmart.xml)
by Erick Antezana 18 Nov '10

18 Nov '10

Hi, I just updated my local instance and it seems that the hapmapmart.xml is missing in the repository? galaxy.tools ERROR 2010-11-18 15:55:48,570 error reading tool from path: data_source/hapmapmart.xml Traceback (most recent call last): File "/tools/shared/bioinfo/app/galaxy/dev/lib/galaxy/tools/__init__.py", line 77, in load_tool tool = self.load_tool( os.path.join( self.tool_root_dir, path ) ) File "/tools/shared/bioinfo/app/galaxy/dev/lib/galaxy/tools/__init__.py", line 131, in load_tool tree = util.parse_xml( config_file ) File "/tools/shared/bioinfo/app/galaxy/dev/lib/galaxy/util/__init__.py", line 102, in parse_xml tree = ElementTree.parse(fname) File "/tools/shared/bioinfo/app/galaxy/dev/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/tools/shared/bioinfo/app/galaxy/dev/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 576, in parse source = open(source, "rb") IOError: [Errno 2] No such file or directory: './tools/data_source/hapmapmart.xml' thanks, Erick

2 1

Problem with Mako and Beaker egg...
by Marc Bras 17 Nov '10

17 Nov '10

Hi, I'm trying to install Galaxy in a production environment. Since I set Postgres DataBase, I've got this warning when I launch any tool: /WARNING:galaxy.eggs:Warning: Beaker (a dependant egg of Mako) cannot be fetched/ And this warning is print in STDERR... so the tool failed... I don't know why it can't fetch Beaker egg... I aldready try to remove : - $GALAXY_ROOT/eggs/Beaker-1.4-py2.5.egg - $GALAXY_ROOT/eggs/Mako-0.2.5-py2.4.egg - $GALAXY_ROOT/eggs/Mako-0.2.5-py2.5.egg but I've got still the problem. I'm open to any solution !! Many thanks for your help, Marc

2 1

galaxy behind proxy/apache2
by Bossers, Alex 17 Nov '10

17 Nov '10

Hello all, I am trying to do the production server setup of galaxy and was finally succesfull in using mysql instead of sqlight. I am facing difficulties in configuring the proxy/apache part though. I followed the wiki but no luck. I do not have any other sites on that machine but I was hoping to either: use www.domain.nl/galaxy<http://www.domain.nl/galaxy> or www.domain.nl:8080<http://www.domain.nl:8080/> to point to the galaxy using apache and proxy.... that would leave the possibility to have www.domain.nl<http://www.domain.nl/> and/or www.domain.nl/other_sites<http://www.domain.nl/other_sites> to still be used. My www is at default /var/www Galaxy is in /opt/galaxy mod-rewrite is installed and enabled even as the proxy parts made the suggested modifictions to universe_wsgi.ini (unsure on the server=0.0.0.0 or 127.0.0.1 part and the part dealing with the setting of the static part... should that be True or False. I would say False. Just to be sure: galaxy must be started using its own galaxy system user right? Apache2 is working fine (on ubuntu maverick server) and the www.domain.nl<http://www.domain.nl/> site incl PHP is working. Also www.domain.nl/phpmyadmin<http://www.domain.nl/phpmyadmin> is fully functional.... I'm puzzled... Any help/direction is appreciated. Or a working example of the universe_wsgi.ini and other config files would be very helpful! Alex

2 1

Send file to 3rd party site
by Chris Zaleski 17 Nov '10

17 Nov '10

Greetings, I have a Galaxy tool we use to send files to another website - I used "data_destination/epigraph.xml" as a template. In this case, the other website is our own web-based database system. Everything is on our own internal network. This was working fine until I recently did a major Galaxy update. We were running a large production pipeline, and intentionally did not upgrade Galaxy for over a year. The problem seems to be when the outside system uses the "DATA_URL" parameter to get the file from Galaxy. Instead of getting the data file, Galaxy returns a HTML file (which seems to be the index page of Galaxy itself). I did a couple tests, and realized that this behavior changes depending on the client using the "DATA_URL". It works in a browser, but fails in other cases: 1) If I take the "DATA_URL", and paste it directly into a browser address bar (Firefox), it works properly. I get the data file I'm supposed to get. 2) In the case of my outside database system, it uses PHP's "copy" function. What gets copied is the incorrect HTML file. 3) I tried using "wget" with the "DATA_URL", and it also gets the same incorrect HTML file. You can see in the wget output that it receives a redirect - the redirected URL turns out to the be galaxy homepage. So maybe PHP's copy function and wget are both getting redirected, but Firefox is not...? I'll attach the wget output, as well as the incorrect HTML file I'm getting from Galaxy. Any help is greatly appreciated. Thanks Chris Zaleski

3 3

Upload to Galaxy via FTP
by Nate Coraor 17 Nov '10

17 Nov '10

Reposted to the Development list for the benefit of those not on the User list. Hello, With the continued advancement of sequencing technology, we've seen the size of files uploaded to Galaxy grow quite large. Although our current file upload methods have worked fine for years, they are not well suited for the extremely large files in common use today. Uploading directly from the browser can be unreliable and browsers don't provide feedback on upload progress and state like they do for downloads. Because of this, we have implemented file uploads to Galaxy via FTP. FTP will allow you to monitor as mentioned above, as well as resume interrupted transfers. To get started using FTP, you'll need to have registered a regular Galaxy account on our public server at: http://main.g2.bx.psu.edu/ Once registered, you can initiate an FTP connection in your preferred FTP client to the same host (main.g2.bx.psu.edu) using your registered email address and password for the login details. Files uploaded to this server won't automatically be imported to Galaxy - rather, you will be presented with a list of the contents of your FTP directory on the standard "Upload File" tool interface. Files not imported within 3 days will be cleaned up from the FTP site. We hope that this service solves a crucial problem in data transfer speed and reliability. Please report any problems or suggestions with the service to us at: galaxy-bugs(a)bx.psu.edu Please note that it may not always be practical to use the public Galaxy servers. If you're routinely working with very large data and having to wait for it to upload, a local Galaxy server could be a more practical solution. Instructions on installing your own server can be found at: http://getgalaxy.org/ http://usegalaxy.org/production In addition, with the recent announcement of Amazon's parallel upload capability for the Simple Storage Service (S3), we are investigating incorporating easy access to S3 buckets for Galaxy instances on the Amazon Elastic Compute Cloud (EC2). But you don't need to wait for the pretty interface, you can already access contents of S3 buckets by pasting links to their contents in the "URL/Text:" field of the "Upload File" tool. For an example of how to do this, see the "Watch how the complete analysis can be performed on the Amazon Cloud" screencast at: http://usegalaxy.org/heteroplasmy Thanks, --nate Galaxy Team

2 3

rpy for Galaxy?
by David Hoover 17 Nov '10

17 Nov '10

I am having great difficulty installing rpy (v1.0.3) for our local Galaxy installation. Is rpy actually supported for Galaxy? Should I even bother trying to install rpy? David Hoover Helix Systems Staff http://helix.nih.gov

4 7

iframe target links with external projector
by Brad Chapman 17 Nov '10

17 Nov '10

Hi all; I'm giving a live demo of Galaxy in a few hours and was prepping with my laptop attached to an external projector. When connected to the projector, the target links in the sidebar load in the full window instead of targeting the galaxy_main iframe. This gets rid of the nice 3 panel layout. On the same machine the target links works as expected if my laptop is not connected to the external projector. Has anyone here seen the same issue when giving a Galaxy demo? Any configuration tips or tricks for Firefox (3.6) that will help? Thanks much, Brad

2 2