January 2012 - galaxy-dev - lists.galaxyproject.org

PBS job failed: Unknown error: -10
by Ryan Golhar 05 Jan '12

05 Jan '12

I have three bam files that I'm trying to merge. The job runs for a bit then errors saying there is a problem. When I look at runner0.log I see: galaxy.jobs.runners.pbs DEBUG 2012-01-05 00:12:50,193 (19) submitting file /home/galaxy/galaxy-dist-9/database/pbs/19.sh galaxy.jobs.runners.pbs DEBUG 2012-01-05 00:12:50,193 (19) command is: java -Xmx2G -jar /home/galaxy/galaxy-dist-9/tool-data/shared/jars/picard/MergeSamFiles.jar MSD=true VALIDATION_STRINGENCY=LENIENT O=/home/galaxy/galaxy-dist-9/database/files/000/dataset_96.dat I=/mnt/isilon/cag/ngs/hiseq/golharr/FALK05/solid5500_2_Falk05_JMF3_5_20111024_1_04-1-Idx_1-1.bam I=/mnt/isilon/cag/ngs/hiseq/golharr/FALK05/solid5500_2_Falk05_JMF3_5_20111024_1_05-2-Idx_1-1.bam I=/mnt/isilon/cag/ngs/hiseq/golharr/FALK05/solid5500_2_Falk05_JMF3_5_20111024_1_06-3-Idx_1-1.bam 2> /home/galaxy/galaxy-dist-9/database/files/000/dataset_97.dat || echo "Error running Picard MergeSamFiles" >&2; cd /home/galaxy/galaxy-dist-9; /home/galaxy/galaxy-dist-9/set_metadata.sh ./database/files ./database/tmp . datatypes_conf.xml ./database/job_working_directory/19/galaxy.json database/tmp/metadata_in_HistoryDatasetAssociation_16_MuM7TP,database/tmp/metadata_kwds_HistoryDatasetAssociation_16_7w3ZvM,database/tmp/metadata_out_HistoryDatasetAssociation_16_m9A_X4,database/tmp/metadata_results_HistoryDatasetAssociation_16_cRV8Uj,,database/tmp/metadata_override_HistoryDatasetAssociation_16_5n8tqL database/tmp/metadata_in_HistoryDatasetAssociation_15_SUEdLf,database/tmp/metadata_kwds_HistoryDatasetAssociation_15_nt8MTH,database/tmp/metadata_out_HistoryDatasetAssociation_15_EEIJ_o,database/tmp/metadata_results_HistoryDatasetAssociation_15_7PhiXX,,database/tmp/metadata_override_HistoryDatasetAssociation_15_e_bdeW galaxy.jobs.runners.pbs DEBUG 2012-01-05 00:12:50,206 (19) queued in default queue as 29.localhost.localdomain galaxy.jobs.runners.pbs DEBUG 2012-01-05 00:12:50,253 (19/29.localhost.localdomain) PBS job state changed from N to R galaxy.jobs.runners.pbs DEBUG 2012-01-05 01:13:28,111 (19/29.localhost.localdomain) PBS job state changed from R to C galaxy.jobs.runners.pbs ERROR 2012-01-05 01:13:28,111 (19/29.localhost.localdomain) PBS job failed: Unknown error: -10 galaxy.datatypes.metadata DEBUG 2012-01-05 01:13:28,272 Cleaning up external metadata files galaxy.datatypes.metadata DEBUG 2012-01-05 01:13:28,286 Failed to cleanup MetadataTempFile temp files from database/tmp/metadata_out_HistoryDatasetAssociation_15_EEIJ_o: No JSON object could be decoded: line 1 column 0 (char 0) galaxy.datatypes.metadata DEBUG 2012-01-05 01:13:28,286 Failed to cleanup MetadataTempFile temp files from database/tmp/metadata_out_HistoryDatasetAssociation_16_m9A_X4: No JSON object could be decoded: line 1 column 0 (char 0) It looks like the job failed with "PBS job failed: Unknown error: -10". Is there a way to set up Galaxy to keep job submission files? I thought there was but don't see the option in universe_wsgi.ini for this.

4 6

Why can't my script read the files in job_working_directory?
by Kipper Fletez-Brant 05 Jan '12

05 Jan '12

Good afternoon Galaxy list, I've got a Python script which calls an R script. When I run a job using Galaxy, I get the following output from R: Error in file(file, "r", encoding = encoding) : cannot open the connection In addition: Warning message: In file(file, "r", encoding = encoding) : cannot open file 'rocprc.R': No such file or directory The difficult thing here is that both scripts are in the same directory. I tried using os.path.realpath(PYTHON SCRIPT) to get the name of the current directory so that I could force R to operate in same; I still get this error. Telling Python to print the results from os.path.realpath(PYTHON SCRIPT) gave me "~/galaxy-dist/database/job_working_directory". Also, the Python script works perfectly when called from the command line, so I know this is a Galaxy issue. Please help, Kipper Fletez-Brant

1 0

Re: [galaxy-dev] [galaxy-user] (OperationalError) unable to open database file
by Nate Coraor 05 Jan '12

05 Jan '12

On Dec 29, 2011, at 12:15 AM, Cai Shaojiang wrote: > Dear friends, > > We are trying to install galaxy on the server (ubuntu 11), just by > following the steps on the page "Get Galaxy: Galaxy Download and > Installation" of galaxy wiki. But when we start running it, it shows > the following error message. > > OperationalError: (OperationalError) unable to open database file > u'INSERT INTO galaxy_session (create_time, update_time, user_id, > remote_host, remote_addr, referer, current_history_id, session_key, > is_valid, prev_session_id, disk_usage) VALUES (?, ?, ?, ?, ?, ?, ?, ?, > ?, ?, ?)' ['2011-12-29 04:52:46.575702', '2011-12-29 04:52:46.575719', > None, '10.50.70.45', '10.50.70.45', None, None, > '5a38b2a7e6d77a7145726cb0881eadf6', 1, None, None] > > It seems something wrong with the write permission. Could you give any > hint where the problem could be? Thanks. Hi Cai, I've moved this over to the galaxy-dev list since it pertains to a local installation. Please make sure that the galaxy-dist/database/ directory is writable by the user running the Galaxy server. Also, if you plan to use this server for anything other than single-user development, I would suggest switching to a PostgreSQL server. This is trivial on Ubuntu (apt-get install postgresql, createuser/createdb, then edit universe_wsgi.ini as described at http://usegalaxy.org/production ). --ndate > > Best regards. > > Yours: Cai > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

3 4

Empty files when importing with no copy from NFS source
by Carlos Borroto 05 Jan '12

05 Jan '12

Hi, While trying to import some fastq files to a data library, using "Upload directory of files" and "Link to files without copying into Galaxy", I end with empty datasets. My data files are accessed through NFS and I'm using the recommended option -noac: 10.2.90.89:/projects on /local/projects type nfs (rw,noac,sloppy,addr=10.2.90.89) To import the files, I created links to them in the directory configured in universe_wsgi.ini. If I try to import the same files but with the option "Copy files into Galaxy", the files are correctly imported. Any advice what to look for? Thanks, Carlos

2 2

Re: [galaxy-dev] [galaxy-user] Make Galaxy continue running when i close the browser
by Jeremy Goecks 05 Jan '12

05 Jan '12

Efthymois, You'll want to run Galaxy as a daemon process. Run % sh run.sh --help to get more information on running Galaxy as a daemon. Also, please direct questions about running/configuring a local Galaxy instance to galaxy-dev (cc'd) rather than galaxy-user, which is for tool and analysis questions. Best, J. On Jan 5, 2012, at 8:06 AM, Makis Ladoukakis wrote: > Dear Galaxy users, > > I have installed a local Galaxy instance on a server and I use it to run certain genomic assembly workflows. Nevertheless with larger datasets completion may take up to one day. How can i make Galaxy to continue the operation even when i close the browser? Is that possible on a local instance or on the main server? > > Thank you, > Efthymios Ladoukakis > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

1 0

Galaxy Proxy by an External Apache Server
by Derrick LIN 05 Jan '12

05 Jan '12

Hi guys, I am trying to setup a new galaxy for my institute. The galaxy itself runs fine in our private network. Now I am trying to configure the proxy so that it can be accessed publicly. The official guide here http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy It seems to assume both galaxy and Apache are in the same server, but this is not the case for us. We have had a dedicated Apache server is running for all public sites. I still tried to follow the recommended rewrite rules, modified the path & URL. We installed the galaxy in a NFS location which can be accessed from Apache server. <IfModule mod_rewrite.c> RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /NFS/PATH/TO/galaxy_dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /NFS/PATH/TO/galaxy_dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /NFS/PATH/TO/galaxy_dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /NFS/PATH/TO/galaxy_dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /NFS/PATH/TO/galaxy_dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://galaxy.privatenet.org:8080$1 [P] </IfModule> When I tested from http://www.publicnet.org/galaxy, it gives 404 error. The log shows that error caused by /var/www/galaxy not found. Then I created a symlink /var/www/galaxy links to /NFS/PATH/TO/galaxy_dist this time, the URL shows the contents of the entire galaxy_dist directory. Will be very appreciated if anyone can point out which part went wrong. Kind regards, Derrick

2 1

Galaxy Server on Bio-Linux
by Bicak, Mesude 05 Jan '12

05 Jan '12

Dear Galaxy Developers, We work in Professor Dawn Field's group (Molecular Evolution and Bioinformatics Research Group) at the NERC Environmental Bioinformatics Centre (NEBC) of the Centre for Ecology and Hydrology (CEH) Research Institute based in Oxford. We develop and distribute Bio-Linux (http://nebc.nerc.ac.uk/tools/bio-linux) which is a customised Ubuntu distribution that comes with 500+ bioinformatics packages. Within our research group we also provide bioinformatics analysis for NERC-funded researchers, and recently started looking into Galaxy as well. It didn't take us long to discover its power and we would like to enable Bio-Linux users to install, run and maintain the Galaxy server with minimal effort, also with the aim to spread the word on Galaxy in Europe! Recently we took on a project with Dr. Casey Bergman from University of Manchester as the Principal Investigator, to package all the necessary Galaxy dependencies for Ubuntu/Bio-Linux. As many pre-requisities are already included with Bio-Linux, we are already some way down this path. New packages that we create will appear in a Launchpad PPA (https://launchpad.net/~nebc/+archive/galaxy) We will be happy to hear any comments regarding these efforts and we hope that this will be a useful resource for all Galaxy users. Once the initial packaging is done, we hope to collaborate with Galaxy team in maintaining and improving this resource. Best wishes, Tim, Soon and Mesude -- Dr. Mesude Bicak <mbicak(a)ceh.ac.uk> Bioinformatician & Bio-Linux Developer NERC Biomolecular Analysis Facility (NBAF) <http://nbaf.nerc.ac.uk> NERC Environmental Bioinformatics Centre (NEBC) <http://nebc.nerc.ac.uk> Molecular Evolution and Bioinformatics Group Centre for Ecology and Hydrology <http://ceh.ac.uk> Maclean Building, Benson Lane Crowmarsh Gifford Wallingford, Oxfordshire OX10 8BB Office Tel: +44 1491 69 2705-- This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.

4 4

load balancing and proxy settings with Apache
by Ryan 04 Jan '12

04 Jan '12

I'm at the final stage of deploying our Galaxy instance. I'm implementing the proxy server and load balancing with 1 job runner, and 5 web runners using Apache. I'm following the guide on: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Web%20Application%20Scal… All the Galaxy universe_wsgi.[webapp|runner].ini changes are fine. When editing the Apache configuration, I discovered a slight problem. First off, instead of modifying /etc/httpd/conf/httpd.conf, I created /etc/httpd/conf.d/galaxy.conf. The /etc/httpd/conf/httpd.conf reads /etc/httpd/conf/*.conf for additional settings. I thought it best to put changes here instead of polluting httpd.conf. One of the last changes in the docs is to Apache's rewriting rules is to change RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] to RewriteRule ^(.*) balancer://galaxy$1 [P] The problem with this however is that "http://<machine>/galaxy" not longer works. Only "http://<machine>/galaxy/" (note the trailing backslash). Without the trailing backslash, HTTP requests never get past Apache's rewrite rules. I *think* the correct change should be: RewriteRule ^/galaxy(.*) balancer://galaxy$1 [P] This seems to work for me (so far) and allows initial requests with and without the trailing backslash. Can I recommend this as a change to the docs? Also, can I recommend documenting a separate galaxy.conf for Apache? I'd be happy to provide mine as a model, if you'd like. Ryan

1 0

Best practices with data on clusters
by Cittaro Davide 04 Jan '12

04 Jan '12

Hi developers, I have a question that may be an OT, but since galaxy can work in a clustered environment withh queueing system, I'll try to ask here. Is there anibody here who copies data in a local temporary directory before performing any analysis step and copy it back into the "final results"? Thanks d Sent from my iPad

2 3

Re: [galaxy-dev] Problem "password authentication failed for user "postgres" "
by Nate Coraor 04 Jan '12

04 Jan '12

On Jan 3, 2012, at 8:35 PM, Huayan Gao wrote: > Thanks Nate! > > I also want to do a mirror site of Galaxy Server in asia, but found the instruction is not so clear/detailed. Do we have a detailed one? Hi Huayan, Please keep replies on the mailing list for the benefit and collaboration of the entire Galaxy community. Have you seen the documentation at: http://usegalaxy.org/production If so, and this is unclear, can you provide specifics about what you need to do that is not covered by the documentation. Thanks, --nate > > Best, > Huayan > > > On 4 Jan, 2012, at 4:16 AM, Nate Coraor wrote: > >> On Dec 8, 2011, at 10:12 PM, Huayan Gao wrote: >> >>> Dear all, >>> >>> I am trying to install Galaxy in my Mac. And got the following error: password authentication failed for user "postgres". >>> >>> I have installed postgres using dmg file and created a database called galaxydb under /Library/PostGresql/9.1 folder. I am able to run psql command. I also created postgres user for it. I modified the universe_wsgi.ini file for the database connection. Could you help me check what I should do next to fix the problem? >> >> Hi Huayan, >> >> Sorry for the delay in response. It probably isn't a good idea to use the 'postgres' superuser to run Galaxy. I'd suggest creating a new Postgres user (role) for Galaxy. The simplest thing to do if you're connecting with a unix socket (i.e. you're not specifying a hostname or IP address in database_connection) is to make the Postgres username match the username of the user running the Galaxy process, and to not require a password for that user. You may need the following in pg_hba.conf, if it's not already there: >> >> local all all ident >> >> So, if I have a local system username of 'nate', I would create a postgres user named 'nate' with access to a database 'galaxy', and set database_connection: >> >> database_connection postgres:///galaxy >> >> --nate >> >>> >>> Here is part output when I run the "run.sh" command. >>> ----------------------------------------- >>> galaxy.model.migrate.check DEBUG 2011-12-09 11:02:30,748 psycopg2 egg successfully loaded for postgres dialect >>> Traceback (most recent call last): >>> File "/Users/huayangao/galaxy-dist/lib/galaxy/web/buildapp.py", line 82, in app_factory >>> app = UniverseApplication( global_conf = global_conf, **kwargs ) >>> File "/Users/huayangao/galaxy-dist/lib/galaxy/app.py", line 39, in __init__ >>> create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) >>> File "/Users/huayangao/galaxy-dist/lib/galaxy/model/migrate/check.py", line 54, in create_or_verify_database >>> dataset_table = Table( "dataset", meta, autoload=True ) >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/schema.py", line 108, in __call__ >>> return type.__call__(self, name, metadata, *args, **kwargs) >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/schema.py", line 236, in __init__ >>> _bind_or_error(metadata).reflecttable(self, include_columns=include_columns) >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 1261, in reflecttable >>> conn = self.contextual_connect() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 1229, in contextual_connect >>> return self.Connection(self, self.pool.connect(), close_with_result=close_with_result, **kwargs) >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 142, in connect >>> return _ConnectionFairy(self).checkout() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 304, in __init__ >>> rec = self._connection_record = pool.get() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 161, in get >>> return self.do_get() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 639, in do_get >>> con = self.create_connection() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 122, in create_connection >>> return _ConnectionRecord(self) >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 198, in __init__ >>> self.connection = self.__connect() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/pool.py", line 261, in __connect >>> connection = self.__pool._creator() >>> File "/Users/huayangao/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/strategies.py", line 80, in connect >>> raise exc.DBAPIError.instance(None, None, e) >>> OperationalError: (OperationalError) FATAL: password authentication failed for user "postgres" >>> None None >>> >>> >>> ------------------------------------- >>> >>> >>> Best, >>> >>> Huayan >> > >

1 0