September 2011 - galaxy-dev - lists.galaxyproject.org

debugging jobs in 'new' state
by Shantanu Pavgi 10 May '12

10 May '12

We experienced an issue where some of the galaxy jobs were sitting in the 'new' state for a quite long time. They were not waiting for cluster resources to become available, but haven't been even queued up through DRMAA. We are currently using non-debug mode and following were my observations: * No indication of new jobs in paster.log file * database/pbs script didn't contain any associated job scripts * in backend database - job table contained their galaxy job id but no command_line input was recorded Also, not all the jobs are waiting in the 'new' state. Many jobs submitted after above waiting jobs got completed successfully on the cluster. Is there any job submission logic within galaxy which is being used for submitting jobs? Any clues on how to debug this issue will be really helpful. -- Thanks, Shantanu.

3 4

bug: unsorted bam files and import into data library
by Florian Wagner 15 Feb '12

15 Feb '12

Hi, when a tool outputs an unsorted bam file, the indexing fails (quietly) and its metadata variable "bam_index" points to an inexistent file. This causes a nasty bug when trying to import the dataset into a data library and actually makes the library unusable unless you delete the broken entry from the - in my case Postgresql - database by hand. Are you working on it? Thanks, Florian

2 1

How to allow anonymous users to run workflows?
by Tim te Beek 29 Jan '12

29 Jan '12

Hi all, Was wondering how I can allow anonymous users to run workflows in my local Galaxy instance, as currently users need to be logged in to run workflows. I'd like drop this requirement in light of the intended publication of a workflow in a journal which demands that "Web services must not require mandatory registration by the user.". Could any you tell me how I can accomplish this? I've seen the option to use an external authentication method which could be employed to artificially 'login' anonymous users for a single session, but it appears this would also disable the normal users administration mechanisms in Galaxy, so I'm not sure this would be a good fit. Any hints on how to proceed, either via this route or otherwise, would be much appreciated. Best regards, Tim

2 3

MergeSamFiles.jar and TMPDIR
by Glen Beane 25 Jan '12

25 Jan '12

We recently updated to the latest galaxy-dist, and learned that the sam_merge.xml tool now uses picard MergeSamFiles.jar to merge the files instead of the samtools merge wrapper sam_merge.py. this is a problem for us because MergeSamFiles.jar does not honor $TMPDIR when creating temporary file names (the jvm developers inexplicably hard code the value of java.io.tmpdir to /tmp in Unix/Linux rather than doing the Right Thing) . On our cluster, TMPDIR is set to something like /scratch/batch_job_id/. This location has plenty of free space, however /tmp does not and now we can't successfully merge largeish bam files. In case anyone else is bit by this, I think there are two options the Picard tools take an optional TMP_DIR= argument that lets us specify the location we want to use for a temporary directory. Initially we ended up modifying the .xml to add TMP_DIR=\$TMPDIR to the arguments to MergeSamFiles.jar. This works, but we could potentially need to do this with multiple Picard tools and not just MergeSamFiles. Now I am probably going to go with the following solution: add something like "export _JAVA_OPTIONS=-Djava.io.tmpdir=$TMPDIR" to the .bashrc file for my Galaxy user. -- Glen L. Beane Senior Software Engineer The Jackson Laboratory (207) 288-6153

3 6

Galaxy installation with mysql database
by Alex R Bigelow 23 Jan '12

23 Jan '12

Hi, I am trying to install a local instance of Galaxy with an infobright mysql server; I created a database called "galaxy" (the user is also "galaxy," and it has all the privileges it should need), and the database_connection line is as follows: database_connection = mysql://galaxy:***********@localhost/galaxy?unix_socket=/tmp/mysql-ib.sock When I do this, I get the following error: Traceback (most recent call last): File "/gen21/alex/Apps/galaxy-dist/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/gen21/alex/Apps/galaxy-dist/lib/galaxy/app.py", line 32, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) File "/gen21/alex/Apps/galaxy-dist/lib/galaxy/model/migrate/check.py", line 65, in create_or_verify_database db_schema = schema.ControlledSchema( engine, migrate_repository ) File "/gen21/alex/Apps/galaxy-dist/eggs/sqlalchemy_migrate-0.5.4-py2.7.egg/migrate/versioning/schema.py", line 24, in __init__ self._load() File "/gen21/alex/Apps/galaxy-dist/eggs/sqlalchemy_migrate-0.5.4-py2.7.egg/migrate/versioning/schema.py", line 36, in _load self.table = Table(tname, self.meta, autoload=True) File "/gen21/alex/Apps/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/schema.py", line 108, in __call__ return type.__call__(self, name, metadata, *args, **kwargs) File "/gen21/alex/Apps/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/schema.py", line 236, in __init__ _bind_or_error(metadata).reflecttable(self, include_columns=include_columns) File "/gen21/alex/Apps/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 1265, in reflecttable self.dialect.reflecttable(conn, table, include_columns) File "/gen21/alex/Apps/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/databases/mysql.py", line 1664, in reflecttable sql = self._show_create_table(connection, table, charset) File "/gen21/alex/Apps/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/databases/mysql.py", line 1835, in _show_create_table raise exc.NoSuchTableError(full_name) NoSuchTableError: migrate_version I found this question in the archives: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-March/002216.html As per both replies, I tried a virtual_env, which didn't work, and I also deleted the "galaxy" database so that it would create a fresh one, but, of course, now it can't connect to the database because it doesn't exist. How do I tell Galaxy to create the database it needs? Thanks again for all the support, Alex Bigelow

3 5

SICER bug in Workflow
by Susan Newman 07 Dec '11

07 Dec '11

I'm working on a SOLiD ChIP-Seq workflow in which I want to use SICER for peak detection. In attempting to add SICER to the workflow, I've run into some difficulty: 1) My input is an interval file which I've created using Convert SAM. In order to connect to the SICER module, I had to use a trick by setting up the connection with the Convert SAM set to BED. The SICER algorithm fails to run when tested with the bed file. If I then change the connected Convert SAM to interval, SICER will run. 2) I need to change the default run parameters but they are not saved in the workflow. There is always reversion to the default values. I get a Server error notice. This workflow also contains Bowtie and MACS. Parameter changes for these algorithms are saved correctly. 3) I would like to run SICER twice in the workflow with different parameters. Is it allowable to load SICER a second time? May I run from the same interval conversion file? I tried this and again could not get the input file recognized as a valid connection. Thanks in advance, Susan Susan S. Newman, Laboratory Manager Genomics Core Facility, L2012 Pennington Biomedical Research Center 6400 Perkins Road Baton Rouge, LA 70808 225-763-0255 (o) 225-763-0257 (lab) http://gcf.pbrc.edu

2 1

FASTA manipulation!
by David PANG 29 Nov '11

29 Nov '11

Dear All, I am new to Galaxy! When I tried to manipulate mt fasta sequences from RNA-Seq, I just add >1, >2...........>1000000 etc! Right now, I am trying to add libraryA to all of the sequences like >libraryA_1, >libraryA_2.........>libraryA_1000000. I can not figure out how to do it on Galaxy. Could somebody help to this end. Thanks David -- David PANG, Ph.D.

2 1

custom indexes
by Candace Seeve 29 Nov '11

29 Nov '11

I'd like to map RNA-Seq data from two different bacterial species to their genomes. How do I go about getting a custom index added to the galaxy server? One of the genomes is not yet published so is it possible to keep the indexes private? Thanks, Candace

2 1

Integration with LIMS
by Sonderegger Bernhard 29 Nov '11

29 Nov '11

Hello, I am exploring the possibility of using a local galaxy installation for light bioinformatics on sequence data in the scope of an existing LIMS system. Ideally I would like to be able to do the following 1. Push datasets from the LIMS to galaxy (directly into the user's account or into a fresh temporary session) 2. Allow the user to perform tasks within galaxy, backtracking and retrying as necessary. 3. Push the results along with the galaxy history (ideally converted to a workflow) back to the LIMS 4. (Allow workflows stored in the LIMS to be pushed to Galaxy along with further datasets or even completely automatic launch of these workflows from within the LIMS) Is this feasible? How much wrok would be required? Since I am new to Galaxy (at least from the development side), I would very much appreciate comments advice and pointers to documentation or existing projects of a similar nature. Thanks in advance, Bernhard

2 1

trannsfer files from remote system to galaxy
by shashi shekhar 29 Nov '11

29 Nov '11

Hi, In my local instance of Galaxy ,I want to add one option in which i can get files from remote system to galaxy in data library . except url is there any option to get remote files from galaxy ? Regards

2 1