July 2011 - galaxy-dev - lists.galaxyproject.org

BWA sampe runs 'forever'
by Assaf Gordon 29 Jul '11

29 Jul '11

Hello, Question for galaxy maintainers: have you encountered situations where BWA jobs run 'forever' (for days) ? A little digging shows that it's the "bwa sampe" step, and SEQanswers thread mention it's somewhat common: http://seqanswers.com/forums/showthread.php?t=11652 http://seqanswers.com/forums/showthread.php?t=6340 One suggested work-around is to add "-A" to "bwa sampe", but there's no way to do it in the current BWA wrapper. (On the command-line with the same input files it does make things much faster). I'm wondering if this is common enough to justify adding "-A" to the wrapper, or is it rare enough and can be ignored. -gordon

2 3

manage library permission interface glitch
by Glen Beane 29 Jul '11

29 Jul '11

I should preface this by saying we are running a galaxy-dist that is a couple months old now (we are planning on updating soon), but here is an annoying issue we are seeing managing data library permissions: The Roles not associated lists are filtered so it shows only roles associated with the access library permission -- this is good, except this is filtered for the access library permission too, meaning once you add roles to the access library permission and save you can't add any other roles to this permission since the "Roles not associated" for access library now only includes roles associated with access library. To add any new roles we need to first remove every role from this permission, making it public, which then causes all roles to get listed in "roles not associated", and then we can associate all of the roles we want. The correct behavior would be to filter every "roles not associated" list for every permission except for "access library" -- this should always list all roles not associated. I will open a new issue for this on Bitbucket unless a developer can confirm that it has been fixed since my last galaxy-dist update a couple months ago. -- Glen L. Beane Senior Software Engineer The Jackson Laboratory (207) 288-6153

2 2

problem with job runner
by shashi shekhar 29 Jul '11

29 Jul '11

Hi In my galaxy instance, whatever jobs i am submitting it goes into queued state. If I restart the server then the previous submitted jobs state changes to running. but the newly submitted jobs again goes to queued state. I am at a loss to understand this behaviour of galaxy and unable to debug it. The job submission uses a customized runner. How is it actually goes into the queued state automatically when all the workers thread are free? Does galaxy_session table is_valid attribute makes jobs state true? Or what all places in tables the queued states are getting stored. I can only see that the jobs table state attribute only stores the state. The server logs points error here: galaxy.jobs ERROR 2011-07-29 11:01:28,098 failure running job 2243 Traceback (most recent call last): File "/home/gwadmin/galaxy-central/ lib/galaxy/jobs/__init__.py", line 202, in __monitor_step self.dispatcher.put( JobWrapper( job, self ) ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 856, in put self.job_runners[runner_name].put( job_wrapper ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/runners/gw.py", line 375, in put job_wrapper.change_state( model.Job.states.QUEUED ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 437, in change_state self.sa_session.flush() Regards shashi

1 0

dataset_*_file directory being compied to file_path when running fastqc
by Chorny, Ilya 29 Jul '11

29 Jul '11

I am testing the latest galaxy central distribution by running FASTQC. It creates a directory call dataset_*_file in the job_working_directory and then copies that directory to file_path when the job is complete. Is that on purpose or is that a bug? Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

2 2

DRMAA options for SGE
by Ka Ming Nip 28 Jul '11

28 Jul '11

Hi, I am trying to configure the proper memory resource requests for my Galaxy tool. This is what I have under the tool_runners section of universe_wsgi.ini [galaxy:tool_runners] ... mytoolname = drmaa://-l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ... When I execute my tool on Galaxy, I get the error below in the shell that I ran "sh run.sh": galaxy.jobs.runners.drmaa ERROR 2011-07-26 09:54:01,930 Uncaught exception queueing job Traceback (most recent call last): File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 112, in run_next self.queue_job( obj ) File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 177, in queue_job job_id = self.ds.runJob(jt) File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/__init__.py", line 331, in runJob File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/helpers.py", line 213, in c File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/errors.py", line 90, in error_check DeniedByDrmException: code 17: error: no suitable queues All the flags I used work with qsub commands on the SGE cluster I use. The tool runs when I comment out the line in universe_wsgi.ini. Thanks, Ka Ming

2 4

Please sign me off of the galaxy list
by Mai Huynh 28 Jul '11

28 Jul '11

Dear Galaxy, Please sign me off of the galaxy email list. Even though I have been unsubscribed, but I still get all the galaxy emails. I have to delete them every single day. Thank you very much for your times. Best regards, Mai H.

2 1

organizing histories in galaxy
by Chaolin Zhang 28 Jul '11

28 Jul '11

Hi, Is there a way to organize related histories together into a "project" or folder? I realize in a lot of cases, a pipeline is designed for processing of a single sample, while a study typically consists of multiple samples that go through the same processing. They are then put together for more downstream analysis. It seems to me that it would be logical to put the processing of each individual file into a history, and the combined analysis into another, which would be bundled together. I, and other users here have a growing list of histories, and it is becoming more and more difficult to organize them ... Thanks! Chaolin

2 1

Galaxy News Page and RSS Feed
by Dave Clements 28 Jul '11

28 Jul '11

Hello all, The new Galaxy wiki features a Galaxy News page and RSS feed for items of interest to the Galaxy community. The news page is a good way to get information out to the community and is less intrusive than posting to the Galaxy mailing lists. Anyone can create a news items by following directions on the news page, or by sending the item to outreach(a)galaxyproject.org. News items automatically become items in the Galaxy News RSS feed, and are also automatically listed in the right bar on the wiki home page. The most recent posting is about the new Galaxy paper "Making whole genome multiple alignments usable for biologists" appearing in Bioinformatics. News Page http://wiki.g2.bx.psu.edu/News RSS Feed http://feed43.com/galaxynews.xml How to post a news item http://wiki.g2.bx.psu.edu/News#Add_a_News_Item Please let me know if you have any questions (or news items). Thanks, Dave C. -- http://getgalaxy.org http://usegalaxy.org/ http://galaxyproject.org/wiki

1 0

Invalid job runner: drmaa
by remy d1 28 Jul '11

28 Jul '11

Hi, I try to configure Galaxy with SGE. It works one time yesterday but now I have the following message all the time : galaxy.jobs DEBUG 2011-07-26 16:50:54,002 dispatching job 118 to drmaa runner galaxy.jobs ERROR 2011-07-26 16:50:54,002 put(): (118) Invalid job runner: drmaa [ An error occurred running this job: *Unable to run job due to a misconfiguration of the Galaxy job running system. Please contact a site administrator.* ] On the web page.* * Here is my universe_wsgi.ini file : [server:main] port = 8000 host = X.X.X.X use_threadpool = True [filter:proxy-prefix] prefix = /galaxy [app:main] paste.app_factory = galaxy.web.buildapp:app_factory filter-with = proxy-prefix database_connection = mysql://galaxy:xxxxxx@localhost :3306/galaxy?unix_socket=/var/run/mysqld/mysqld.sock file_path = database/files new_file_path = database/tmp job_working_directory = database/users_space/user_galaxy cluster_files_directory = database/users_space/user_galaxy use_nglims = False nglims_config_file = tool-data/nglims.yaml static_enabled = True static_cache_time = 360 static_dir = %(here)s/static/ static_images_dir = %(here)s/static/images static_favicon_dir = %(here)s/static/favicon.ico static_scripts_dir = %(here)s/static/scripts/ static_style_dir = %(here)s/static/june_2007_style/blue log_events = True log_actions = True debug = True use_interactive = True library_import_dir = /home/galaxy/galaxy-dist/tmpdir user_library_import_dir = True allow_library_path_paste = True admin_users = me@... enable_pages = True enable_api = True set_metadata_externally = True retry_metadata_internally = True outputs_to_working_directory = True start_job_runners = drmaa default_cluster_job_runner = drmaa:/// [galaxy:tool_runners] biomart = local:/// encode_db1 = local:/// hbvar = local:/// microbial_import1 = local:/// ucsc_table_direct1 = local:/// ucsc_table_direct_archaea1 = local:/// ucsc_table_direct_test1 = local:/// upload1 = local:/// [galaxy_amqp] I checked many things, but it did not change anything... Any help would gracefull. Regards.

3 4

Re: [galaxy-dev] job runner error
by Peter Cock 28 Jul '11

28 Jul '11

On Wed, Jul 27, 2011 at 2:14 PM, shashi shekhar <meshashi29(a)gmail.com> >>> Hi all, >>> >>> I have added separate job runner in my galaxy instance. So far it was >>> running successfully from many months. >>> Now its giving error as below and i am unable to trace it. Please >>> suggest. >>> >>> Traceback (most recent call last): >>> ... >>> OperationalError: (OperationalError) database is locked None None >>> >>> Thanks >>> Shashi > On Wed, Jul 27, 2011 at 7:00 PM, Peter Cock <p.j.a.cock(a)googlemail.com> >> You're not using SQLite as the database are you? I've seen that kind of >> error message from locking problems (e.g. when the SQLite DB was on >> a CIFS mapped drive), and it wouldn't surprise me that with a separate >> thread for the job runner you get contention over the SQLite DB. >> >> If you are using PostgreSQL (or MySQL) as recommended for a production >> Galaxy server, then I'm not sure what could be wrong. On Wed, Jul 27, 2011 at 2:42 PM, shashi shekhar <meshashi29(a)gmail.com> wrote: > > I am using sqlite database . > I think that to avoid this database problem you will have to move to PostgreSQL (which is what the Galaxy team recommend), or possibly MySQL (not sure what the downsides are). This might be hard if you already have a number of users and datafiles. I've CC'd the list again so hopefully Nate or one of the other Galaxy Devs can confirm this. Peter

2 1