July 2011 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Existing efforts to convert the QIIME pipeline to Galaxy?
by Amanda Zuzolo 27 Jul '11

27 Jul '11

Hello all, I'm working on a local instance of Galaxy at George Mason University. We'd been looking into integrating Qiime and I've found the toolkit very helpful, thanks! I did have one question, though: would you mind uploading the tool_conf file or chunk of text with the qiime functions in it? This would be helpful to myself, as well as others interested in the toolkit! Thank you, Amanda Zuzolo Bioengineering Major, George Mason University

2 1

Re: [galaxy-dev] Regarding submitting jobs as actual user
by Chorny, Ilya 27 Jul '11

27 Jul '11

I put the wrong link in there. The link below is to pull the code. Ilya https://bitbucket.org/ichorny/galaxy-central/overview From: Chorny, Ilya Sent: Wednesday, July 27, 2011 1:22 PM To: galaxy-dev(a)lists.bx.psu.edu Cc: Wohlfrom, Tobias; Duddy, John Subject: Regarding submitting jobs as actual user I completed the changes to galaxy to submit DRMAA jobs (SGE) as the actual galaxy user. You can see the code at the fork below. Below is a brief description of how to turn it which is also described in the universe_wsgi.ini.sample. https://ichorny@bitbucket.org/ichorny/galaxy-central # Uncomment drmaa_external_runjob_script and drmaa_external_killjob_script pameters and have them point to the # absolute path for scripts/drmaa_external_runner.py and scripts/drmaa_external_killer.py. # The scripts directory is located in the top level galaxy directory. The parameters when # uncommented allow for submission to the drmaa queue with the user name of the user submitting # the job and not the galaxy user. In order for this to work the actual user must log into galaxy # and the galaxy authentication must be consistent with the authentication on the server in which the # drmaa queue is running (i.e. the username must have an account on the server and be allowed to # submit jobs to the queue). The galaxy user must also be given sudo permission to execute # scripts/drmaa_external_runner.py and scripts/drmaa_external_killer.py in /etc/sudoers # Example: # galaxy ALL = (root) NOPASSWD: SETENV: /opt/galaxy/scripts/drmaa_external_runner.py # galaxy ALL = (root) NOPASSWD: SETENV: /opt/galaxy/scripts/drmaa_external_killer.py # Also the # Defaults requiretty # in /etc/sudoers must be commented out #drmaa_external_runjob_script = None #drmaa_external_killjob_script = None Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

1 0

Regarding submitting jobs as actual user
by Chorny, Ilya 27 Jul '11

27 Jul '11

I completed the changes to galaxy to submit DRMAA jobs (SGE) as the actual galaxy user. You can see the code at the fork below. Below is a brief description of how to turn it which is also described in the universe_wsgi.ini.sample. https://ichorny@bitbucket.org/ichorny/galaxy-central # Uncomment drmaa_external_runjob_script and drmaa_external_killjob_script pameters and have them point to the # absolute path for scripts/drmaa_external_runner.py and scripts/drmaa_external_killer.py. # The scripts directory is located in the top level galaxy directory. The parameters when # uncommented allow for submission to the drmaa queue with the user name of the user submitting # the job and not the galaxy user. In order for this to work the actual user must log into galaxy # and the galaxy authentication must be consistent with the authentication on the server in which the # drmaa queue is running (i.e. the username must have an account on the server and be allowed to # submit jobs to the queue). The galaxy user must also be given sudo permission to execute # scripts/drmaa_external_runner.py and scripts/drmaa_external_killer.py in /etc/sudoers # Example: # galaxy ALL = (root) NOPASSWD: SETENV: /opt/galaxy/scripts/drmaa_external_runner.py # galaxy ALL = (root) NOPASSWD: SETENV: /opt/galaxy/scripts/drmaa_external_killer.py # Also the # Defaults requiretty # in /etc/sudoers must be commented out #drmaa_external_runjob_script = None #drmaa_external_killjob_script = None Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

1 0

Upload of images - jpg/png - "binary file contains inappropriate content"
by Jelle Scholtalbers 27 Jul '11

27 Jul '11

Hi all, I would like to be able to upload images to my Galaxy instance - in particular jpg/png to data libraries. I can't however find out how to overcome the problem of inappropriate content in binary file. How to go about? When I create a sniffer class in the galaxy.datatypes.images.py that always returns true (for testing purposes) I still don't get the jpg uploaded without an error (sniffer is incl. in the datatypes_conf.xml ). class Jpg( Image ): file_ext = "jpg" def sniff(self, filename): """Determine if the file is in jpg format. """ return True It seems I'm missing something obvious and could use some help. Cheers, Jelle

6 14

problems with two coupled select menus using dynamic_options and refresh_on_change
by Leandro Hermida 27 Jul '11

27 Jul '11

Hi, I am trying to create a form where there are two select menus, both using dynamic_options, where one is the parent menu that is chosen first and based on the selection(s) there is a refresh and the second menu gets populated based on the first menu's choice(s). <param name="names" type="select" multiple="true" optional="true" dynamic_options="dyn_options('names')" refresh_on_change="true" label="Filter By Names"/> <param name="values" type="select" multiple="true" optional="true" dynamic_options="dyn_options('values', $names)" label="Filter by Values" /> There seems to be a problem when the form loads for the first time as $names is not selected yet. I get a syntax error like "Invalid string", when I remove $names as a parameters from my second dynamic_options function call everything is fine. Does someone know how to achieve the functionality I'm aiming for in Galaxy without having to use the deprecated <pages> tags? Is there a way in Cheetah to set a default value for $names and if so how would I do this in the tool xml? regards, Leandro

1 0

Tool XML parameter options from locally stored files
by SHAUN WEBB 27 Jul '11

27 Jul '11

Hi, I am trying to write a tool xml file and want to do the following: I have a number of GTF files stored locally and referenced in a data table. I give the user a drop down list of those available using the following code: <param name="gtf" type="select" label="GTF File"> <options from_data_table="gtf_files"/> </param> I then want them to choose an annotation type based on those present in the selected gtf file above. It seems simple enough to do this if the gtf file is in your history already: <param name="feature" type="select" multiple="true" label="Extract features"> <options from_dataset="gtf"> <column name="name" index="2"/> <column name="value" index="2"/> <filter type="unique_value" name="unique" column="2"/> </options> </param> But is there a way to query locally stored files via datatables/.loc files to pull out options when the file is dynamically chosen by a different parameter? This code is also wrapped in a conditional, I'm not sure if that will affect anything. Thanks Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 5

Re: [galaxy-dev] set_metadata_externally =True
by Chorny, Ilya 27 Jul '11

27 Jul '11

Well I got all the rw permissions changed correctly except for the metadata files written to the temp directory (new_file_path). I still can't seem to track down where those are written. Any help would be greatly appreciated. Best, Ilya From: galaxy-dev-bounces(a)lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Monday, July 25, 2011 4:29 PM To: galaxy-dev(a)lists.bx.psu.edu Subject: [galaxy-dev] set_matadata_externally =True I am still working on the patch to submit jobs as the actual user under SGE . I have it working for the default case but it fails for the case of set_metadata_externally = TRUE. Can someone point me to where the metadata files are created so that I can change the permissions for those files? Also, I have to addusers to the galaxy group in order for this to work because the galaxy_*.sh scripts are created as executable only by members of the galaxy group. I want to change this so that all users can execute these scripts. Any help on where in the code these scripts are created? Upon completion I will create a galaxy central fork and upload my changes to it. Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

1 0

job runner error
by shashi shekhar 27 Jul '11

27 Jul '11

Hi all, I have added separate job runner in my galaxy instance. So far it was running successfully from many months. Now its giving error as below and i am unable to trace it. Please suggest. Traceback (most recent call last): File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 202, in __monitor_step self.dispatcher.put( JobWrapper( job, self ) ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 847, in put self.job_runners[runner_name].put( job_wrapper ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/runners/gw.py", line 370, in put job_wrapper.change_state( model.Job.states.QUEUED ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 428, in change_state self.sa_session.flush() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 1438, in _flush transaction.commit() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 382, in commit t[1].commit() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 1036, in commit self._do_commit() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 1060, in _do_commit self.connection._commit_impl() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 750, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) OperationalError: (OperationalError) database is locked None None Thanks Shashi

2 1

automatic installation of galaxy tools...
by Kostas Karasavvas 27 Jul '11

27 Jul '11

Hi all, I am working on a way to add/install tools to galaxy in a more automated way. I need that for the galaxy-taverna interoperability work we are doing (https://trac.nbic.nl/elabfactory/wiki/eGalaxy) The generator produces a galaxy tool (.zip or an .xml + .rb) that I want to install as easily as possible. Since, this is sth that other people might be interested I was thinking of generalising it. The idea is to create a script (background process) where we specify an association between a galaxy tool directory and a galaxy section. Every time something is copied in that directory it will be picked up by the script, which will check if it is a valid tool, and add it to tool_conf.xml in the specified section (+optionally restart the server). I am assuming that people would have already worked on/thought of something similar for their own needs. So, my questions are: a) anyone knows if sth similar already exists and how reusable it is, and b) does it sound like something useful (apparently, to people that manage tools frequently!) Cheers, Kostas

2 1

galaxy sge Job output not returned from cluster
by ambarish biswas 26 Jul '11

26 Jul '11

Hi All, I'm getting this error "*Job output not returned from cluster*" galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:00:59,552 (4915/17234) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:01:03,552 (4915/17234) state change: job is running 139.80.26.223 - - [26/Jul/2011:17:01:03 +1300] "POST /galaxy/root/history_item_updates HTTP/1.1" 200 - " http://bioanalysis.otago.ac.nz/galaxy/history" "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.205 Safari/534.16" galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:01:04,632 (4915/17234) state change: job finished, but failed galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:01:04,632 Job output not returned from cluster galaxy.jobs DEBUG 2011-07-26 17:01:04,762 job 4915 ended I tried changing permission in __init__.py as mentioned in http://www.mail-archive.com/galaxy-dev@lists.bx.psu.edu/msg01613.html but no luck. I saw Erick has the same problem, and looks like he got a fix as mentioned http://gmod.827538.n3.nabble.com/Job-output-not-returned-from-cluster-td879… but no detail of what should be done. Thanks in advance. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866

2 2