- galaxy-dev - lists.galaxyproject.org

job runner error
by shashi shekhar 27 Jul '11

27 Jul '11

Hi all, I have added separate job runner in my galaxy instance. So far it was running successfully from many months. Now its giving error as below and i am unable to trace it. Please suggest. Traceback (most recent call last): File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 202, in __monitor_step self.dispatcher.put( JobWrapper( job, self ) ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 847, in put self.job_runners[runner_name].put( job_wrapper ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/runners/gw.py", line 370, in put job_wrapper.change_state( model.Job.states.QUEUED ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 428, in change_state self.sa_session.flush() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 1438, in _flush transaction.commit() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/orm/session.py", line 382, in commit t[1].commit() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 1036, in commit self._do_commit() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 1060, in _do_commit self.connection._commit_impl() File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 750, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/home/gwadmin/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.4.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) OperationalError: (OperationalError) database is locked None None Thanks Shashi

2 1

automatic installation of galaxy tools...
by Kostas Karasavvas 27 Jul '11

27 Jul '11

Hi all, I am working on a way to add/install tools to galaxy in a more automated way. I need that for the galaxy-taverna interoperability work we are doing (https://trac.nbic.nl/elabfactory/wiki/eGalaxy) The generator produces a galaxy tool (.zip or an .xml + .rb) that I want to install as easily as possible. Since, this is sth that other people might be interested I was thinking of generalising it. The idea is to create a script (background process) where we specify an association between a galaxy tool directory and a galaxy section. Every time something is copied in that directory it will be picked up by the script, which will check if it is a valid tool, and add it to tool_conf.xml in the specified section (+optionally restart the server). I am assuming that people would have already worked on/thought of something similar for their own needs. So, my questions are: a) anyone knows if sth similar already exists and how reusable it is, and b) does it sound like something useful (apparently, to people that manage tools frequently!) Cheers, Kostas

2 1

galaxy sge Job output not returned from cluster
by ambarish biswas 27 Jul '11

27 Jul '11

Hi All, I'm getting this error "*Job output not returned from cluster*" galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:00:59,552 (4915/17234) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:01:03,552 (4915/17234) state change: job is running 139.80.26.223 - - [26/Jul/2011:17:01:03 +1300] "POST /galaxy/root/history_item_updates HTTP/1.1" 200 - " http://bioanalysis.otago.ac.nz/galaxy/history" "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.205 Safari/534.16" galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:01:04,632 (4915/17234) state change: job finished, but failed galaxy.jobs.runners.drmaa DEBUG 2011-07-26 17:01:04,632 Job output not returned from cluster galaxy.jobs DEBUG 2011-07-26 17:01:04,762 job 4915 ended I tried changing permission in __init__.py as mentioned in http://www.mail-archive.com/galaxy-dev@lists.bx.psu.edu/msg01613.html but no luck. I saw Erick has the same problem, and looks like he got a fix as mentioned http://gmod.827538.n3.nabble.com/Job-output-not-returned-from-cluster-td879… but no detail of what should be done. Thanks in advance. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866

2 2

Getting a list of workflows a user can run via the API
by Duddy, John 27 Jul '11

27 Jul '11

I am doing an integration with Galaxy, and part of what I need to do is trigger workflows. To do that, I need to list them. I can do this if the user owns the workflows, but the API does not return workflows that have been shared with the user. Is there a way via the API to discover the sharing relationships? Or will I need to add this to the API? Thanks! John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy(a)illumina.com<mailto:jduddy@illumina.com>

2 1

set_matadata_externally =True
by Chorny, Ilya 26 Jul '11

26 Jul '11

I am still working on the patch to submit jobs as the actual user under SGE . I have it working for the default case but it fails for the case of set_metadata_externally = TRUE. Can someone point me to where the metadata files are created so that I can change the permissions for those files? Also, I have to addusers to the galaxy group in order for this to work because the galaxy_*.sh scripts are created as executable only by members of the galaxy group. I want to change this so that all users can execute these scripts. Any help on where in the code these scripts are created? Upon completion I will create a galaxy central fork and upload my changes to it. Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

1 0

API workflow_execute.py using ldda
by Andrew Warren 26 Jul '11

26 Jul '11

Hello, I have been working on running a workflow programmatically on my local galaxy instance using library datasets. It took me a few hours to figure out the problem I was having so I thought I would document it here in case it helps anyone. Also, I have a question regarding 'ldda' IDs (see below). After looking at this post: http://gmod.827538.n3.nabble.com/Running-a-workflow-programatically-td25725… the comments in workflow_execute.py I was trying to run a workflow in the following way: 1) get the dataset ID's from the library ./display.py <api_key> http://localhost:8080/api/libraries/ebfb8f50c6abde6d/contents Collection Members ------------------ #1: /api/libraries/ebfb8f50c6abde6d/contents/c6ca0ddb55be603a016bf252082cbf1f name: / type: folder id: c6ca0ddb55be603a016bf252082cbf1f #2: /api/libraries/ebfb8f50c6abde6d/contents/a0e1686ccf7a028a name: /http://smane.vbi.vt.edu/SRR014335.fastq type: file id: a0e1686ccf7a028a #3: /api/libraries/ebfb8f50c6abde6d/contents/98a64dc1a6526c8e name: /http://smane.vbi.vt.edu/SRR014339.fastq type: file id: 98a64dc1a6526c8e #4: /api/libraries/ebfb8f50c6abde6d/contents/2764e29c8c7b7454 name: /http://smane.vbi.vt.edu/Yeast.gtf type: file id: 2764e29c8c7b7454 2) get input IDs for the workflow ./display.py <api_key> http://localhost:8080/api/workflows/f2db41e1fa331b3e Member Information ------------------ url: /api/workflows/f2db41e1fa331b3e inputs: {'102': {'value': '', 'label': 'Illumina2'}, '104': {'value': '', 'label': 'ReferenceAnnotation'}, '96': {'value': '', 'label': 'Illumina1'}} id: f2db41e1fa331b3e Then I ran: python /opt/galaxy_server/galaxy-dist/scripts/api/workflow_execute.py <api_key> http://localhost:8080/api/workflows f2db41e1fa331b3e 'Test API' '102=ldda=a0e1686ccf7a028a' '104=ldda=2764e29c8c7b7454' '96=ldda=98a64dc1a6526c8e' This gives the following error at the end of the stack trace for HTTP Error 500: *ValueError: invalid literal for int() with base 10: 'file.37'* * * After looking through the code for a while I saw that when the "src id type" is given as "ld" then the decoded name is split and the number is used to retrieve the ldda or "library_dataset_dataset_association" So instead I ran the above command with: '102=ld=a0e1686ccf7a028a' '104=ld=2764e29c8c7b7454' '96=ld=98a64dc1a6526c8e' And everything worked, woo! This brings me to my question: Where can I find ldda ID's ? I imagine I could find them in my database and there looks like some functions in lib/galaxy/web/api/workflows.py for handling them but I am having trouble locating them via an API script or on the library management interface. Note: I am assuming that the ID's returned by display.py are in fact "ld" ID's since this is the ID type I used to avoid the error listed above. Thanks, Andrew

1 0

Question about dependency management
by Singh, Indresh 26 Jul '11

26 Jul '11

Greetings, We installed Galaxy at JCVI for internal user but we are discovering that we are missing lots of third party tools/dependencies. My question; is there any script/tool that we can use to checks for all Galaxy dependency software. Also is there any test suite that you guys use to make sure there are no major issues before release. If yes then we can use that test suite to verify our local install and identify any missing dependencies. Thanks again in advance. Regards, Indresh Singh J. Craig Venter Institute(JCVI) 9704 Medical Center Drive Rockville, MD 20850, USA Phone: (301)-795-7129 Fax: (240)-243-0600 www.jcvi.org<http://www.jcvi.org>

2 1

Re: [galaxy-dev] Saved Histories - default view on deleted histories (bug)
by Jeremy Goecks 25 Jul '11

25 Jul '11

Jack, Thanks for following up on this issue. I've fixed the problem in galaxy-central changeset c176363cb7b2: https://bitbucket.org/galaxy/galaxy-central/changeset/c176363cb7b2/ Finally, please send all emails to our mailing lists rather than individual Galaxy developers or other list members. Sending email to the mailing lists ensures that everyone can see and contribute to questions and issues; in addition, emails to mailing lists are tracked and archived so that they are available in the future. Best, J. On Jul 18, 2011, at 11:25 AM, Yuelin Zhu wrote: > Hi Jeremy, > > Sorry for bugging you. But I have been trying to fix the history > problem with my local instance of Galaxy for a while. Basically all > the histories in my Galaxy go to deleted container, including previous > or new created ones (please see some screen shots in the attachments). > I even I tried a new instance of the galaxy-central and see the same > problem. The problem started two or three weeks ago after an update. > I noticed you fixed a similar (same?) bug > (https://bitbucket.org/galaxy/galaxy-central/issue/596/saved-histories-defau…) > so I bring my problem to your attention. Your help will be greatly > appreciated. Please let me know if you need any more information. > > Jack > ---- > Yuelin Jack Zhu

2 1

Error preparing job
by SHAUN WEBB 25 Jul '11

25 Jul '11

Hi all, I'm getting the following error running a custom tool. I'm not really sure where the problem is coming from, it doesn't happen every time I run the tool. Can anyone point me in the right direction? Thanks Shaun Traceback (most recent call last): File "/storage/home/galaxy/test/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 58, in run_job job_wrapper.prepare() File "/storage/home/galaxy/test/galaxy-dist/lib/galaxy/jobs/__init__.py", line 372, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/storage/home/galaxy/test/galaxy-dist/lib/galaxy/tools/__init__.py", line 1585, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/storage/home/galaxy/test/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/storage/home/galaxy/test/galaxy-dist/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1311417753_92_55587.py", line 88, in respond NotFound: cannot find 'checktype' -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 1

Galaxy cloud installation in iPlant Atmosphere?
by Julin Maloof 24 Jul '11

24 Jul '11

Hi All (cross-posting to galaxy-dev and iplant support) The iPlant collaborative is running a "cloud" computer service based on the Amazon platform. https://pods.iplantcollaborative.org/wiki/display/atmosphere/iPlant+Atmosph… http://www.iplantcollaborative.org/discover/atmosphere I am interested in trying to get the cloud version of Galaxy running on an iPlant virtual machine. I was wondering if anyone had already done this? Thanks, Julin Julin Maloof Associate Professor Department of Plant Biology 1 Shields Ave University of California, Davis Davis, CA 95616 (530) 752 8077

1 0