November 2013 - galaxy-dev - lists.galaxyproject.org

history item status not updating
by Robert Baertsch 12 May '14

12 May '14

I submitted a workflow that in turn submits a drm job to sun grid engine. The queue had an error (probably due to a problem with automount). Traceback (most recent call last): File "/inside/depot4/galaxy/lib/galaxy/jobs/runners/__init__.py", line 60, in run_next method(arg) File "/inside/depot4/galaxy/lib/galaxy/jobs/runners/drmaa.py", line 169, in queue_job external_job_id = self.ds.runJob(jt) File "/inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/__init__.py", line 331, … [View More]

3 3

Help with cluster setup
by Jurgens de Bruin 01 May '14

01 May '14

Hi, This is my first Galaxy installation setup so apologies for stupid questions. I am setting up Galaxy on a Cluster running Torque as the resource manager. I am working through the documentation but I am unclear on some things: Firstly I am unable to find : *start_job_runners *within the universe_wsgi.ini and I dont want to just add this anywhere - any help on this would be create. Further more this is my job_conf.xml : <?xml version="1.0"?> <!-- A sample job config that … [View More]

4 8

Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
by Guest, Simon 25 Apr '14

25 Apr '14

Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a … [View More]particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: <tool_dependency> <package name="bwa" version="0.6.2"> <install version="1.0"> <actions> <action type="download_by_url">http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2</action> <action type="shell_command">make</action> <action type="move_file"> <source>bwa</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> </package> </tool_dependency> We have something like this (NB: element and attribute names are for illustrative purposes only): <tool_dependency> <package name="bwa" version="0.6.2"> <use_native> <actions> <action type="module_load">bwa/0.6.2</action> </actions> </use_native> </package> </tool_dependency> This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with the existing download-and-make one. We might want a bit of flexibility on what actions are supported (in case we want to support Software Collections, for example). What do you think? cheers, Simon PS: In case it wasn't already clear, solving this problem well is quite important to us here at AgResearch. ;-) ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ======================================================================= [View Less]

4 7

registration problem
by Wesley Schaal 22 Apr '14

22 Apr '14

Galaxy-Dev, I have a new galaxy installation under Linux. It's running as an Apache proxy (subdirectory) with authentication through galaxy (not apache). When trying to register a new user (which will be the administrator), the registration page declare success ("Now logged in as foo") but I am still just an anonymous user. If I try to login, it appears to succeed but I'm still anonymous. If I purposely enter the wrong password, the system complains as it should but I never get proper … [View More]

3 2

Jobs deleted staying in 'dr' status
by Mathieu Bahin 10 Apr '14

10 Apr '14

Hi all, We have been developing our own Galaxy instance for a while now. We have a cluster on which the job are sent to be executed, it is managed through SGE. Usually, communication between SGE and DRMAA is ok and we don't have any problem with that. When a job is deleted by the user, most of the times, the job disappears but sometimes, we don't know why, the job stays and has the status 'dr' within SGE. If we don't kill it 'manually', it stays forever. It is not always the same tools … [View More]

4 3

Re: [galaxy-dev] Plugins
by Michael E. Cotterell 12 Mar '14

12 Mar '14

Over next couple days, I'll take a look at what's in lib/galaxy/web/base/pluginframework.py and see if there is a way to refactor it with my stuff in order to get a general plugin framework (galaxy.pluginframework) as proposed by James. Sorry for the delay. Teaching a math class this semester is really eating up my time. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA & TA, University of Georgia Department … [View More]Liaison, CS Graduate Student Association, University of Georgia mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) mepcott(a)uga.edu (mailto:mepcott@uga.edu) mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) http://michaelcotterell.com/ On Tuesday, September 3, 2013 at 12:45 PM, Carl Eberhard wrote: > It is web-specific and only handles the mako/static serving cases. I think the only areas we were generalizing at this point were the directory structures and main configuration. > > Agreed on more generalization and the hook system. Michael's stuff is a good direction to move in. > > I know John was looking to use it for dynamic job destinations and dynamic toolbox filters. I'd like to see some lab/community discussion on (even broad stroke) functional requirements and use cases. > > > On Tue, Sep 3, 2013 at 12:27 PM, James Taylor <james(a)jamestaylor.org (mailto:james@jamestaylor.org)> wrote: > > Carl, is what you have now totally web specific? It would be great to > > have a general plugin framework (galaxy.pluginframework) which the web > > stuff could perhaps extend? > > > > > > On Tue, Sep 3, 2013 at 10:56 AM, Carl Eberhard <carlfeberhard(a)gmail.com (mailto:carlfeberhard@gmail.com)> wrote: > > > Heya, Michael > > > > > > Right now our only plugin code is located in > > > lib/galaxy/web/base/pluginframework.py. It's only purpose so far is to: > > > - allow locally modified/created code to serve mako templates and static > > > pages/resources from paste. > > > - serve as a super class for the visualization framework. > > > > > > I think your git repo has some great ideas. > > > > > > Carl > > > > > > > > > On Fri, Aug 30, 2013 at 2:25 PM, Michael Cotterell <mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com)> > > > wrote: > > > > > > > > Is this plugin code hosted anywhere (like in central)? > > > > > > > > > > > > On Monday, August 26, 2013, James Taylor wrote: > > > > > > > > > > Michael, > > > > > > > > > > Carl has been working on a plugin framework for Galaxy, and John > > > > > Chilton has made some improvements. It would be great if you could > > > > > look over what each other has done and come up with some suggestions. > > > > > I personally am in favor of a very generic hook system where a hook > > > > > can be defined anywhere in the code and then the user can provide any > > > > > callable. > > > > > > > > > > I'm fond of the way sup implemented hooks, see the use of HookManager > > > > > here: https://github.com/sup-heliotrope/sup/blob/develop/lib/sup/time.rb > > > > > > > > > > Your approach is quite similar and I like it. > > > > > > > > > > This makes it easy to have a script that can list all available hooks > > > > > with their documentation. > > > > > > > > > > > > > > > -- > > > > > James Taylor, Assistant Professor, Biology/CS, Emory University > > > > > > > > > > > > > > > On Fri, Aug 23, 2013 at 10:29 AM, Michael E. Cotterell > > > > > <mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com)> wrote: > > > > > > James, > > > > > > > > > > > > Has there been any more talk about this? I'd be willing to work in my > > > > > > reference implementation into central as a pull request, but I don't want to > > > > > > do that unless there is an agreed upon roadmap or something. > > > > > > > > > > > > Thanks! > > > > > > > > > > > > Sincerely, > > > > > > Michael E. Cotterell > > > > > > > > > > > > Ph.D. Student in Computer Science, University of Georgia > > > > > > Instructor of Record, Graduate RA & TA, University of Georgia > > > > > > Department Liaison, CS Graduate Student Association, University of > > > > > > Georgia > > > > > > mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) (mailto:mepcotterell@gmail.com) > > > > > > mepcott(a)uga.edu (mailto:mepcott@uga.edu) (mailto:mepcott@uga.edu) > > > > > > mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) (mailto:mec@cs.uga.edu) > > > > > > http://michaelcotterell.com/ > > > > > > > > > > > > > > > > > > On Tuesday, July 16, 2013 at 2:08 PM, Michael E. Cotterell wrote: > > > > > > > > > > > > > James, > > > > > > > > > > > > > > That's exactly what I was thinking. In my opinion, hooks are the way > > > > > > > to go (as seen in my example), but if there's a better way then I'm game for > > > > > > > that to. I just want to be able to extend Galaxy without modifying Galaxy > > > > > > > itself. > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > Sincerely, > > > > > > > Michael E. Cotterell > > > > > > > > > > > > > > Ph.D. Student in Computer Science, University of Georgia > > > > > > > Instructor of Record, Graduate RA & TA, University of Georgia > > > > > > > Faculty Liaison, CS Graduate Student Association, University of > > > > > > > Georgia > > > > > > > mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) (mailto:mepcotterell@gmail.com) > > > > > > > mepcott(a)uga.edu (mailto:mepcott@uga.edu) (mailto:mepcott@uga.edu) > > > > > > > mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) (mailto:mec@cs.uga.edu) > > > > > > > http://michaelcotterell.com/ > > > > > > > > > > > > > > > > > > > > > On Tuesday, July 16, 2013 at 2:05 PM, James Taylor wrote: > > > > > > > > > > > > > > > Michael, > > > > > > > > > > > > > > > > Galaxy is all about plugins! Tools, datatypes, batch systems, > > > > > > > > visualizations, et cetera. > > > > > > > > > > > > > > > > Additional ways to extend Galaxy through plugins are very welcome. > > > > > > > > We > > > > > > > > just need to make sure that whatever solution is chosen is extremely > > > > > > > > general. > > > > > > > > > > > > > > > > -- > > > > > > > > James Taylor, Assistant Professor, Biology/CS, Emory University > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 16, 2013 at 1:57 PM, Michael E. Cotterell > > > > > > > > <mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) (mailto:mepcotterell@gmail.com)> wrote: > > > > > > > > > Are there any plans in the works to support plugins within Galaxy? > > > > > > > > > One of the tools I'm working on is actually more of an extension of the > > > > > > > > > workflow canvas/editor's user interface. Right now, I've minimized the > > > > > > > > > modifications required to the editor.mako file to simply adding a mako > > > > > > > > > include near the bottom of the file. However, like I just said, this > > > > > > > > > requires editing a Galaxy file. It would be a lot better if Galaxy supported > > > > > > > > > plugins with hooks, thus allowing me to extend the user interface (or any > > > > > > > > > other part of the application) by registering hooks in a plugin. > > > > > > > > > > > > > > > > > > To demonstrate how easy it would be to integrate plugins into a > > > > > > > > > python application, I've written up a simple plugin manager script (~100 > > > > > > > > > loc) an placed it in a gist: https://gist.git > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Sincerely, > > > > Michael E. Cotterell > > > > > > > > Ph.D. Student in Computer Science, University of Georgia > > > > Graduate RA & TA, University of Georgia > > > > mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) > > > > mepcott(a)uga.edu (mailto:mepcott@uga.edu) > > > > mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) > > > > http://michaelcotterell.com/ > > > > > > [View Less]

2 4

Difficulty dispatching toolshed tool jobs via job_conf.xml
by Guest, Simon 27 Feb '14

27 Feb '14

I am having trouble getting a toolshed tool to be dispatched to the destination I list in the job_conf.xml file. My job_conf.xml file looks like this: <?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="20"/> <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner" /> </plugins> <handlers> <handler id="main"… [View More]/> </handlers> <destinations default="condor"> <destination id="local" runner="local"/> <destination id="condor" runner="condor">  </destination> </destinations> <tools> <tool id="upload1" destination="local"/>  <tool id="ucsc_table_direct1" destination="local"/>  ... stuff omitted ... <tool id="toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS: infoseq46/5.0.0" destination="local"/> </tools> </job_conf> So, you can see I have a default destination of condor, but I'm trying to run my toolshed EMBOSS infoseq tool on local. However, it is stubbornly running on condor. In lib/galaxy/tools/__init__.py:1132, I see this comment which got me wondering: # In the toolshed context, there is no job config. Is it possible to define tool destinations for toolshed tools? Are there some gotchas that I should know about? Any other ideas why my job is ignoring the config in job_conf.xml? (By the way, I can change say the upload1 tool to run on Condor by setting its destination in that file, so it is doing something.) The other thing I saw in the source code is stuff about old_id and toolshed guids. Do I need to understand this stuff? The paster.log contains the following when I submit the infoseq job: 147.158.130.216 - - [11/Sep/2013:13:53:34 +1300] "GET /tool_runner?tool_id=toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS%3A%20infoseq46/5.0.0 HTTP/1.1" 200 - "http://galaxy-dev.agresearch.co.nz/" "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" 147.158.130.216 - - [11/Sep/2013:13:53:40 +1300] "POST /tool_runner/index HTTP/1.1" 200 - "http://galaxy-dev.agresearch.co.nz/tool_runner?tool_id=toolshed-dev.agresea…" "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" galaxy.jobs DEBUG 2013-09-11 13:53:40,886 (92) Working directory for job is: /home/galaxy-dev/galaxy/database/job_working_directory/000/92 galaxy.tools DEBUG 2013-09-11 13:53:40,886 Tool::get_job_destination: {'runner': 'condor', 'legacy': False, 'params': {}, 'tags': None, 'url': None, 'converted': False, 'id': 'condor'}. galaxy.jobs.handler DEBUG 2013-09-11 13:53:40,894 (92) Dispatching to condor runner (I added the debug output for Tool::get_job_destination to see what was going on.) Any ideas? cheers, Simon ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ======================================================================= [View Less]

3 8

Blast+: new feature filter/mask... by taxid before job in one task
by Berner, Thomas 26 Feb '14

26 Feb '14

Hi guys, We recently installed NCBI BLAST + to our local Galaxy instance and now we need to provide the possibility to filter/mask ... by taxon id (taxid) using the command line option (-window_masker_taxid) of BLAST (cf. http://www.ncbi.nlm.nih.gov/books/NBK1763/) before the job will be performed in one task. However searching in the documentation and the mailing list, I did not find anything about this. So it would be great to provide the possibility to subselect by taxid (e.g. NCBI BLAST … [View More]

3 4

Galaxy Tool Shed packages for Biopython
by Peter Cock 18 Feb '14

18 Feb '14

Hi all, I've send this to both the Galaxy and Biopython developers lists, and hope this will make sense to both groups. If you've not heard of Galaxy, start here: http://galaxyproject.org - while the easy to guess Biopython website is at http://biopython.org Brad Chapman and I are both Biopython core developers, and are also both on the "IUC" Galaxy Tool Shed committee because we've been quite involved in wrapping and writing tools for use on Galaxy. Fellow committee member Björn Grüning has … [View More]

2 3

shed tool dependencies
by Geert Vandeweyer 13 Feb '14

13 Feb '14

Hi, Is there a default notation to specify perl modules (threading modules) in a tool configuration or in a submission to the toolshed? I've got some tools to share that make extensive use of these modules, and they are not default in a perl distribution. Hence, I'd like to inform the users that they need to install these modules (or that they get installed by the the toolshed using cpan ?) Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp … [View More]

4 3