December 2009 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] [galaxy-bugs] /tmp directory
by juan perin 05 Feb '10

05 Feb '10

Thanks for the answer! I've copied this response to the dev list email instead, after realizing its probably best there. I'm also having some trouble getting our PBS system configured. I have pbs://hostname/ as my pbs parameter in the universe file, as well as everything else specified in the documentation. I'm getting no error in the interface, and the backend never actually gets the job. The error I see in the logs says: galaxy.jobs.runners.pbs ERROR 2009-10-30 13:40:27,762 Uncaught exception queueing job Traceback (most recent call last): File "/home/perin/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 144, in run_next self.queue_job( obj ) File "/home/perin/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 219, in queue_job script = pbs_symlink_template % (job_wrapper.galaxy_lib_dir, " ".join(job_wrapper.get_input_fnames() + output_fnames), self.app.config.pbs_stage_path, exec_dir, command_line) TypeError: sequence item 1: expected string, DatasetPath found I'm guessing this has something to do with my config parameters, and the queue its trying to submit to. the pbs://hostname/ value was used under the assumption that it would default to the 'default' queue on our cluster, so I'm wondering if I need to hard code those values into the connection string? I'm a little confused as to how the syntax translates to the actual required string. My guess is, for our values: pbs://variome.chop.edu/default Perhaps its related to the scrambling of the egg instead? I simply passed LIBTORQUE_DIR to the directory containing my libtorqure.so etc... ? Thanks in advance. Juan Perin On Fri, Oct 30, 2009 at 12:07 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: > Juan Perin wrote: > > I'm hosting galaxy locally and am trying to figure out how to change the >> temporary upload location of files from the root / /tmp folder to something >> else. My /tmp partition is too small to handle large fastq files and this >> is causing me problems. I see where to change things in the universe file, >> but that's the final location, not the temporary space. >> > > Hi Juan, > > A quick solution is to set $TEMP in your shell environment. In the future, > we'll force uploads to honor new_file_path in universe_wsgi.ini. > > --nate >

5 7

order of multiple output datasets
by Alexander Rose 22 Jan '10

22 Jan '10

Hi, (i'm using current galaxy_dist r3173:f7dee0438854) when you have a tool in galaxy that creates multiple output datasets, the order in which you define them in the xml file is not preserved in the history. I believe this is because galaxy collects the output objects in a dict. I made some changes (see the diff below) to use an odict() instead. However this works only partially, the tool gets executed and the order of the output datasets is preserved, but instead of the message you normally see after executing a tool an error is raised (see below). I don't know what went wrong and where, I particularly don't understand how the message.mako template works. Alex * Error URL: http://127.0.0.1:8080/tool_runner/index File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/WebError-0.8a-py2.5.egg/weberror/evalexception/middleware.py', line 364 in respond File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Beaker-1.4-py2.5.egg/beaker/middleware.py', line 152 in __call__ File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/base.py', line 125 in __call__ body = self.call_body_method( method, trans, kwargs ) File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/base.py', line 144 in call_body_method return method( trans, **kwargs ) File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/controllers/tool_runner.py', line 59 in index return trans.fill_template( template, history=history, toolbox=toolbox, tool=tool, util=util, add_frame=add_frame, **vars ) File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 593 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 604 in fill_template_mako return template.render( **data ) File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/template.py', line 133 in render File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 364 in _render File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 381 in _render_context File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 414 in _exec_template File '/Users/alexrose/mc-galaxy/galaxy_dist/database/compiled_templates/base.mako.py', line 37 in render_body __M_writer(unicode(next.body())) File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 255 in <lambda> File '/Users/alexrose/mc-galaxy/galaxy_dist/database/compiled_templates/message.mako.py', line 45 in render_body __M_writer(unicode(_(message))) File '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/gettext.py', line 400 in ugettext tmsg = self._catalog.get(message, missing) TypeError: unhashable instance * Diff Index: /Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Wed Dec 16 17:10:05 2009 +0100 +++ b/lib/galaxy/tools/__init__.py Fri Dec 18 16:20:53 2009 +0100 @@ -427,7 +427,7 @@ # Parse tool help self.parse_help( root ) # Description of outputs produced by an invocation of the tool - self.outputs = {} + self.outputs = odict() out_elem = root.find("outputs") if out_elem: for data_elem in out_elem.findall("data"): Index: /Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/tools/actions/__init__.py --- a/lib/galaxy/tools/actions/__init__.py Wed Dec 16 17:10:05 2009 +0100 +++ b/lib/galaxy/tools/actions/__init__.py Fri Dec 18 16:20:53 2009 +0100 @@ -7,6 +7,7 @@ from galaxy.jobs import JOB_OK import galaxy.tools from types import * +from galaxy.util.odict import odict import logging log = logging.getLogger( __name__ ) @@ -117,7 +118,7 @@ input_values[ input.name ] = galaxy.tools.SelectToolParameterWrapper( input, input_values[ input.name ], tool.app, other_values = incoming ) else: input_values[ input.name ] = galaxy.tools.InputValueWrapper( input, input_values[ input.name ], incoming ) - out_data = {} + out_data = odict() # Collect any input datasets from the incoming parameters inp_data = self.collect_input_datasets( tool, incoming, trans )

2 1

Crash while attempting to saving bowtie output
by Chris Cole 12 Jan '10

12 Jan '10

Hi, On our local install, we've just enabled bowtie NGS mapping and although the search runs fine and the output is viewable in Galaxy, any attempt to save it brings down the server. The only error in paster.log is: galaxy.datatypes.registry WARNING 2009-12-18 16:32:38,517 unknown mimetype in data factory sam Tried it on large files and small files with the same effect. Trying save as fails with an 'unable to read file error'. Anyone have any ideas? Cheers, Chris

3 9

replacing deprecated <code> tags
by Martin Aryee 06 Jan '10

06 Jan '10

Hi, I'm trying to work out how to run custom python code as a tool is loaded (to get generate options for a select parameter) and after job completion (to set metadata). It looks like the <code> tags would have been the place to point to my code, but since they are now deprecated I was wondering if someone could point me in the right direction? Thanks, Martin Aryee.

2 4

using the 'user' information in the the "tool config file"
by Hotz, Hans-Rudolf 05 Jan '10

05 Jan '10

Hi By accident, my colleague found out, that we can use the $userEmail variable in the 'command' tag of the "tool config file", eg: <command interpreter="perl">script.pl $input $userEmail $output </command> Access to the 'user' information ('Anonymous' or the user's e-mail address if logged in) is very handy, as it gives the potential to execute the script differently depending on the value of $userEmail . Now, we would like to use the 'user' information in order to create different options in the interface for the tool, eg: <inputs> <param name="input" type="select" label="available files" display="radio" dynamic_options="ds_Options()"/> </inputs> <outputs> <data format="fasta" name="output"/> </outputs> <code file="more_code.py" /> where "ds_Options()" is defined in the extra python code "more_code.py" I can pass a value to "ds_Options()" which we get as an example from a first page of the interface, eg: <inputs> <page> <param name="dir" type="select" display="radio"> <option value="foo">FOO</option> <option value="bar">BAR</option> </param> </page> <page> <param name="input" type="select" label="available files" display="radio" dynamic_options="ds_Options(dir)"/> </page> </inputs> <outputs> <data format="fasta" name="output"/> </outputs> <code file="more_code.py" /> ...but how do I pass just the 'user' information to the function? As I can use '$userEmail' in the 'command' tag, I tried (with my limited understanding of galaxy and python) the obvious: dynamic_options="ds_Options(userEmail)" but I get NameError: name 'userEmail' is not defined Is there any way to pass the 'user' information? Alternatively, is there a direct way to get access to the 'user' information within the function? Thank you very much for your help Hans

5 5

Controlling frames for export tool
by Chris Zaleski 04 Jan '10

04 Jan '10

Greetings, I have implemented a tool to export data to an external site. I used "tools/data_destination/epigraph.xml" as a template & this works nicely. I am simply wondering if there is a way to control the destination frame of the external website. Currently the entire window is replaced. If possible, I would like it to appear in the content area (galaxy_main?), similar to the way many of the "Get Data" tools work (eg: UCSC Main). Thanks very much, Chris Zaleski

5 6

Security
by Goyder, Matthew 04 Jan '10

04 Jan '10

Dear Dev Team, We are planning on installing the Galaxy framework on our production systems and understandably we are trying to examine the exposure to security risks that we might experience with deploying the system. Has Galaxy been profiled for security flaws? Are there any areas of the system which are historically, or have recently been identified as having security problems? Many thanks, Matt Goyder ----------------------------------------- Confidentiality Notice: The following mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. The recipient is responsible to maintain the confidentiality of this information and to use the information only for authorized purposes. If you are not the intended recipient (or authorized to receive information for the intended recipient), you are hereby notified that any review, use, disclosure, distribution, copying, printing, or action taken in reliance on the contents of this e-mail is strictly prohibited. If you have received this communication in error, please notify us immediately by reply e-mail and destroy all copies of the original message. Thank you.

2 1

Galaxy support to time series
by /fox/ 31 Dec '09

31 Dec '09

Hi :) I'm completely new to galaxy so sorry for bothering you with maybe silly questions. I'd like to create a web service based on galaxy to process time series using clustering algorithms in order to semplify complex data into representative clusters. Our tools are python scripts that take as input a matrix containing time series, each row is a timeseries and each column a point of the time series. Usually we use CSV for files. We have files like: id|01/01/2009|02/01/2009|03/01/2009|04/01/2009 id1|5|3|3.21|8 id2|2|1|9.1|2 id3|5.2|2.43|1|4 Where the first row is the header and the others row the the id of each time series and then the time series itself. Have I to add a new datatype?Or can i use tabular? outputs and inputs are always saved on file right?and at any process are parsed again? I need then the plot of the time series and the clusters how can i do it? Thank you -- f. "E' in un giorno di pioggia che ti ho conosciuta e il vento dell'ovest rideva gentile..." (Modena City Ramblers) http://fox.noblogs.org/

2 2

Licence
by Goyder, Matthew 22 Dec '09

22 Dec '09

Hello, If we were to significantly modify Galaxy (remove the logo / change colors / write new functionality within the package) must we then make all of the source code available publically? Thanks, Matt Goyder -- matthew.goyder(a)nationwidechildrens.org Battelle Center for Mathematical Medicine The Research Institute at Nationwide Children's Hospital Office: (614)355-2395

2 1

server [http://galaxy.psu.edu/] down?
by Nina Stoletzki 22 Dec '09

22 Dec '09

Hi Galaxy I believe your servers may be down? http://g2.bx.psu.edu/ http://galaxy.psu.edu/ Do you know when they will be accessible again? Cheers, NIna

2 1