Re: [galaxy-dev] [galaxy-bugs] /tmp directory
by juan perin
Thanks for the answer! I've copied this response to the dev list email
instead, after realizing its probably best there.
I'm also having some trouble getting our PBS system configured. I have
pbs://hostname/ as my pbs parameter in the universe file, as well as
everything else specified in the documentation. I'm getting no error in the
interface, and the backend never actually gets the job. The error I see in
the logs says:
galaxy.jobs.runners.pbs ERROR 2009-10-30 13:40:27,762 Uncaught exception
queueing job
Traceback (most recent call last):
File "/home/perin/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 144,
in run_next
self.queue_job( obj )
File "/home/perin/galaxy-dist/lib/galaxy/jobs/runners/pbs.py", line 219,
in queue_job
script = pbs_symlink_template % (job_wrapper.galaxy_lib_dir, "
".join(job_wrapper.get_input_fnames() + output_fnames),
self.app.config.pbs_stage_path, exec_dir, command_line)
TypeError: sequence item 1: expected string, DatasetPath found
I'm guessing this has something to do with my config parameters, and the
queue its trying to submit to. the pbs://hostname/ value was used under the
assumption that it would default to the 'default' queue on our cluster, so
I'm wondering if I need to hard code those values into the connection
string? I'm a little confused as to how the syntax translates to the actual
required string. My guess is, for our values:
pbs://variome.chop.edu/default
Perhaps its related to the scrambling of the egg instead? I simply passed
LIBTORQUE_DIR to the directory containing my libtorqure.so etc...
? Thanks in advance.
Juan Perin
On Fri, Oct 30, 2009 at 12:07 PM, Nate Coraor <nate(a)bx.psu.edu> wrote:
> Juan Perin wrote:
>
> I'm hosting galaxy locally and am trying to figure out how to change the
>> temporary upload location of files from the root / /tmp folder to something
>> else. My /tmp partition is too small to handle large fastq files and this
>> is causing me problems. I see where to change things in the universe file,
>> but that's the final location, not the temporary space.
>>
>
> Hi Juan,
>
> A quick solution is to set $TEMP in your shell environment. In the future,
> we'll force uploads to honor new_file_path in universe_wsgi.ini.
>
> --nate
>
12 years, 8 months
order of multiple output datasets
by Alexander Rose
Hi,
(i'm using current galaxy_dist r3173:f7dee0438854)
when you have a tool in galaxy that creates multiple output datasets, the order in which you define them in the xml file is not preserved in the history.
I believe this is because galaxy collects the output objects in a dict. I made some changes (see the diff below) to use an odict() instead.
However this works only partially, the tool gets executed and the order of the output datasets is preserved, but instead of the message you normally see after executing a tool an error is raised (see below).
I don't know what went wrong and where, I particularly don't understand how the message.mako template works.
Alex
* Error
URL: http://127.0.0.1:8080/tool_runner/index
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/WebError-0.8a-py2.5.egg/weberror/evalexception/middleware.py', line 364 in respond
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/debug/prints.py', line 98 in __call__
environ, self.app)
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/wsgilib.py', line 539 in intercept_output
app_iter = application(environ, replacement_start_response)
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Beaker-1.4-py2.5.egg/beaker/middleware.py', line 152 in __call__
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/recursive.py', line 80 in __call__
return self.application(environ, start_response)
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Paste-1.6-py2.5.egg/paste/httpexceptions.py', line 632 in __call__
return self.application(environ, start_response)
File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/base.py', line 125 in __call__
body = self.call_body_method( method, trans, kwargs )
File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/base.py', line 144 in call_body_method
return method( trans, **kwargs )
File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/controllers/tool_runner.py', line 59 in index
return trans.fill_template( template, history=history, toolbox=toolbox, tool=tool, util=util, add_frame=add_frame, **vars )
File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 593 in fill_template
return self.fill_template_mako( filename, **kwargs )
File '/Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 604 in fill_template_mako
return template.render( **data )
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/template.py', line 133 in render
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 364 in _render
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 381 in _render_context
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 414 in _exec_template
File '/Users/alexrose/mc-galaxy/galaxy_dist/database/compiled_templates/base.mako.py', line 37 in render_body
__M_writer(unicode(next.body()))
File '/Users/alexrose/mc-galaxy/galaxy_dist/eggs/py2.5-noplatform/Mako-0.2.5-py2.5.egg/mako/runtime.py', line 255 in <lambda>
File '/Users/alexrose/mc-galaxy/galaxy_dist/database/compiled_templates/message.mako.py', line 45 in render_body
__M_writer(unicode(_(message)))
File '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/gettext.py', line 400 in ugettext
tmsg = self._catalog.get(message, missing)
TypeError: unhashable instance
* Diff
Index: /Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/tools/__init__.py
--- a/lib/galaxy/tools/__init__.py Wed Dec 16 17:10:05 2009 +0100
+++ b/lib/galaxy/tools/__init__.py Fri Dec 18 16:20:53 2009 +0100
@@ -427,7 +427,7 @@
# Parse tool help
self.parse_help( root )
# Description of outputs produced by an invocation of the tool
- self.outputs = {}
+ self.outputs = odict()
out_elem = root.find("outputs")
if out_elem:
for data_elem in out_elem.findall("data"):
Index: /Users/alexrose/mc-galaxy/galaxy_dist/lib/galaxy/tools/actions/__init__.py
--- a/lib/galaxy/tools/actions/__init__.py Wed Dec 16 17:10:05 2009 +0100
+++ b/lib/galaxy/tools/actions/__init__.py Fri Dec 18 16:20:53 2009 +0100
@@ -7,6 +7,7 @@
from galaxy.jobs import JOB_OK
import galaxy.tools
from types import *
+from galaxy.util.odict import odict
import logging
log = logging.getLogger( __name__ )
@@ -117,7 +118,7 @@
input_values[ input.name ] = galaxy.tools.SelectToolParameterWrapper( input, input_values[ input.name ], tool.app, other_values = incoming )
else:
input_values[ input.name ] = galaxy.tools.InputValueWrapper( input, input_values[ input.name ], incoming )
- out_data = {}
+ out_data = odict()
# Collect any input datasets from the incoming parameters
inp_data = self.collect_input_datasets( tool, incoming, trans )
12 years, 8 months
Crash while attempting to saving bowtie output
by Chris Cole
Hi,
On our local install, we've just enabled bowtie NGS mapping and although
the search runs fine and the output is viewable in Galaxy, any attempt
to save it brings down the server.
The only error in paster.log is:
galaxy.datatypes.registry WARNING 2009-12-18 16:32:38,517 unknown
mimetype in data factory sam
Tried it on large files and small files with the same effect.
Trying save as fails with an 'unable to read file error'. Anyone have
any ideas?
Cheers,
Chris
12 years, 8 months
replacing deprecated <code> tags
by Martin Aryee
Hi,
I'm trying to work out how to run custom python code as a tool is loaded (to
get generate options for a select parameter) and after job completion (to
set metadata). It looks like the <code> tags would have been the place to
point to my code, but since they are now deprecated I was wondering if
someone could point me in the right direction?
Thanks,
Martin Aryee.
12 years, 9 months
using the 'user' information in the the "tool config file"
by Hotz, Hans-Rudolf
Hi
By accident, my colleague found out, that we can use the $userEmail variable
in the 'command' tag of the "tool config file", eg:
<command interpreter="perl">script.pl $input $userEmail $output </command>
Access to the 'user' information ('Anonymous' or the user's e-mail address
if logged in) is very handy, as it gives the potential to execute the script
differently depending on the value of $userEmail .
Now, we would like to use the 'user' information in order to create
different options in the interface for the tool, eg:
<inputs>
<param name="input" type="select" label="available files"
display="radio"
dynamic_options="ds_Options()"/>
</inputs>
<outputs>
<data format="fasta" name="output"/>
</outputs>
<code file="more_code.py" />
where "ds_Options()" is defined in the extra python code "more_code.py"
I can pass a value to "ds_Options()" which we get as an example from a first
page of the interface, eg:
<inputs>
<page>
<param name="dir" type="select" display="radio">
<option value="foo">FOO</option>
<option value="bar">BAR</option>
</param>
</page>
<page>
<param name="input" type="select" label="available files"
display="radio"
dynamic_options="ds_Options(dir)"/>
</page>
</inputs>
<outputs>
<data format="fasta" name="output"/>
</outputs>
<code file="more_code.py" />
...but how do I pass just the 'user' information to the function?
As I can use '$userEmail' in the 'command' tag, I tried (with my limited
understanding of galaxy and python) the obvious:
dynamic_options="ds_Options(userEmail)"
but I get
NameError: name 'userEmail' is not defined
Is there any way to pass the 'user' information?
Alternatively, is there a direct way to get access to the 'user' information
within the function?
Thank you very much for your help
Hans
12 years, 9 months
Controlling frames for export tool
by Chris Zaleski
Greetings,
I have implemented a tool to export data to an external site. I used
"tools/data_destination/epigraph.xml" as a template & this works nicely. I
am simply wondering if there is a way to control the destination frame of
the external website. Currently the entire window is replaced. If possible,
I would like it to appear in the content area (galaxy_main?), similar to the
way many of the "Get Data" tools work (eg: UCSC Main).
Thanks very much,
Chris Zaleski
12 years, 9 months
Security
by Goyder, Matthew
Dear Dev Team,
We are planning on installing the Galaxy framework on our production systems and understandably we are trying to examine the exposure to security risks that we might experience with deploying the system. Has Galaxy been profiled for security flaws? Are there any areas of the system which are historically, or have recently been identified as having security problems?
Many thanks,
Matt Goyder
----------------------------------------- Confidentiality Notice:
The following mail message, including any attachments, is for the
sole use of the intended recipient(s) and may contain confidential
and privileged information. The recipient is responsible to
maintain the confidentiality of this information and to use the
information only for authorized purposes. If you are not the
intended recipient (or authorized to receive information for the
intended recipient), you are hereby notified that any review, use,
disclosure, distribution, copying, printing, or action taken in
reliance on the contents of this e-mail is strictly prohibited. If
you have received this communication in error, please notify us
immediately by reply e-mail and destroy all copies of the original
message. Thank you.
12 years, 9 months
Galaxy support to time series
by /fox/
Hi :)
I'm completely new to galaxy so sorry for bothering you with maybe silly
questions.
I'd like to create a web service based on galaxy to process time series
using clustering algorithms in order to semplify complex data into
representative clusters.
Our tools are python scripts that take as input a matrix containing time
series, each row is a timeseries and each column a point of the time series.
Usually we use CSV for files.
We have files like:
id|01/01/2009|02/01/2009|03/01/2009|04/01/2009
id1|5|3|3.21|8
id2|2|1|9.1|2
id3|5.2|2.43|1|4
Where the first row is the header and the others row the the id of each
time series and then the time series itself.
Have I to add a new datatype?Or can i use tabular?
outputs and inputs are always saved on file right?and at any process are
parsed again?
I need then the plot of the time series and the clusters how can i do it?
Thank you
--
f.
"E' in un giorno di pioggia che ti ho conosciuta
e il vento dell'ovest rideva gentile..."
(Modena City Ramblers)
http://fox.noblogs.org/
12 years, 9 months
Licence
by Goyder, Matthew
Hello,
If we were to significantly modify Galaxy (remove the logo / change colors / write new functionality within the package) must we then make all of the source code available publically?
Thanks,
Matt Goyder
--
matthew.goyder(a)nationwidechildrens.org
Battelle Center for Mathematical Medicine
The Research Institute at Nationwide Children's Hospital
Office: (614)355-2395
12 years, 9 months
