run_functional_tests.sh -sid option
by Peter
Hi,
I'm trying to use run_functional_tests.sh to run all the tests
for a section (group of tools). I've read the output from:
./run_functional_tests.sh help
For example, from the tools_conf.xml.sample we have
<section name="ENCODE Tools" id="EncodeTools">
<tool file="encode/gencode_partition.xml" />
<tool file="encode/random_intervals.xml" />
</section>
And looking at these tools' XML files,
<tool id="gencode_partition1" name="Gencode Partition">
and:
<tool id="random_intervals1" name="Random Intervals">
I'd like to run the functional tests for the ENCODE tools,
Using the switch for an individual tool id (-id) works,
./run_functional_tests.sh -id gencode_partition1
...
Ran 1 test in 22.302s
...
and so does this (well, it say the tool doesn't have any
tests which is true in this example):
./run_functional_tests.sh -id random_intervals1
...
Ran 1 test in 0.027s
...
However, I also tried using the section id switch (-sid),
./run_functional_tests.sh -sid EncodeTools
...
Ran 0 tests in 0.000s
...
I also tried using the section name,
/run_functional_tests.sh -sid "ENCODE Tools"
Is this (-sid not working) a known issue, or am I using it wrong?
Thanks,
Peter
8 years, 11 months
Sharing tool definitions between XML files
by Peter
Hi all,
I'm wondering if there is any established way to share parameter
definitions between tool wrapper XML files?
For example, I am currently working on NCBI BLAST+ wrappers, and these
tools have a lot in common. For example, the output format option will
be a select parameter, and it will be the same for blastn, blastp,
tblastn, etc. I can just cut and paste the definition but this seems
inelegant and will be a long term maintenance burden if it ever needs
updating (lots of places would need to be updated the same way). I'd
like to have a shared XML snippet defining this parameter which I
could then reference/include from each tool wrapper than needs it.
I'm thinking of something like XML's <!ENTITY ...> might work. Is this possible?
Peter
10 years, 7 months
Output file naming / dataset's label
by Tanguy Le Carrour
Dear Galaxy Team,
I started using Galaxy few months ago. I'm now setting up my first
"real" workflows.
I would like to keep the name of my input file through the various steps
as dataset's label, maybe also modify it a bit, and finally give a
"proper" name to the result file downloaded.
A simple example:
1) date-sample_name.gff3 --[step1]-> date-sample_name-step1.tsv
2) date-sample_name-step1.tsv --[step2]->
date-sample_name-step1-step2.txt
3 download date-sample_name-step1-step2.txt
I've seen that I can use the 'label' attribute in the tool config "data"
tag to do so. But I don't know how to access the ${input.label} (?) and
how to modify it.
I doesn't stop me using my workflows, but it would definitely help my
users not to get lost when they use them.
Best regards,
Tanguy Le Carrour
--
Tanguy LE CARROUR
Functional Genomics Center Zurich
ETH Zurich / University of Zurich
Winterthurerstrasse 190
8057 Zurich
Switzerland
10 years, 9 months
swap issue
by Isabelle Phan
Hello,
I am running galaxy, using a postgres backend.
The galaxy server systematically gets killed whenever I try uploading a bam file. The samtools process is still running.
This is the end of the log:
2010-08-26 18:08:11,371 INFO sqlalchemy.engine.threadlocal.TLEngine.0x...4610 {'param_1': 8}
sqlalchemy.engine.threadlocal.TLEngine.0x...4610 INFO 2010-08-26 18:08:11,371 {'param_1': 8}
2010-08-26 18:08:11,378 INFO sqlalchemy.pool.QueuePool.0x...4ed0 Connection <connection object at 0x567bf30; dsn: 'dbname=galaxy host=/var closed: 0> being returned to pool
sqlalchemy.pool.QueuePool.0x...4ed0 INFO 2010-08-26 18:08:11,378 Connection <connection object at 0x567bf30; dsn: 'dbname=galaxy host=/var closed: 0> being returned to pool
10.1.20.44 - - [26/Aug/2010:18:08:06 -0700] "POST /root/history_item_updates HTTP/1.1" 200 - "http://galaxy:8080/history" "Mozilla/5.0 (X1 (x86_64); en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"
Killed
I have the following options set in universe_wsgi.ini:
database_engine_option_server_side_cursors = True
database_engine_option_strategy = threadlocal
It seems to be running out of swap. I monitored memory usage with 'top' and galaxy died when available swap hit 0k free.
I am not superuser on the machine where galaxy is running and I have no other server available. Is there something I could do to prevent galaxy from dying?
Thanks,
Isabelle
--
Isabelle Phan, DPhil
Seattle Biomedical Research Institute
+1(206)256 7113
10 years, 10 months
Problems with run_functional_tests.sh and history?
by Peter
Hi all,
I'm having trouble with the Galaxy functional tests. On our Linux server
things seem to work, but I'm having trouble on my Mac. This affects
multiple tests.
For example, using a test for my own tool wrapper, on Linux my test
passes. On my Mac, the tool is missing the required binary so I
expect the test to fail with error level 127 and a message to stderr.
However, I get the following cryptic error:
$ ./run_functional_tests.sh -id tmhmm2
...
functional_tests.py INFO 2010-10-28 11:57:01,129 Embedded web server started
functional_tests.py INFO 2010-10-28 11:57:01,185 Functional tests will
be run against localhost:8980
nose.plugins.manager DEBUG 2010-10-28 11:57:01,210
DefaultPluginManager load plugin sqlalchemy =
sqlalchemy.test.noseplugin:NoseSQLAlchemy
nose.plugins.manager DEBUG 2010-10-28 11:57:01,215
DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML
TMHMM 2.0 ( tmhmm2 ) > Test-1 ... Error - <type
'exceptions.TypeError'>: 'NoneType' object is not callable
URL: http://localhost:8980/history
File '/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Paste-1.7.2-py2.5.egg/paste/exceptions/errormiddleware.py',
line 144 in __call__
app_iter = self.application(environ, sr_checker)
File '/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Paste-1.7.2-py2.5.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File '/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Paste-1.7.2-py2.5.egg/paste/httpexceptions.py',
line 636 in __call__
return self.application(environ, start_response)
File '/Users/xxx/repositories/galaxy-central/lib/galaxy/web/framework/base.py',
line 154 in __call__
return body( environ, start_response )
File '/Users/xxx/repositories/galaxy-central/lib/galaxy/web/framework/__init__.py',
line 668 in render
template.render_context( context )
File '/Users/xxx/repositories/galaxy-central/eggs/Mako-0.2.5-py2.5.egg/mako/template.py',
line 146 in render_context
runtime._render_context(self, self.callable_, context, *args, **kwargs)
File '/Users/xxx/repositories/galaxy-central/eggs/Mako-0.2.5-py2.5.egg/mako/runtime.py',
line 381 in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File '/Users/xxx/repositories/galaxy-central/eggs/Mako-0.2.5-py2.5.egg/mako/runtime.py',
line 414 in _exec_template
callable_(context, *args, **kwargs)
File '/Users/xxx/repositories/galaxy-central/database/compiled_templates/root/history.mako.py',
line 55 in render_body
__M_writer(u'\n<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01
Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">\n\n<html>\n\n<head>\n<title>')
File '/Users/xxx/repositories/galaxy-central/lib/galaxy/web/framework/__init__.py',
line 665 in write
response_write( d.encode( 'utf-8' ) )
TypeError: 'NoneType' object is not callable
CGI Variables
-------------
CONTENT_LENGTH: '0'
HTTP_ACCEPT: 'text/html; */*'
HTTP_ACCEPT_ENCODING: 'identity'
HTTP_CONNECTION: 'close'
HTTP_COOKIE: '$Version=1;
galaxysession=eb142648ac45b770a711a07bbd8189c4a80d9868f9ceff96f0635391728a47dfe5161bca180a2aae;
$Path="/"'
HTTP_HOST: 'localhost:8980'
PATH_INFO: '/history'
REMOTE_ADDR: '127.0.0.1'
REQUEST_METHOD: 'GET'
SERVER_NAME: '127.0.0.1'
SERVER_PORT: '8980'
SERVER_PROTOCOL: 'HTTP/1.1'
Configuration
-------------
static_enabled: False
use_translogger: False
WSGI Variables
--------------
application: <paste.recursive.RecursiveMiddleware object at 0xbfd2e10>
paste.cookies: (<SimpleCookie:
galaxysession='eb142648ac45b770a711a07bbd8189c4a80d9868f9ceff96f0635391728a47dfe5161bca180a2aae'>,
'$Version=1; galaxysession=eb142648ac45b770a711a07bbd8189c4a80d9868f9ceff96f0635391728a47dfe5161bca180a2aae;
$Path="/"')
paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>]
paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler
object at 0xbfd3110>
paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at
0xccf1f10>
paste.recursive.forward: <paste.recursive.Forwarder from />
paste.recursive.include: <paste.recursive.Includer from />
paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from />
paste.recursive.script_name: ''
paste.throw_errors: True
webob._parsed_query_vars: (MultiDict([]), '')
wsgi process: 'Multithreaded'
------------------------------------------------------------
ERROR
======================================================================
ERROR: TMHMM 2.0 ( tmhmm2 ) > Test-1
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/xxx/repositories/galaxy-central/test/functional/test_toolbox.py",
line 160, in test_tool
self.do_it( td )
File "/Users/xxx/repositories/galaxy-central/test/functional/test_toolbox.py",
line 24, in do_it
self.new_history()
File "/Users/xxx/repositories/galaxy-central/test/base/twilltestcase.py",
line 291, in new_history
self.check_history_for_string('Your history is empty')
File "/Users/xxx/repositories/galaxy-central/test/base/twilltestcase.py",
line 215, in check_history_for_string
self.visit_page( "history" )
File "/Users/xxx/repositories/galaxy-central/test/base/twilltestcase.py",
line 1040, in visit_page
tc.code( 200 )
File "/Users/xxx/repositories/galaxy-central/eggs/twill-0.9-py2.5.egg/twill/commands.py",
line 133, in code
should_be))
TwillAssertionError: code is 500 != 200
----------------------------------------------------------------------
Ran 1 test in 0.824s
FAILED (errors=1)
functional_tests.py INFO 2010-10-28 11:57:02,079 Shutting down
functional_tests.py INFO 2010-10-28 11:57:02,079 Shutting down
embedded web server
...
###############################################################################
Here is my suggested fix:
diff -r 82743fcbf23c lib/galaxy/web/framework/__init__.py
--- a/lib/galaxy/web/framework/__init__.py Thu Oct 28 11:41:18 2010 +0100
+++ b/lib/galaxy/web/framework/__init__.py Thu Oct 28 11:56:16 2010 +0100
@@ -660,6 +660,9 @@
## return template.render( **data )
def render( environ, start_response ):
response_write = start_response(
self.response.wsgi_status(), self.response.wsgi_headeritems() )
+ if response_write is None:
+ log.error( "Error: this request returned None from
start_response: %s" % self.request.browser_url )
+ return []
class StreamBuffer( object ):
def write( self, d ):
response_write( d.encode( 'utf-8' ) )
###############################################################################
With the patch, I get:
$ ./run_functional_tests.sh -id tmhmm2
...
functional_tests.py INFO 2010-10-28 11:52:37,742 Embedded web server started
functional_tests.py INFO 2010-10-28 11:52:37,800 Functional tests will
be run against localhost:9383
nose.plugins.manager DEBUG 2010-10-28 11:52:37,825
DefaultPluginManager load plugin sqlalchemy =
sqlalchemy.test.noseplugin:NoseSQLAlchemy
nose.plugins.manager DEBUG 2010-10-28 11:52:37,829
DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML
TMHMM 2.0 ( tmhmm2 ) > Test-1 ... galaxy.web.framework ERROR
2010-10-28 11:52:38,685 Error: this request returned None from
start_response: http://localhost:9383/history
ERROR
======================================================================
ERROR: TMHMM 2.0 ( tmhmm2 ) > Test-1
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/xxx/repositories/galaxy-central/test/functional/test_toolbox.py",
line 160, in test_tool
self.do_it( td )
File "/Users/xxx/repositories/galaxy-central/test/functional/test_toolbox.py",
line 24, in do_it
self.new_history()
File "/Users/xxx/repositories/galaxy-central/test/base/twilltestcase.py",
line 291, in new_history
self.check_history_for_string('Your history is empty')
File "/Users/xxx/repositories/galaxy-central/test/base/twilltestcase.py",
line 217, in check_history_for_string
tc.find(subpatt)
File "/Users/xxx/repositories/galaxy-central/eggs/twill-0.9-py2.5.egg/twill/commands.py",
line 239, in find
raise TwillAssertionError("no match to '%s'" % (what,))
TwillAssertionError: no match to 'Your'
-------------------- >> begin captured logging << --------------------
galaxy.web.framework: ERROR: Error: this request returned None from
start_response: http://localhost:9383/history
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 1 test in 0.821s
FAILED (errors=1)
functional_tests.py INFO 2010-10-28 11:52:38,691 Shutting down
functional_tests.py INFO 2010-10-28 11:52:38,691 Shutting down
embedded web server
...
###############################################################################
I'm still puzzled about what is going wrong - this affects other Galaxy tests
as well - not just mine. Is there something obvious wrong with the setup on
this machine? Actually using Galaxy running on this Mac seems to work.
Thanks,
Peter
10 years, 11 months
Display of tabular data in Galaxy
by Peter
Hi all,
I've noticed in the right hand column for tabular data Galaxy shows
the first few lines with a nice column based layout. Looking at the
source, you actually define an HTML table for this.
However, on the main central column, when viewing a tabular dataset
(by clicking on the eye icon from an entry in the right hand column),
the data is shown as raw text. As long as the data in each column is
about the same length, this works OK. However, if you have variable
length fields then the columns do not line up.
It would be nice if the main display showed the data respecting the
columns. Has this been considered? My guess is that doing this was
rejected due to the computational load to reformat it, and the network
load to send the marked up data.
(I did try and search the mailing list archives but didn't find any
discussion of this)
Regards,
Peter
11 years, 1 month
Variabels in label elements
by Bossers, Alex
Hi All,
Does someone know how to get the value of a type="select" param_name displayed in the output file label?
I know how to do it if its just a non-nested param setup-up using ${input1.value_label} where the input1 is the select_name. But this fails if the select is inside a conditional..... From cheetah script I can call these values like IF $tool.cmd=="something":.....
But whatever I try in the <outputs><data> fails (I though it to be ${tool.cmd.value_label} but that complains on cmd not defined.... All possible iteration tried :(
Snippet of a generalised tool.xml below.
Any suggestions or hints where to get documentation on this syntax is welcomed!
Alex
<command>
/opt/tools/$tool.cmd
$cmd_extra
$input_delta
#if $tool.cmd=="show-aligns":
$tool.extra1
$tool.extra2
#end if
> $out_tool
</command>
<inputs>
<conditional name="tool">
<param name="cmd" type="select" value="show-snps" label="Select tool" >
<option value="show-snps">show snps</option>
<option value="show-aligns">show aligns</option>
</param>
<when value="show-aligns">
<param name="extra1" type="text" size="40" value="" label="IdR" help="FastA header ref sequence" />
<param name="extra2" type="text" size="40" value="" label="IdQ" help="FastA header query sequence" />
</when>
<when value="show-snps" />
</conditional>
<param name="input_delta" type="data" format="tabular" label="MUMmer delta file" />
<param name="cmd_extra" type="text" size="40" value="" label="Extra cmd line options" help="see specific cmd line options below for each tool" />
</inputs>
<outputs>
<data name="out_tool" format="text" label="tool ${tool.cmd.value_label}" />
</outputs>
11 years, 2 months
