galaxy-dev March 2011

galaxy-dev@lists.galaxyproject.org

71 participants
111 discussions

substitute or replace a value?
by David Hoover 26 Apr '11

26 Apr '11

Is there a tool for doing a global or column-based substitution in a text or tabular file? For example, replace all instances of the number 2 in a column to a number 1? David Hoover Helix Systems Staff http://helix.nih.gov

3 2

DB access and passowrd field
by Bossers, Alex 25 Apr '11

25 Apr '11

Hi All, I have some tools that generate some tabular output. The next step is to aggregate the data for some filtering and analysis. In my view the best way to do by using a DB (MySQL in my case). Tools and cmd line wrappers can easily connect to this DB (another database than galaxy is using of course) when providing the details for access like address:port, username and password. Is there any way to embed a text field for a galaxy tool that is of type password or blinded? Don't want to share user specific login details when sharing histories or galaxy pages.Or is this a bad thing to pursue anyway? Thanks for any directions. Alex

2 1

Output file naming / dataset's label
by Tanguy Le Carrour 14 Apr '11

14 Apr '11

Dear Galaxy Team, I started using Galaxy few months ago. I'm now setting up my first "real" workflows. I would like to keep the name of my input file through the various steps as dataset's label, maybe also modify it a bit, and finally give a "proper" name to the result file downloaded. A simple example: 1) date-sample_name.gff3 --[step1]-> date-sample_name-step1.tsv 2) date-sample_name-step1.tsv --[step2]-> date-sample_name-step1-step2.txt 3 download date-sample_name-step1-step2.txt I've seen that I can use the 'label' attribute in the tool config "data" tag to do so. But I don't know how to access the ${input.label} (?) and how to modify it. I doesn't stop me using my workflows, but it would definitely help my users not to get lost when they use them. Best regards, Tanguy Le Carrour -- Tanguy LE CARROUR Functional Genomics Center Zurich ETH Zurich / University of Zurich Winterthurerstrasse 190 8057 Zurich Switzerland

4 8

2011 Galaxy Community Conference, May 25-26, Lunteren, The Netherlands
by Dave Clements 11 Apr '11

11 Apr '11

Hello all, We are pleased to announce the *2011 Galaxy Community Conference*, being held *May 25-26 in Lunteren, The Netherlands*. The meeting will feature two full days of presentations and discussion on extending Galaxy to use new tools and data sources, deploying Galaxy at your organization, and best practices for using Galaxy to further your own and your community's research. *Link: http://galaxy.psu.edu/gcc2011/ * *Overview *This event aims to engage a broader community of developers, data producers, tool creators, and core facility and other research hub staff to become an active part of the Galaxy community. We'll cover defining resources in the Galaxy framework, increasing their visibility and making them easier to use and integrate with other resources, how to extend Galaxy to use custom data sources and custom tools, and best practices for using Galaxy in your organization. Additional topics include, but are not limited to: * Talks submitted by the Galaxy community * Integration of tools (including NGS analysis tools) and distributed job management * Deployment of Galaxy instances on local resources and on the Cloud * Management of large datasets with the Galaxy Library System * Using the Galaxy LIMS functionality at NGS sequencing facilities * Visualizing Data without leaving Galaxy * Performing reproducible research * Performing and sharing complex analyses with Workflows * An "Introduction to Galaxy" session, offered on May 24, for Galaxy newcomers. *Registration *The conference fee is €100 on or before April 24, and €120 after that. The meeting is being held at the Conference Centre De Werelt in Lunteren, The Netherlands, which is also the conference hotel. You are encouraged to register early, as space at the hotel (and at the "Intro to Galaxy" session) is limited and is likely to fill up before the conference itself does. *Link: http://galaxy.psu.edu/gcc2011/Register.html ** Abstract Submission *Abstracts are now being accepted for short oral presentations. Proposals on any topic of interest to the Galaxy community are welcome and encouraged. The abstract submission deadline is the end of February 28. *Link: http://galaxy.psu.edu/gcc2011/Abstracts.html * *Sponsors *The 2011 Galaxy Community Conference is co-sponsored by the US National Science Foundation (NSF), and the Netherlands Bioinformatics Centre (NBIC). NBIC is a collaborative institute of the bioinformatics groups in the Netherlands. Together, these groups perform cutting-edge research, develop novel tools and support platforms, create an e-science infrastructure and educate the next generations of bioinformaticians. *Links: http://www.nbic.nl/ and http://www.nsf.gov/ * We are looking forward to a great conference and hope to see you in the Netherlands! The Galaxy and NBIC Teams -- http://galaxy.psu.edu/gcc2011/ http://getgalaxy.org http://usegalaxy.org/

2 2

Missing Page Editor Components
by Charlis Hsia 05 Apr '11

05 Apr '11

Hi all, Wonder if I can get some help from here for the following questions: I just pulled the galaxy-central and installed it. It runs fine. However, when I tried to use the Edit Pages function, there is no "Embed Galaxy Object" and "Insert Link to Galaxy Object" in the "Page Editor". I have set the "enable_pages = True" in "universe_wsgi.ini". Wonder what else can cause the problem? Also, is there any documentations for the control syntax used in datatypes and tools configure xmls? Thanks! Charlie

3 3

Updating BLAST+ wrapper
by Peter Cock 05 Apr '11

05 Apr '11

Hi all, The NCBI have just released BLAST 2.2.25+ which includes some interesting new stuff of interest to the tabular output, i.e. "Added support for query and subject length to tabular output" I would therefore like to update my BLAST+ wrappers in Galaxy to add these two columns to the 'extended tabular' output option. They are going to be very helpful as you can now calculate things like the percentage identity (or similarity) compared to the query or sequence. If there are no objections I propose to add these as columns 23 and 24 (i.e. at the end), so minimise disruption to anyone already using the tabular output. I would hope to have a branch ready for merging next week... Peter

2 14

Alternative Bowtie/TopHat tools
by Assaf Gordon 04 Apr '11

04 Apr '11

Still very much work-in-progress, but I'd like to hear any comments or ideas that you might have. No need to install, just visit here to see TopHat's interface: http://cancan.cshl.edu/publicgalaxy/root?tool_id=cshl_tophat_se1 Main differences from the existing TopHat tool: 1. Far fewer visible options 2. Select "Existing gene mode" (use GTF file) or do De-novo junction detection (or both) 3. Allow insertions/deletions detection (with the latest tophat). 4. Allow user to specify custom command line parameters. 5. Accepts Fasta, FastqSanger, FastqIllumina (but not generic "fastq", per your feedback). If you think there are other critical and often used options, let me know. updated code is available here: http://cancan.cshl.edu/labmembers/gordon/files/cshl_rnaseq_tools_1.tar.bz2 -gordon

1 1

Server error from history view
by Sarah Diehl 04 Apr '11

04 Apr '11

Hi all, I encounter the same error that was already mentioned in this list (with the same email title). For me it also does only happen when debug is set to false. However I could not find a solution to it in the archive of the mailing list. I tried it with both galaxy-dist and galaxy-central, newest version. The error is as follows: 10.1.5.190 - - [10/Mar/2011:11:41:20 +0200] "GET /history HTTP/1.1" 500 - "http://solweb2/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.14) Gecko/20110301 Fedora/3.6.14-1.fc14 Firefox/3.6.14" Error - <type 'exceptions.TypeError'>: 'NoneType' object is not callable URL: http://solweb2/history File '/usr/lib/python2.6/site-packages/paste/exceptions/errormiddleware.py', line 144 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/lib/python2.6/site-packages/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/usr/lib/python2.6/site-packages/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/galaxy/galaxy_server/lib/galaxy/web/framework/base.py', line 154 in __call__ return body( environ, start_response ) File '/galaxy/galaxy_server/lib/galaxy/web/framework/__init__.py', line 676 in render template.render_context( context ) File '/galaxy/galaxy_server/eggs/Mako-0.2.5-py2.6.egg/mako/template.py', line 146 in render_context runtime._render_context(self, self.callable_, context, *args, **kwargs) File '/galaxy/galaxy_server/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 381 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/galaxy/galaxy_server/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 414 in _exec_template callable_(context, *args, **kwargs) File '/galaxy/galaxy_server/database/compiled_templates/root/history.mako.py', line 55 in render_body __M_writer(u'\n<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">\n\n<html>\n\n<head>\n<title>') File '/galaxy/galaxy_server/lib/galaxy/web/framework/__init__.py', line 673 in write response_write( d.encode( 'utf-8' ) ) TypeError: 'NoneType' object is not callable CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_CHARSET: 'ISO-8859-1,utf-8;q=0.7,*;q=0.7' HTTP_ACCEPT_ENCODING: 'gzip,deflate' HTTP_ACCEPT_LANGUAGE: 'en-us,en;q=0.5' HTTP_CONNECTION: 'Keep-Alive' HTTP_COOKIE: 'galaxysession=f611411aa8a0c665eb9e71b6b35ec98d69a19a7826eff47d201855aea491539e3c45f483349e57b4' HTTP_HOST: 'solweb2' HTTP_REFERER: 'http://solweb2/' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.14) Gecko/20110301 Fedora/3.6.14-1.fc14 Firefox/3.6.14' HTTP_X_FORWARDED_FOR: '10.1.5.190' HTTP_X_FORWARDED_HOST: 'solweb2' HTTP_X_FORWARDED_SERVER: 'solweb2' ORGINAL_HTTP_HOST: 'localhost:8080' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/history' REMOTE_ADDR: '10.1.5.190' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1' Configuration ------------- __file__: '/galaxy/galaxy_server/universe_wsgi.ini' admin_users: 'diehl@immunbio.mpg.de,lims_mq@solweb1.immunbio.mpg.de' allow_library_path_paste: 'True' allow_user_creation: 'True' allow_user_deletion: 'False' cookie_path: '/' database_connection: 'postgres://galaxy:galaxy@localhost/galaxy' database_engine_option_server_side_cursors: 'True' database_engine_option_strategy: 'threadlocal' datatypes_config_file: 'datatypes_conf.xml' debug: 'False' enable_api: 'True' enable_sequencer_communication: 'True' file_path: 'database/files' gbrowse_display_sites: 'wormbase,tair,modencode_worm,modencode_fly' genetrack_display_sites: 'main,test' here: '/galaxy/galaxy_server' id_secret: 'MPI IMMUNBIOLOGIE' library_import_dir: '/galaxy/galaxy_server/data' log_actions: 'True' log_events: 'True' log_level: 'DEBUG' new_file_path: 'database/tmp' new_user_dataset_access_role_default_private: 'True' require_login: 'False' static_cache_time: '360' static_dir: '/galaxy/galaxy_server/static/' static_enabled: 'True' static_favicon_dir: '/galaxy/galaxy_server/static/favicon.ico' static_images_dir: '/galaxy/galaxy_server/static/images' static_scripts_dir: '/galaxy/galaxy_server/static/scripts/' static_style_dir: '/galaxy/galaxy_server/static/june_2007_style/blue' tool_config_file: 'tool_conf.xml' tool_data_path: 'tool-data' tool_path: 'tools' ucsc_display_sites: 'main,test,archaea,ucla' use_interactive: 'False' use_lint: 'False' use_profile: 'False' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x63ab690> paste.cookies: (<SimpleCookie: galaxysession='f611411aa8a0c665eb9e71b6b35ec98d69a19a7826eff47d201855aea491539e3c45f483349e57b4'>, 'galaxysession=f611411aa8a0c665eb9e71b6b35ec98d69a19a7826eff47d201855aea491539e3c45f483349e57b4') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x692eb50> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x263ad90> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True webob._parsed_query_vars: (MultiDict([]), '') wsgi process: 'Multithreaded' ------------------------------------------------------------ Any help would be greatly appreciated. Best regards, Sarah

2 1

Around Pittsburgh on April 6? Attend the Intro to Galaxy Sessions @ Pitt
by Dave Clements 01 Apr '11

01 Apr '11

Hello all, Dan Blankenberg will be giving two workshops on Galaxy at the University of Pittsburgh on April 6. The presentations are open to the public. See below for details and please contact Dan, or Carrie Iwema at Pitt, if you have any questions. Thanks, Dave C. Intro to Galaxy http://galaxy.psu.edu/ Dan Blankenberg, PhD Center for Comparative Genomics & Bioinformatics Penn State University Galaxy allows you to do analyses you cannot do anywhere else without the need to install or download anything. You can analyze multiple alignments, compare genomic annotations, profile metagenomic samples & more... Wednesday 6th April *10 am – 12 pm Intro to Galaxy (general interest) * * * *2 pm - 4 pm Working w/NGS Data (advanced users) * University of Pittsburgh Falk Library Conference Room B You are welcome to bring your laptop. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Carrie L. Iwema, PhD, MLS Information Specialist in Molecular Biology Health Sciences Library System University of Pittsburgh 200 Scaife Hall 3550 Terrace St Pittsburgh, PA 15261 412-383-6887412-648-8819 (fax)iwema(a)pitt.eduwww.hsls.pitt.edu/molbio ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- http://galaxy.psu.edu/gcc2011/ http://getgalaxy.org http://usegalaxy.org/

5 5

Bitten by a Cheetah (long)
by Assaf Gordon 31 Mar '11

31 Mar '11

Hello all, After 7 enjoyable hours of cheetah debugging ... for those (like me) who are not well versed in python or cheetah templates, here are couple of debugging tips that can help. =1= RTFM. It's here: http://www.cheetahtemplate.org/docs/users_guide_html/users_guide.html =2= printing something to STDERR, to show the progress of the cheetah template compilation To run python one-liners in your cheetech template, use "#silent". Assuming this is my Galaxy XML tool: ============= <tool id="cshl_bowtie_se1" name="Bowtie" version="0.1.3" > <command interpreter="sh">bowtie_se1.sh "$output" "$input1" #if str($report_aligned_fastq) == "True" "$output_aligned_fastq_2" #else "no-al" #end if #if str($report_nonmappers_fastq) == "True" "$output_nonmappers_fastq_2" #else "no-un" #end if #if str($report_maxmappers_fastq) == "True" "$output_maxmappers_fastq_1" #else "no-max" #end if #if $mismatches_strategy.choice == "endtoend" -v $mismatches_strategy.end_to_end_limit #else if $mismatches_strategy.choice == "seed" --seedlen $mismatches_strategy.seedlen --seedmms $mismatches_strategy.seedmms $mismatches_strategy.best $mismatches_strategy.strata #end if #if $multi_mappers_strategy.choice == "m_and_k" -k $multi_mappers_strategy.max_mapping -m $multi_mappers_strategy.max_mapping #else if $multi_mappers_strategy.choice == "unique" -k 1 -m 1 #else if $multi_mappers_strategy.choice == "all" -a #else if $multi_mappers_strategy.choice == "random_one": -k 1 #end if #if $input1.ext == "fastqillumina": --solexa1.3-quals #else if $input1.ext == "fasta": -f #else if $input1.ext == "fastqsolexa": --solexa-qual #end if [ and so on and on and on ...] ============= Then to do something directly in python, just put "#silent" followed a python command. The most useful command is to print something to STDERR, so it will appear on the screen when you execute the tool. The "#silent" means it will not be part of the generated command line. Example: ========== ... ... <command interpreter="sh">bowtie_se1.sh "$output" "$input1" #silent sys.stderr.write("Hello World from cheetah generated code\n"); #if str($report_aligned_fastq) == "True" "$output_aligned_fastq_2" #else "no-al" #end if ... ... =========== When the tool is executed (from inside Galaxy), and the job is prepared, and the command line is built, the cheetah template will be filled, and the message will be printed to STDERR. =3= Printing the value of a galaxy parameter Same "#silent" as before, but add the variable as a format parameter: === #silent sys.stderr.write(" Execute by user: %s\n" % ( $userEmail ) ) === NOTE: there are several levels of parameters interpolation going on here. "$userEmail" is a cheetah variable (obviously, because "$" doesn't mean anything in python). It is first replaced by cheetah, then executed by python, getting this to work is sometimes tricky. If you have some elabrate conditional/repeat/group parameters, it could look like this: ======= #silent sys.stderr.write(" mismatch strategy parameters: %s\n" % ( str( $mismatch_strategy ) ) ) ======= And the output would look like this (on the terminal showing Galaxy's log output): ========= mismatches_strategy = {'seedmms': <galaxy.tools.InputValueWrapper object at 0x203adfd0>, 'strata': <galaxy.tools.InputValueWrapper object at 0x203ad150>, 'choice': 'seed', 'seedlen': <galaxy.tools.InputValueWrapper object at 0x203adc90>, '__current_case__': 1, 'best': <galaxy.tools.InputValueWrapper object at 0x203ad590>} ========= This assumes that you have a valid XML <param> named "mismatch_strategry", which in my case was defined as: ====== <conditional name="mismatches_strategy" > <param name="choice" type="select" label="Mismatches strategy"> <option value="seed">Seed/Extent</option> <option value="endtoend">End-to-End mismatches limit (-v)</option> <option value="custom">Use custom options (below)</option> </param> <when value="endtoend"> <param name="end_to_end_limit" type="integer" size="2" value="0" label="Max. number of mismatches allowed in the entire read (-v)" help="" /> </when> <when value="seed"> <param name="seedlen" type="integer" size="4" value="28" label="Seed length (--seendlen)" help="Length of the seed fragment in the read. 28 is a good choice for smallRNAs. Use larger values for long reads of DNA/RNA." /> <param name="seedmms" type="integer" size="4" value="3" label="Allows mismatches in the seed (--seedmms)" help="How many mismatches are allowed in the seed fragment of the read. Enter values between 0 and 3." /> <param name="best" type="boolean" truevalue="--best" falsevalue="" checked="True" label="Report best stratum (--best)" help="hits guaranteed best stratum; ties broken by quality" /> <param name="strata" type="boolean" truevalue="--strata" falsevalue="" checked="false" label="Supress hits in sub-optimal strata (--strata)" help="hits in sub-optimal strata aren't reported (requires --best, -m, -k"/> </when> <when value="custom"> </when> </conditional> ====== =4= Show all available parameters in the cheetah template The safe way that I've found is: ======== #silent sys.stderr.write("!!!! Cheetah Template Variables !!!!\n") #silent sys.stderr.write(" searchList = '%s'\n" % (str($searchList)) ) #silent sys.stderr.write("!!!! end-of-list !!!!\n") ======== "$searchList" is the cheetah variable containing all the templated variables. The output will be ugly, but should always work. example: ============== !!!! Cheetah Template Variables !!!! searchList = '[{}, <cheetah_DynamicallyCompiledCheetahTemplate_1301603121_44_70792.DynamicallyCompiledCheetahTemplate object at 0x6b70c50>, {'input2': <galaxy.tools.DatasetFilenameWrapper object at 0x6b76fd0>, 'userEmail': 'gordon(a)cshl.edu', 'input1': <galaxy.tools.DatasetFilenameWrapper object at 0x69902d0>, 'user_custom_options': <galaxy.tools.InputValueWrapper object at 0x6b768d0>, 'userId': '2', 'output_maxmappers_fastq_1': <galaxy.tools.DatasetFilenameWrapper object at 0x2aaaacd33b90>, 'dbkey': 'mm9', 'output_maxmappers_fastq_2': <galaxy.tools.DatasetFilenameWrapper object at 0x62e5890>, 'multi_mappers_strategy': {'__current_case__': 2, 'choice': 'unique'}, 'GALAXY_DATA_INDEX_DIR': '/localdata1/gordon/projects/galaxy_dev/tool-data', 'GALAXY_DATATYPES_CONF_FILE': '/localdata1/gordon/projects/galaxy_dev/datatypes_conf.xml', 'min_insertion': <galaxy.tools.InputValueWrapper object at 0x62e1850>, 'report_nonmappers_fastq': <galaxy.tools.InputValueWrapper object at 0x6b76490> , 'mismatches_strategy': {'seedmms': <galaxy.tools.InputValueWrapper object at 0x6b76710>, 'strata': <galaxy.tools.InputValueWrapper object at 0x6b764d0>, 'choice': 'seed', 'seedlen': <galaxy.tools.InputValueWrapper object at 0x6b76d90>, '__current_case__': 1, 'best': <galaxy.tools.InputValueWrapper object at 0x6b76d10>}, 'output_nonmappers_fastq_1': <galaxy.tools.DatasetFilenameWrapper object at 0x2aaaacd33b10>, 'report_aligned_fastq': <galaxy.tools.InputValueWrapper object at 0x6b76a10>, '__new_file_path__': '/localdata1/gordon/projects/galaxy_dev/database/tmp', 'output_nonmappers_fastq_2': <galaxy.tools.DatasetFilenameWrapper object at 0x2aaaacd33610>, 'database': {'source': 'fromlist', 'databasepath': <galaxy.tools.SelectToolParameterWrapper object at 0x62e1c10>, '__current_case__': 1}, 'report_maxmappers_fastq': <galaxy.tools.InputValueWrapper object at 0x6b76790>, 'max_insertion': <galaxy.tools.InputValueWrapper object at 0x62e1450>, 'suppress_non_mappers': <galaxy.tool s.InputValueWrapper object at 0x6b76410>, 'output_aligned_fastq_2': <galaxy.tools.DatasetFilenameWrapper object at 0x6b6fa10>, 'output_aligned_fastq_1': <galaxy.tools.DatasetFilenameWrapper object at 0x6ab15d0>, 'GALAXY_ROOT_DIR': '/localdata1/gordon/projects/galaxy_dev', 'output': <galaxy.tools.DatasetFilenameWrapper object at 0x6b76a90>, 'chromInfo': '/localdata1/gordon/projects/galaxy_dev/tool-data/shared/ucsc/chrom/mm9.len', '__app__': <galaxy.tools.RawObjectWrapper object at 0x699c1d0>}]' !!!! end-of-list !!!! ============== A better way (but I'm not sure it's generic enough, or if it's specific to Galaxy, python gurus please chime in): ===== #silent sys.stderr.write("!!!! Cheetah Template Variables !!!!\n") #for k,v in $searchList[2].items() #silent sys.stderr.write(" %s = %s\n" % (str(k), str(v) )) #end for #silent sys.stderr.write("!!!! end-of-list !!!!\n") ===== The output in this case will be: ======== !!!! Cheetah Template Variables !!!! input2 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_251.dat userEmail = gordon(a)cshl.edu input1 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_183.dat user_custom_options = userId = 2 output_maxmappers_fastq_1 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_533.dat dbkey = mm9 output_maxmappers_fastq_2 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_534.dat multi_mappers_strategy = {'__current_case__': 2, 'choice': 'unique'} GALAXY_DATA_INDEX_DIR = /localdata1/gordon/projects/galaxy_dev/tool-data GALAXY_DATATYPES_CONF_FILE = /localdata1/gordon/projects/galaxy_dev/datatypes_conf.xml min_insertion = 0 report_nonmappers_fastq = True mismatches_strategy = {'seedmms': <galaxy.tools.InputValueWrapper object at 0x6a95b10>, 'strata': <galaxy.tools.InputValueWrapper object at 0x6a95e90>, 'choice': 'seed', 'seedlen': <galaxy.tools.InputValueWrapper object at 0x6a95390>, '__current_case__': 1, 'best': <galaxy.tools.InputValueWrapper object at 0x6a95210>} output_nonmappers_fastq_1 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_531.dat report_aligned_fastq = True __new_file_path__ = /localdata1/gordon/projects/galaxy_dev/database/tmp output_nonmappers_fastq_2 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_532.dat database = {'source': 'fromlist', 'databasepath': <galaxy.tools.SelectToolParameterWrapper object at 0x6a95350>, '__current_case__': 1} report_maxmappers_fastq = True max_insertion = 250 suppress_non_mappers = --quiet output_aligned_fastq_2 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_530.dat output_aligned_fastq_1 = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_529.dat GALAXY_ROOT_DIR = /localdata1/gordon/projects/galaxy_dev output = /localdata1/gordon/projects/galaxy_dev/database/files/000/dataset_528.dat chromInfo = /localdata1/gordon/projects/galaxy_dev/tool-data/shared/ucsc/chrom/mm9.len __app__ = galaxy.app:UniverseApplication !!!! end-of-list !!!! ======== Who knew one can access "GALAXY_ROOT_DIR" as a tool parameter ? (execpt for the galaxy developers, of course... :) ) =5= Divide-and-conquer a long template If you have strange cheetah compilation problems and can't understand what's wrong (sorry, python/cheetah code makes no sense to me...), use "#breakpoint" to stop the compilation at any given point in the template. If the cheetah template compiled without error, then all the statements above your "#breakpoint" are valid, and you can move the "#breakpoint" to a further point in the template until you pin-point the offending statement. The tool will obviously not run successfully without the entire command line, but we're debugging the template here, not the tool. =6= The hazards of an "$input" parameter With most other parameters, a typo in a variable name will produce a helpful cheetah error message. Exmaple: ========= <tool id="cshl_dummy" name="dummy" description="" > <command interpreter="sh">echo $bar</command> <inputs> <param type="integer" name="foo" size="2" value="42"/> </inputs> <outputs> <data format="txt" name="output" /> </outputs> </tool> ========= Since there's no "$bar" parameter (the name is "$foo"), the cheetah error will be: ======== Traceback (most recent call last): File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/jobs/runners/drmaa.py", line 127, in queue_job job_wrapper.prepare() File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/jobs/__init__.py", line 371, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/tools/__init__.py", line 1527, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/localdata1/gordon/projects/galaxy_dev/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1301604373_52_56153.py", line 83, in respond NotFound: cannot find 'bar' ======== Helpful enough, even without a line number or code fragment. But if your typo is "$input" (and your parameters are named "$input1" or "$input2" or similar), the error will completely useless. Example: The following bug (using "$input" when there's no "$input" parameter defined): ========== <tool id="cshl_dummy" name="dummy" description="" > <command interpreter="sh">echo $input</command> <inputs> <param type="integer" name="foo" size="2" value="42"/> </inputs> <outputs> <data format="txt" name="output" /> </outputs> </tool> =========== Will throw the following exception: =========== Traceback (most recent call last): File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/jobs/runners/drmaa.py", line 127, in queue_job job_wrapper.prepare() File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/jobs/__init__.py", line 371, in prepare self.command_line = self.tool.build_command_line( param_dict ) File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/tools/__init__.py", line 1527, in build_command_line command_line = fill_template( self.command, context=param_dict ) File "/localdata1/gordon/projects/galaxy_dev/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/localdata1/gordon/projects/galaxy_dev/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1301604408_95_71749.py", line 83, in respond File "<string>", line 0 ^ SyntaxError: unexpected EOF while parsing ============= Probably because there is a hidden "input" parameter in galaxy code that is used in the output format detection code, but I haven't found the exact spot. This 'bug' alone took me two hours to solve :( Hope this will help to others in future debuggings... -gordon

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev March 2011