- galaxy-dev - lists.galaxyproject.org

Converting __oc__ and __cc__ back to curly braces?
by Samuel Lampa 16 Nov '12

16 Nov '12

Hi, I'm creating a galaxy tool [0] for querying external SPARQL (the Semantic Web query language) endpoints to get back data in tabular formats. I run into a problem though, since SPARQL queries contain curly braces, and Galaxy obviously converts these to an internal representation __oc__ and __cc__ (I found the commit for this at [1]) My SPARQL query for testing looks like this: SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 25 Thus, when I send this to my (python) script as a commmand line argument, the curly braces comes in galaxy's internal representation. Thus I was wondering if there is a way to get the braces in their correct format from galaxy, or if I should just go agead and convert them back in my own script? (i.e. will this behaviour be stable enough to rely on?) Best // Samuel [0] See the sparql_import.* files in https://github.com/samuell/galaxy_semwebtools [1] http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-February/001837.html -- Developer at SNIC-UPPMAX www.uppmax.uu.se Developer at Dept of Pharm Biosciences www.farmbio.uu.se Twitter - twitter.com/samuellampa Blog - saml.rilspace.org G+ - gplus.to/saml

2 3

Re: [galaxy-dev] [galaxy-user] new_file_path setting in universe_wsgi.ini
by Nate Coraor 16 Nov '12

16 Nov '12

On Nov 9, 2012, at 9:24 AM, greg wrote: > I see this setting in the universe_wsgi.ini file: > > # Temporary files are stored in this directory. > new_file_path = /scratch/galaxy > > Is this only used by the web server or will job runners pick up this > setting and use it for temp files? > > > I'm also wondering: > > 1. What sort of files go here? > 2. How much space does it need? Hi Greg, I've moved this over to the galaxy-dev list since it concerns a local installation. Jobs will use this directory if passed the value of $__new_file_path__ on the command line. It's up to the tool to decide whether to use this and how it will be used. How much space it needs will depend entirely on what tools you use and what inputs and parameters you use with those tools. --nate > > Thanks, > > Greg > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

2 2

Is there any timeout in Galaxy
by Jennifer Jackson 16 Nov '12

16 Nov '12

Hi Akshay, Please post all technical questions directly to the galaxy-dev mailing list. I'm sure we'll be able to help out shortly, Jen Galaxy team http://wiki.galaxyproject.org/Support#Mailing_Lists Hello, My name is Akshay and I am a computer science graduate student at UGA in Dr. Kissinger's Lab. We have created an extension to galaxy which allowed users to add web services as tools into galaxy, these could be either used as stand-alone or in a workflow. I wanted to know if Galaxy employs any time out strategy. Consider a case where I have a workflow with BLASTP now it takes some time to run blast and then get result form it. When I try that experiment as standalone (i.e., running the run method first and then after several mins getting result using job id works) however when I create a workflow in which one tool being run method of blast followed by getresult, galaxy seems to time out. Would appreciate your feedback on the same. -Akshay Choche

2 1

"This link may not be followed from within Galaxy" using nginx and upload module.
by Marcelo Timmerman 16 Nov '12

16 Nov '12

Any idea would be appreciated. I get a "This link may not be followed from within Galaxy" after following these steps: Get Data -> Upload File -> File Format: Type "lpe" -> select "lped" from autocomplete list. I've compiled latest stable nginx in CentOS 6.3 with upload module http://www.grid.net.ru/nginx/upload.en.html Galaxy being started by galaxy user nginx started by root user (this scenario reports a nginx worker process owned by galaxy) I'm serving Galaxy at a subdirectory as explained in http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy I'm not using External user authentication I suspect this is a problem in nginx.conf upload module directives. Do you know if it's ok to change location /_x_accel_redirect/ { with location /galaxy/_x_accel_redirect/ { ? Besides I couldn't find an _upload_done directory. Do you have an "_upload_done" or "_upload" directory? Where it's located? This is my nginx.conf server { ... more directives location /galaxy { proxy_pass http://galaxy_app; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } location /galaxy/static { alias /home/galaxy/galaxy-dist/static; expires 24h; } ....... # more directives location /galaxy/_x_accel_redirect/ { internal; alias /; } location /galaxy/_upload { upload_store /home/galaxy/galaxy-dist/database/tmp/upload_store; upload_pass_form_field ""; upload_set_form_field "__${upload_field_name}__is_composite" "true"; upload_set_form_field "__${upload_field_name}__keys" "name path"; upload_set_form_field "${upload_field_name}_name" "$upload_file_name"; upload_set_form_field "${upload_field_name}_path" "$upload_tmp_path"; upload_pass_args on; upload_pass /_upload_done; } location /galaxy/_upload_done { set $dst /tool_runner/index; if ($args ~ nginx_redir=([^&]+)) { set $dst $1; } rewrite "" $dst; } } Cheers

2 1

Scatter plot integration in pages
by Joachim Jacob 16 Nov '12

16 Nov '12

Dear devs, How will the scatterplot in the future be integrated in Galaxy, e.g, for saving and displaying a scatterplot in pages? Kind regards Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

3 3

Re: [galaxy-dev] tool_id for tool_shed tools
by Greg Von Kuster 16 Nov '12

16 Nov '12

Similar entries for other tool shed tools that are installed into a Galaxy instance using the Galaxy Admin interface (not the tool migration stage process) are available in the shed_tool_con.xml file (or whatever shed-related files you have defined in your Galaxy config). You can also find tool ids for all of the tools loaded into your Galaxy tool panel by inspecting the single file name integrated_tool_conf.xml. Using this file, you won't have to inspect more than the single file. These files are all described in the tool shed wiki at the following page. http://wiki.galaxyproject.org/GalaxyToolPanel Greg Von Kuster On Nov 16, 2012, at 2:52 AM, Andreas Kuntzagk wrote: > Thanks, that helps in this case. > But I might also need the ids for other toolshed tools to adjust the job runner settings. > > Andreas > > On 15.11.2012 15:23, Greg Von Kuster wrote: >> Hi Andreas, >> >> Assuming you have executed the migration process for the tools so they are now installed into your Galaxy instance from the tool shed, you can check the entries in the migrated_tools_conf.xml file and get the ids. Entries will look something like the following where you can get the id from the <id> tag. >> >> <tool file="toolshed.g2.bx.psu.edu/repos/devteam/picard/e0232cbac965/picard/rgPicardMar…" guid="toolshed.g2. >> bx.psu.edu/repos/devteam/picard/rgPicardMarkDups/1.56.0"> >> <tool_shed>toolshed.g2.bx.psu.edu</tool_shed> >> <repository_name>picard</repository_name> >> <repository_owner>devteam</repository_owner> >> <installed_changeset_revision>e0232cbac965</installed_changeset_revision> >> <id>toolshed.g2.bx.psu.edu/repos/devteam/picard/rgPicardMarkDups/1.56.0</id> >> <version>1.56.0</version> >> </tool> >> >> Greg Von Kuster >> >> >> On Nov 15, 2012, at 9:10 AM, Andreas Kuntzagk wrote: >> >>> Hi, >>> >>> I upgraded to the latest version and again some tools moved to tool_shed. I now need to set job runners for these tools. Is there an easy way to get the tool_ids other then hovering the mouse over the tool name and write down the id? >>> >>> regards, Andreas >>> >>> -- >>> Andreas Kuntzagk >>> >>> SystemAdministrator >>> >>> Berlin Institute for Medical Systems Biology at the >>> Max-Delbrueck-Center for Molecular Medicine >>> Robert-Roessle-Str. 10, 13125 Berlin, Germany >>> >>> http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> > > -- > Andreas Kuntzagk > > SystemAdministrator > > Berlin Institute for Medical Systems Biology at the > Max-Delbrueck-Center for Molecular Medicine > Robert-Roessle-Str. 10, 13125 Berlin, Germany > > http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

1 0

cannot start galaxy after upgade
by Anne Pajon 15 Nov '12

15 Nov '12

Hi, I'm having some problems restarting our galaxy server after upgrading it to the latest version today... Here is the error message: python path is: /opt/local/home/webapp/galaxy/eggs/numpy-1.6.0-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/boto-2.5.2-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/mercurial-2.2.3-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/Fabric-1.4.2-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/ssh-1.7.14-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/Whoosh-0.3.18-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/pycrypto-2.5-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/python_lzo-1.08_2.03_static-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/bx_python-0.7.1_7b95ff194725-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/amqplib-0.6.1-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/pexpect-2.4-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/Babel-0.9.4-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/MarkupSafe-0.12-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/Mako-0.4.1-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/WebHelpers-0.2-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/simplejson-2.1.1-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/wchartype-0.1-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/elementtree-1.2.6_20050316-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/docutils-0.7-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/WebOb-0.8.5-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/Routes-1.12.3-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg, /opt/local/home/webapp/galaxy/eggs/PasteDeploy-1.3.3-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/PasteScript-1.7.3-py2.7.egg, /opt/local/home/webapp/galaxy/eggs/Paste-1.6-py2.7.egg, /opt/local/home/webapp/galaxy/lib, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages/MySQL_python-1.2.3-py2.7-linux-x86_64.egg, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages/termcolor-1.1.0-py2.7.egg, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages/PyYAML-3.10-py2.7-linux-x86_64.egg, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages/py_mysql2pgsql-0.1.3-py2.7.egg, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages/distribute-0.6.10-py2.7.egg, /opt/local/home/webapp/software/python/python2.7.2/lib/python27.zip, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/plat-linux2, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/lib-tk, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/lib-old, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/lib-dynload, /opt/local/home/webapp/software/mercurial/mercurial-2.1/lib/python2.7/site-packages, /opt/local/home/webapp/software/python/python2.7.2/lib/python2.7/site-packages galaxy.tool_shed.tool_shed_registry DEBUG 2012-11-15 14:12:54,841 Loading references to tool sheds from tool_sheds_conf.xml galaxy.tool_shed.tool_shed_registry DEBUG 2012-11-15 14:12:54,844 Loaded reference to tool shed: Galaxy main tool shed galaxy.tool_shed.tool_shed_registry DEBUG 2012-11-15 14:12:54,844 Loaded reference to tool shed: Galaxy test tool shed galaxy.model.migrate.check DEBUG 2012-11-15 14:12:54,884 psycopg2 egg successfully loaded for postgres dialect galaxy.model.migrate.check INFO 2012-11-15 14:12:54,975 At database version 107 galaxy.tool_shed.migrate.check DEBUG 2012-11-15 14:12:54,991 psycopg2 egg successfully loaded for postgres dialect Traceback (most recent call last): File "/opt/local/home/webapp/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 36, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/opt/local/home/webapp/galaxy/lib/galaxy/app.py", line 47, in __init__ verify_tools( self, db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) File "/opt/local/home/webapp/galaxy/lib/galaxy/tool_shed/migrate/check.py", line 56, in verify_tools tool_shed_accessible, missing_tool_configs_dict = check_for_missing_tools( app, tool_panel_configs, latest_tool_migration_script_number ) File "/opt/local/home/webapp/galaxy/lib/galaxy/tool_shed/migrate/common.py", line 46, in check_for_missing_tools tool_dependencies_dict = tool_shed_decode( text ) File "/opt/local/home/webapp/galaxy/lib/galaxy/tool_shed/encoding_util.py", line 15, in tool_shed_decode a, b = value.split( ":" ) ValueError: too many values to unpack Removing PID file paster.pid Any help would be greatly appreciated. Thanks! Cheers, Anne. -- Anne Pajon, Ph.D. - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk | +44 (0)1223 40 4334 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered charity in England and Wales (1089464), Scotland (SC041666) and the Isle of Man (1103) A company limited by guarantee. Registered company in England and Wales (4325234) and the Isle of Man (5713F). Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

2 4

Fix for KeyError in templates/show_params.mako
by Jim Johnson 15 Nov '12

15 Nov '12

A user encountered a KeyError when clicking the "View Details" icon from an older Cuffdiff history item: File '/galaxy/PRODUCTION/database/compiled_templates/show_params.mako.py', line 106 in render_body __M_writer(unicode( inputs_recursive(tool.inputs, params_objects, depth=1) )) File '/galaxy/PRODUCTION/database/compiled_templates/show_params.mako.py', line 33 in inputs_recursive return render_inputs_recursive(context.locals_(__M_locals),input_params,param_values,depth) File '/galaxy/PRODUCTION/database/compiled_templates/show_params.mako.py', line 202 in render_inputs_recursive __M_writer(unicode(input.value_to_display_text(param_values[input.name], trans.app))) KeyError: 'multiread_correct' This can be fixed in templates/show_params.mako by checking if the input.name key exists in the param_values, surrounding the contents of the for loop with: %if input.name in param_values: %endif $ hg diff templates/show_params.mako diff -r 340438c62171 templates/show_params.mako --- a/templates/show_params.mako Wed Nov 14 18:50:20 2012 -0500 +++ b/templates/show_params.mako Thu Nov 15 08:15:47 2012 -0600 @@ -12,23 +12,25 @@ <%def name="inputs_recursive( input_params, param_values, depth=1 )"> %for input_index, input in enumerate( input_params.itervalues() ): - %if input.type == "repeat": - %for i in range( len(param_values[input.name]) ): - ${ inputs_recursive(input.inputs, param_values[input.name][i], depth=depth+1) } - %endfor - %elif input.type == "conditional": - <% current_case = param_values[input.name]['__current_case__'] %> + %if input.name in param_values: + %if input.type == "repeat": + %for i in range( len(param_values[input.name]) ): + ${ inputs_recursive(input.inputs, param_values[input.name][i], depth=depth+1) } + %endfor + %elif input.type == "conditional": + <% current_case = param_values[input.name]['__current_case__'] %> + <tr> + ${ inputs_recursive_indent( text=input.test_param.label,depth=depth )} +  + <td>${input.cases[current_case].value}</td> + </tr> + ${ inputs_recursive(input.cases[current_case].inputs, param_values[input.name], depth=depth+1) } + %elif getattr(input, "label", None): <tr> - ${ inputs_recursive_indent( text=input.test_param.label,depth=depth )} -  - <td>${input.cases[current_case].value}</td> + ${inputs_recursive_indent( text=input.label,depth=depth )} + <td>${input.value_to_display_text(param_values[input.name], trans.app)}</td> </tr> - ${ inputs_recursive(input.cases[current_case].inputs, param_values[input.name], depth=depth+1) } - %elif getattr(input, "label", None): - <tr> - ${inputs_recursive_indent( text=input.label,depth=depth )} - <td>${input.value_to_display_text(param_values[input.name], trans.app)}</td> - </tr> + %endif %endif %endfor </%def>

2 1

Working with files outside galaxy without copying?
by Samuel Lampa 15 Nov '12

15 Nov '12

Hi, We are looking into ways to integrate galaxy into the workflows of users at our cluster, with lots of NGS users running all and any kind of analyses on their typically huge amounts of data. For this we use a parallel file system, available on all compute nodes. This file system, although approx 1PB in size, is constantly filling up, and thus we are not very attracted by the idea of copying files into/out of galaxy for each analysis. Thus, we would be interested to know what are the options for working with existing/external(to galaxy) file systems? Eg. would it be possible to link files into some kind of galaxy file system (I'm not totally clear about how galaxy stores it's data, although I found out that stuff is created in database/files), from outside? ... or is there any work going on for selecting any file system path as input in galaxy workflows? ... or any other hints? As said, I'm quite new to galaxy, trying to grok my head around how we can use it, so all hints are welcome. Best Regards // Samuel -- Developer at SNIC-UPPMAX www.uppmax.uu.se Developer at Dept of Pharm Biosciences www.farmbio.uu.se Twitter - twitter.com/samuellampa Blog - saml.rilspace.org G+ - gplus.to/saml

3 4

Re: [galaxy-dev] Is there any timeout in Galaxy
by Dave Clements 15 Nov '12

15 Nov '12

Hi Akshay, I don't know the answer to your question. So: 1) I went to http://galaxyproject.org/search/web/ and searched for "timeout" and then "timeout workflow." There are lots of email threads (and I learned some stuff :-), but I didn't find a direct answer to your question. 2) I'm sending your question to the galaxy-dev list where it will be seen by hundreds of galaxy admins and tool developers. (And this thread will also be found by any future searches.) Thanks, Dave C On Thu, Nov 15, 2012 at 8:33 AM, <akshaychoche(a)gmail.com> wrote: > Hello, > > My name is Akshay and I am a computer science graduate student at UGA in > Dr. Kissinger's Lab. We have created an extension to galaxy which allowed > users to add web services as tools into galaxy, these could be either used > as stand-alone or in a workflow. I wanted to know if Galaxy employs any > time out strategy. > > Consider a case where I have a workflow with BLASTP now it takes some time > to run blast and then get result form it. When I try that experiment as > standalone (i.e., running the run method first and then after several mins > getting result using job id works) however when I create a workflow in > which one tool being run method of blast followed by getresult, galaxy > seems to time out. > > Would appreciate your feedback on the same. > > -Akshay Choche > Graduate Research Assistant (Kissinger Research Group) > Center for Tropical and Emerging Global Diseases > The University of Georgia, > Athens, GA - 30605 > Tele: 256-520-6541 > Email: achoche(a)uga.edu, akshaychoche(a)gmail.com > http://www.cs.uga.edu/~choche > -- <http://galaxyproject.org/wiki/GCC2012>http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

1 0