- galaxy-dev - lists.galaxyproject.org

Cluster Usage
by Robert Jackson 16 Jun '11

16 Jun '11

Does anyone have practical information for installation and set up for the galaxy distribution on a cluster?? Robert C. Jackson Software Systems Specialist III The University of Texas Pan-American Computer Support Services Division of Information Technology Phone: 956-665-2455 Fax: 956-665-8777

2 1

Re: [galaxy-dev] [galaxy-user] Error running latest Galaxy Distribution: numpy.dtype exception
by Nate Coraor 16 Jun '11

16 Jun '11

Michael Steder wrote: > I'm just following the documentation here: https://bitbucket.org/galaxy/galaxy-central/wiki/GetGalaxy > > And I ran into the following error: > > $ hg clone https://bitbucket.org/galaxy/galaxy-dist > $ cd galaxy-dist > $ sh run.sh > Fetch successful. > Traceback (most recent call last): > File "/Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/buildapp.py", line 81, in app_factory > from galaxy.app import UniverseApplication > File "/Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/app.py", line 11, in <module> > from galaxy.tools.imp_exp import load_history_imp_exp_tools > File "/Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/tools/imp_exp/__init__.py", line 6, in <module> > from galaxy.web.base.controller import UsesHistory > File "/Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/web/base/controller.py", line 12, in <module> > from galaxy.visualization.tracks.data_providers import get_data_provider > File "/Users/steder/GO/GoGalaxy/galaxy-dist/lib/galaxy/visualization/tracks/data_providers.py", line 16, in <module> > from bx.arrays.array_tree import FileArrayTreeDict > File "numpy.pxd", line 119, in init bx.arrays.array_tree (lib/bx/arrays/array_tree.c:11323) > ValueError: numpy.dtype does not appear to be the correct type object > > I was able to run the previous stable version of Galaxy. Is there something I can do to get galaxy-dist running? Hi Mike, I'm moving this over to galaxy-dev since it deals with a local installation. It should be working, this would have been a problem for a short time using galaxy-central, but should not have affected -dist. Could you send the output of: python ./scripts/get_platforms.py python -V > > Should I be running galaxy-central instead? No, galaxy-dist is fine (and should be less prone to issues due to cloning/updating at the wrong moment). Thanks, --nate > > ~Mike > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

3 8

CSV import of Sample information Error
by Joe Cruz 16 Jun '11

16 Jun '11

Hello Everyone, I wanted to ask if anyone has experience successfully importing sample information through a csv file. I know there is an issue already created for csv file import, which I left a comment earlier: https://bitbucket.org/galaxy/galaxy-central/issue/398/csv-import-of-sample-… \ but my bug doesn't seem to deal with an 'Invalid Request ID' as the issue above states. This is the error I get: URL: http://localhost/requests_common/add_samples?cntrller=requests&id=a65cf3ca8… File '/home/galaxy1/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/home/galaxy1/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/galaxy1/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/galaxy1/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/galaxy1/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/galaxy1/galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/home/galaxy1/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 84 in decorator return func( self, trans, *args, **kwargs ) File '/home/galaxy1/galaxy-dist/lib/galaxy/web/controllers/requests_common.py', line 829 in add_samples return self.__import_samples( trans, cntrller, request, displayable_sample_widgets, libraries, **kwd ) File '/home/galaxy1/galaxy-dist/lib/galaxy/web/controllers/requests_common.py', line 1109 in __import_samples sample_copy_select_field=sample_copy_select_field ) File '/home/galaxy1/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 650 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/home/galaxy1/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 661 in fill_template_mako return template.render( **data ) File '/home/galaxy1/galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/template.py', line 133 in render return runtime._render(self, self.callable_, args, data) File '/home/galaxy1/galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 364 in _render _render_context(template, callable_, context, *args, **_kwargs_for_callable(callable_, data)) File '/home/galaxy1/galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 381 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/home/galaxy1/galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 414 in _exec_template callable_(context, *args, **kwargs) File '/home/galaxy1/galaxy-dist/database/compiled_templates/base.mako.py', line 40 in render_body __M_writer(unicode(next.body())) File '/home/galaxy1/galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 255 in <lambda> return lambda *args, **kwargs:callable_(self.context, *args, **kwargs) File '/home/galaxy1/galaxy-dist/database/compiled_templates/requests/common/ add_samples.mako.py', line 164 in render_body __M_writer(unicode(render_samples_grid( cntrller, request, displayable_sample_widgets, action='edit_samples', adding_new_samples=True, encoded_selected_sample_ids=[], render_buttons=False, grid_header=grid_header ))) File '/home/galaxy1/galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line 255 in <lambda> return lambda *args, **kwargs:callable_(self.context, *args, **kwargs) File '/home/galaxy1/galaxy-dist/database/compiled_templates/requests/common/ common.mako.py', line 749 in render_render_samples_grid sample_widget_history = sample_widget[ 'history' ] KeyError: 'history' I greatly appreciate your help and patience. Thanks, joe

2 10

Setting up Galaxy with SGE
by Ka Ming Nip 16 Jun '11

16 Jun '11

Hello, I have been trying to set up Galaxy to run its jobs on my SGE cluster using the Unified Method, based on the steps described in the wiki: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster My local instance of Galaxy is installed under my home directory, but my cluster is on a different file system. I need to "ssh" to the head node before I can submit jobs. Is there any way to set up Galaxy with a SGE cluster on a different file system? Thanks, Ka Ming

5 5

Conditional parameters regression: KeyError: '__current_case__'
by Peter Cock 16 Jun '11

16 Jun '11

Hi Dan, I just started work on updating my MIRA wrapper, and fell over a regression present of 6079:5e6254f2b9e3, steps to reproduce on Linux:- 1. Install latest Galaxy 2. Install my MIRA wrapper v0.0.1 from the tool shed (don't need MIRA for this) 3. Run Galaxy 4. Select the MIRA tool 5. Change "Backbones/reference chromosomes?" or "Sanger/Capillary reads?" or "454 reads?" or "Solexa/Illumina reads?" to yes 6. Instead of new file parameter appearing, get a KeyError: '__current_case__' exception (details below). Shorter example without a 3rd party tool: 1. Install latest Galaxy 2. Run Galaxy 3. Select the "Convert SOLiD output to fastq" tool 4. Change "Is this a mate-pair run?" to yes 5. Instead of new file parameters appearing, get a KeyError: '__current_case__' exception. By going through recent commits and retesting, I found the commit which caused this regression: changeset: 6078:894112e0b9d5 user: Daniel Blankenberg <dan(a)bx.psu.edu> date: Thu Jun 09 11:26:41 2011 -0400 summary: Some fixes for interplay between DataToolParameter, implicit datatype conversion and dynamic select lists. Closes #491. I guess there are downsides to running my development environment off galaxy-central rather than galaxy-dist ... Peter ----- Exception details, KeyError: '__current_case__' clear this clear this URL: http://127.0.0.1:8081/tool_runner/index Module weberror.evalexception.middleware:364 in respond view << try: __traceback_supplement__ = errormiddleware.Supplement, self, environ app_iter = self.application(environ, detect_start_response) try: return_iter = list(app_iter)>> app_iter = self.application(environ, detect_start_response) Module paste.debug.prints:98 in __call__ view << try: status, headers, body = wsgilib.intercept_output( environ, self.app) if status is None: # Some error occurred>> environ, self.app) Module paste.wsgilib:539 in intercept_output view << data.append(headers) return output.write app_iter = application(environ, replacement_start_response) if data[0] is None: return (None, None, app_iter)>> app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ view << environ['paste.recursive.script_name'] = my_script_name try: return self.application(environ, start_response) except ForwardRequestException, e: middleware = CheckForRecursionMiddleware(>> return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ view << []).append(HTTPException) try: return self.application(environ, start_response) except HTTPException, exc: return exc(environ, start_response)>> return self.application(environ, start_response) Module galaxy.web.framework.base:145 in __call__ view << kwargs.pop( '_', None ) try: body = method( trans, **kwargs ) except Exception, e: body = self.handle_controller_exception( e, trans, **kwargs )>> body = method( trans, **kwargs ) Module galaxy.web.controllers.tool_runner:68 in index view << # so make sure to create a new history if we've never had one before. history = trans.get_history( create=True ) template, vars = tool.handle_input( trans, params.__dict__ ) if len(params) > 0: trans.log_event( "Tool params: %s" % (str(params)), tool_id=tool_id )>> template, vars = tool.handle_input( trans, params.__dict__ ) Module galaxy.tools:966 in handle_input view << # Update state for all inputs on the current page taking new # values from `incoming`. errors = self.update_state( trans, self.inputs_by_page[state.page], state.inputs, incoming ) # If the tool provides a `validate_input` hook, call it. validate_input = self.get_hook( 'validate_input' )>> errors = self.update_state( trans, self.inputs_by_page[state.page], state.inputs, incoming ) Module galaxy.tools:1159 in update_state view << group_state = state[input.name] = {} # TODO: we should try to preserve values if we can self.fill_in_new_state( trans, input.cases[current_case].inputs, group_state, context ) group_errors = dict() group_old_errors = dict()>> self.fill_in_new_state( trans, input.cases[current_case].inputs, group_state, context ) Module galaxy.tools:876 in fill_in_new_state view << context = ExpressionContext( state, context ) for input in inputs.itervalues(): state[ input.name ] = input.get_initial_value( trans, context ) def get_param_html_map( self, trans, page=0, other_values={} ):>> state[ input.name ] = input.get_initial_value( trans, context ) Module galaxy.tools.parameters.basic:1384 in get_initial_value view << # Also collect children via association object dataset_collector( data.children ) dataset_collector( history.datasets ) most_recent_dataset = most_recent_dataset.pop() if most_recent_dataset is not None:>> dataset_collector( history.datasets ) Module galaxy.tools.parameters.basic:1378 in dataset_collector view << return False for i, data in enumerate( datasets ): if data.visible and not data.deleted and data.state not in [data.states.ERROR, data.states.DISCARDED] and ( isinstance( data.datatype, self.formats) or is_convertable( data ) ): if self.options and self._options_filter_attribute( data ) != filter_value: continue>> if data.visible and not data.deleted and data.state not in [data.states.ERROR, data.states.DISCARDED] and ( isinstance( data.datatype, self.formats) or is_convertable( data ) ): Module galaxy.tools.parameters.basic:1373 in is_convertable view << def dataset_collector( datasets ): def is_convertable( dataset ): target_ext, converted_dataset = dataset.find_conversion_destination( self.formats, converter_safe = self.converter_safe( context, trans ) ) if target_ext is not None: return True>> target_ext, converted_dataset = dataset.find_conversion_destination( self.formats, converter_safe = self.converter_safe( context, trans ) ) Module galaxy.tools.parameters.basic:1458 in converter_safe view << if input.is_dynamic and ( input.dynamic_options or ( not input.dynamic_options and not input.options ) or not input.options.converter_safe ): converter_safe[0] = False #This option does not allow for conversion, i.e. uses contents of dataset file to generate options self.tool.visit_inputs( other_values, visitor ) return False not in converter_safe >> self.tool.visit_inputs( other_values, visitor ) Module galaxy.tools:931 in visit_inputs view << callback( "", input, value[input.name] ) else: input.visit_inputs( "", value[input.name], callback ) def handle_input( self, trans, incoming, history=None ):>> input.visit_inputs( "", value[input.name], callback ) Module galaxy.tools.parameters.grouping:455 in visit_inputs view << return rval def visit_inputs( self, prefix, value, callback ): current_case = value['__current_case__'] new_prefix = prefix + "%s|" % ( self.name ) for input in self.cases[current_case].inputs.itervalues():>> current_case = value['__current_case__'] KeyError: '__current_case__'

2 5

SLURM integration
by Martin Dahlö 16 Jun '11

16 Jun '11

Hello there I am working on getting galaxy to work with the slurm queuing system. I think i have solved the slurm part of the problem (submit, status, running etc), but some local problems still exist. Our site is using the modules system (http://modules.sourceforge.net/) to handle software, and there are some kinks to work out with that, but as far as i can see, the slurm is working. It is still a work in progress, but i think it could be useful for other people who are trying to do the same thing. All the adaptations are at the moment very specific for our site, but I hope to make it possible to integrate it with the vanilla galaxy. I have a short description of what i have done at this blog: https://www.uppnex.uu.se/blog/26/galaxy-uppmax A bitbucket for the code can be found here: https://bitbucket.org/dahlo/galaxy-central Best regards Martin Dahlö --- Bioinformatic support martin.dahlo(a)scilifelab.uu.se +46 18 611 59 59 +46 706 70 79 55 http://www.scilifelab.uu.se/ http://www.uppnex.uu.se ---

2 1

The new hg based Galaxy Tool Shed
by Peter Cock 16 Jun '11

16 Jun '11

Hi Greg et al, I've just been looking over your slides from last week about the new 'Galaxy Tool Shed', which are posted online here: http://wiki.g2.bx.psu.edu/GCC2011 http://wiki.g2.bx.psu.edu/GCC2011?action=AttachFile&do=get&target=GalaxyToo… They talk about how you will be tracking individual tools in hg repositories. I can see two ways this might work: (1) Each of these tool specific repositories (or branches if you just make one repository for each tool owner) would be a full fork of the Galaxy code base. This allows in principle tools to include changes to core functionality (but that seems dangerous due to potential merge clashes), and any existing tool contributor's pre-existing hg forks on bitbucket might be reused. (2) Each of these tool specific repositories would ONLY track the tool specific files you'd add to Galaxy to install the tool. So, typically there would be an XML file, perhaps a wrapper script, maybe a sample loc file, and a plain text readme file. I'm guessing you've gone for something along the lines of idea (2), but I would love to hear more about how this will all work. e.g. Where would the tool shed repositories be hosted, and would tool authors use hg to work with them, or something like the current web based tool upload? Regards, Peter

5 15

API
by Frederick van Staden 16 Jun '11

16 Jun '11

Anyone know where i can get the galaxy API? Regards Fred

6 12

Re: [galaxy-dev] Passing username to drmaa.py
by Chorny, Ilya 15 Jun '11

15 Jun '11

I am trying to figure out how to pass the current user name to drama.py. I have had little success so far. None of the environmental variables contain any current user information. Any thoughts? Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

2 5

Re: [galaxy-dev] galaxy-dev] format="input"
by Jim Johnson 14 Jun '11

14 Jun '11

I added an attribute "format_source" (analogous to metadata_source) to the <data> tag to deal with that kind of issue: see: https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax <inputs> <param name="input" type="data" format="sff,fasta,fastq" label="454 reads" /> <param name="qual_file" type="data" format="qual" optional="true" label="Quality reads file (if input is a fasta)" /> </inputs> <outputs> <data name="output" format_source="input" label="${tool.name} on ${on_string}: cleaned reads"/> <data name="out_qual" format_source="qual_file" label="${tool.name} on ${on_string}: cleaned reads"> <filter>qual_file != None</filter> </data> </outputs> JJ > Hi everyone, > > I'm facing a little problem ... > > I have the following input : > <param name="input" type="data" format="sff,fasta,fastq" label="454 > reads" /> > <param name="qual_file" type="data" format="qual" optional="true" > label="Quality reads file (if input is a fasta)" /> > > and the following output: > <data name="output" format="input" label="${tool.name} on ${on_string}: > cleaned reads"/> > <data name="out_qual" format="qual" label="${tool.name} on ${on_string}: > cleaned reads"> > <filter>qual_file != None</filter> > </data> > > If I provide the tool a sff file, the output is sff, if I only provide a > fasta file and no qual file the output will be a fasta. > However if I provide a fasta+qual the output will be considered as qual > file even if it's a fasta file. > > I don't get how it can mark my output file as a qual file knowing it's > not a valid input file format. > > Any ideas ? > thx, > > Jerome >

3 6