November 2011 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] ***UNCHECKED*** Re: HTTP Error 404 pushing to toolshed.g2.bx.psu.edu
by Greg Von Kuster 10 Nov '11

10 Nov '11

Hello Jim, It turns out that our nginx configuration setting for file size was too small to allow for your new archive and the nginx error page to report the problem was missing (hence, the 404). This has all now been corrected, so you should have no problem uploading your new version for defuse. Thanks for discovering this problem, and sorry for any inconvenience this may have caused. Greg Von Kuster On Nov 9, 2011, at 10:14 PM, Jim Johnson wrote: > Greg, > > I had been trying to push from my local clone, as I had before. That failed with a 404 error. > I next tried uploading the tarball, which also resulted in a 404 error. > > My original upload that opened the repository had a defuse subdir. > I moved the contents of the subdir to the toplevel with the changeset 1 ( 1:36306d8086fa) > > This update was for the latest release of DeFuse version 0.4.3. > > Feel free to try whatever you need to. I granted you permissions on the repository if that helps. > I imagine we could delete the repository and recreate it need be. > > Thanks, > > JJ > > > > > On 11/9/11 8:32 PM, Greg Von Kuster wrote: >> >> Hi Jim, >> >> Thanks for sending your archive. >> >> I can successfully clone your defuse repository, but I have not tried pushing anything to it - I don't want to write to your repository. I noticed that your last change set removed the defuse directory. Did you do that by uploading a new tarball that did not include the defuse directory or did you have a locally cloned defuse repository from which you were pushing changes? >> >> Did the 404 error occur when you attempted to upload the tarball (you sent me) to the repository, or was it a specific file from the tarball? >> >> Thanks! >> >> >> On Nov 9, 2011, at 8:13 PM, Jim Johnson wrote: >> >>> Here it is. >>> JJ >>> >>> On 11/9/11 5:14 PM, Greg Von Kuster wrote: >>>> Hi Jim, >>>> >>>> Would you mind attaching a tarball of the changed files you were attempting to upload or push and send them to me? It would help me figure out the problem. >>>> >>>> Thanks! >>>> >>>> Greg >>>> >>>> On Nov 9, 2011, at 4:35 PM, Jim Johnson wrote: >>>> >>>>> I successfully pushed updates to my gmap and mothur toolshed repositories yesterday, >>>>> but got an error when I tried to push changes to defuse. >>>>> >>>>> % hg push >>>>> pushing to http://jjohnson@toolshed.g2.bx.psu.edu/repos/jjohnson/defuse >>>>> searching for changes >>>>> abort: HTTP Error 404: Not Found >>>>> >>>>> >>>>> I also tried uploading via the webform and got the same 404 error from nginx/1.1.4 >>>>> >>>>> Any insights? >>>>> >>>>> Thanks, >>>>> >>>>> JJ >>>>> >>>>> >>>>> >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> http://lists.bx.psu.edu/ >>>> Greg Von Kuster >>>> Galaxy Development Team >>>> greg(a)bx.psu.edu >>>> >>>> >>>> >>> <defuse_update.tar.gz> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

How to ensure that param A is less than param B?
by Steven Platt 10 Nov '11

10 Nov '11

Hi everyone, I have 2 numerical params specified as follows inside the <inputs> tags <param type="integer" name="Mymin" label="Start here" help="Must be an ODD number (11-51)" min="11" max="51" value="29"/> <param type="integer" name="Mymax" label="End here" help="Must be an ODD number (21-71)" min="21" max="71" value="45"/> The values are happily restricted between their respective min/max values, but can someone tell me how I can check that Mymin is less than Mymax, and/or that both values are odd numbers? I realise that I could do this with the script that Galaxy executes but I'm hoping that there's a more elegant way which will retain the tool interface and prompt the user to enter correct values. Thanks Steve ----------------------------------------- ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

1 0

Configuration of a local install - mi-deploy ?
by Clare Sloggett 10 Nov '11

10 Nov '11

Hi all, Most of our playing around with Galaxy has been in getting it working on our local cloud, but now for the first time I'm configuring a non-cloud local install of galaxy-dist (set up as per http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Production%20Server) So I have some naive questions! Would it be a sensible approach to grab the tools_fabfile script from mi-deployment and use it in this case? Or should I be using the Tool Shed for installing the base set of tools? Also, if I have problems with this server sending out emails (which may be the case) am I going to run into trouble with user/password management or can I just admin everything manually? There will only be a very small number of users on this server. If I have missed any good 'getting started' documentation on config/admin of a local install please point me in the right direction. I've been looking at http://wiki.g2.bx.psu.edu/Admin . Thanks, Clare -- E: sloc(a)unimelb.edu.au P: 03 903 53357 M: 0414 854 759

2 1

Re: [galaxy-dev] ***UNCHECKED*** Re: HTTP Error 404 pushing to toolshed.g2.bx.psu.edu
by Greg Von Kuster 09 Nov '11

09 Nov '11

Hi Jim, Thanks for sending your archive. I can successfully clone your defuse repository, but I have not tried pushing anything to it - I don't want to write to your repository. I noticed that your last change set removed the defuse directory. Did you do that by uploading a new tarball that did not include the defuse directory or did you have a locally cloned defuse repository from which you were pushing changes? Did the 404 error occur when you attempted to upload the tarball (you sent me) to the repository, or was it a specific file from the tarball? Thanks! On Nov 9, 2011, at 8:13 PM, Jim Johnson wrote: > Here it is. > JJ > > On 11/9/11 5:14 PM, Greg Von Kuster wrote: >> Hi Jim, >> >> Would you mind attaching a tarball of the changed files you were attempting to upload or push and send them to me? It would help me figure out the problem. >> >> Thanks! >> >> Greg >> >> On Nov 9, 2011, at 4:35 PM, Jim Johnson wrote: >> >>> I successfully pushed updates to my gmap and mothur toolshed repositories yesterday, >>> but got an error when I tried to push changes to defuse. >>> >>> % hg push >>> pushing to http://jjohnson@toolshed.g2.bx.psu.edu/repos/jjohnson/defuse >>> searching for changes >>> abort: HTTP Error 404: Not Found >>> >>> >>> I also tried uploading via the webform and got the same 404 error from nginx/1.1.4 >>> >>> Any insights? >>> >>> Thanks, >>> >>> JJ >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > > <defuse_update.tar.gz> Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

HTTP Error 404 pushing to toolshed.g2.bx.psu.edu
by Jim Johnson 09 Nov '11

09 Nov '11

I successfully pushed updates to my gmap and mothur toolshed repositories yesterday, but got an error when I tried to push changes to defuse. % hg push pushing to http://jjohnson@toolshed.g2.bx.psu.edu/repos/jjohnson/defuse searching for changes abort: HTTP Error 404: Not Found I also tried uploading via the webform and got the same 404 error from nginx/1.1.4 Any insights? Thanks, JJ

1 0

dropdown user-type
by Nikolai Vazov 09 Nov '11

09 Nov '11

Hi, I can't find out how to use the dropdown "User type" which is displayed after I select User > Register in the start page of Galaxy. Do I have to define the types myself? Are they pre-set? The dropdown menu just says "Select one", but the options are empty. Thank you in advance Nikolay -- Nikolay Vazov, PhD Research Computing Centre - http://hpc.uio.no USIT, University of Oslo

2 1

Error working with workflows: KeyError: 'input_bam'
by Carlos Borroto 09 Nov '11

09 Nov '11

Hi, I'm trying to develop a workflow using several tools from GATK and Picard sections among others. I'm able to run all the tools and from the history I'm creating a workflow, but I'm running into an error. I have copied the full debugging trace from my local Galaxy, I was able to import this workflow into the test server and here it is: http://test.g2.bx.psu.edu/u/cjav/w/gatk---unified-genotyper The test server also gave me an error, but I don't have the option to see the debugging trace. Thanks a lot for any help, Carlos URL: http://localhost:8888/workflow/run?id=f597429621d6eb2b File '/home/cborroto/galaxy_dist/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/cborroto/galaxy_env/lib/python2.7/site-packages/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/home/cborroto/galaxy_env/lib/python2.7/site-packages/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/cborroto/galaxy_env/lib/python2.7/site-packages/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/cborroto/galaxy_env/lib/python2.7/site-packages/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/cborroto/galaxy_dist/lib/galaxy/web/framework/base.py', line 145 in __call__ body = method( trans, **kwargs ) File '/home/cborroto/galaxy_dist/lib/galaxy/web/controllers/workflow.py', line 1438 in run incoming=kwargs ) File '/home/cborroto/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 691 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/home/cborroto/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 702 in fill_template_mako return template.render( **data ) File '/home/cborroto/galaxy_dist/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/home/cborroto/galaxy_dist/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/home/cborroto/galaxy_dist/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/home/cborroto/galaxy_dist/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/home/cborroto/galaxy_dist/database/compiled_templates/base.mako.py', line 42 in render_body __M_writer(unicode(next.body())) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 160 in render_body __M_writer(unicode(do_inputs( tool.inputs, step.state.inputs, errors.get( step.id, dict() ), "", step ))) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 40 in do_inputs return render_do_inputs(context.locals_(__M_locals),inputs,values,errors,prefix,step,other_values) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 406 in render_do_inputs __M_writer(unicode(do_inputs( input.cases[ current_case ].inputs, group_values, group_errors, new_prefix, step, other_values ))) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 317 in do_inputs return render_do_inputs(context,inputs,values,errors,prefix,step,other_values) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 412 in render_do_inputs __M_writer(unicode(row_for_param( input, values[ input.name ], other_values, errors, prefix, step ))) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 315 in row_for_param return render_row_for_param(context,param,value,other_values,error_dict,prefix,step) File '/home/cborroto/galaxy_dist/database/compiled_templates/workflow/run.mako.py', line 503 in render_row_for_param value = other_values[ param.name ] = param.get_initial_value( t, other_values ) File '/home/cborroto/galaxy_dist/lib/galaxy/tools/parameters/basic.py', line 762 in get_initial_value if self.need_late_validation( trans, context ): File '/home/cborroto/galaxy_dist/lib/galaxy/tools/parameters/basic.py', line 745 in need_late_validation dep_value = context[ dep_name ] KeyError: 'input_bam'

2 5

Re: [galaxy-dev] making data libraries from database tables
by Greg Von Kuster 09 Nov '11

09 Nov '11

Hi James, Since genomic data files are often very large, Galaxy does not store them in a database, so this specific scenario has not been implemented as far as I know. However, you may be able to implement what you've described without too much difficulty. If you could implement a layer on top of your database that leverages Galaxy's features for uploading a directory of files or file system paths (maybe better in this case) without copying the data into Galaxy's default file store, it should be fairly trivial to make Galaxy work with it. Using this combination, Galaxy will read the data (without making any changes to it) in order to generate metadata associated with the data. The metadata is stored separately from the raw data. I was at the Pac Bio meeting, so we definitely met there. Good to hear from you! On Nov 7, 2011, at 8:58 PM, James Ireland wrote: > Hi Greg, > > Thanks for the fast response! I think we might have met last year at the PacBio 3rd party software vendor meeting. > > So, I had seen the documents for the data repository and the"Uploading a Directory of Files" with the "Copy data into Galaxy?" option de-selected seems the closest analog to what I want to do. In my complete and utterly naive understanding of how Galaxy works, if I could wrap my data repository (in this case, my db) with the same sort of functionality as a file directory (scan, load, etc) then I would guess that the integration wouldn't be that painful. Obviously, this would require custom development. This is important enough to my company that we'd be willing to work on doing this - but I'm guessing I'm way off base? > > This seems like it would be a fairly common request - to your knowledge, has anyone outside Galaxy rolled their own solution along these lines? > > Thanks again, > -J > > > On Mon, Nov 7, 2011 at 11:41 AM, Greg Von Kuster <greg(a)bx.psu.edu> wrote: > Hello James, > > This is not currently possible - the options for uploading files to Galaxy data libraries is documented in our wiki at http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files > > On Nov 7, 2011, at 2:11 PM, James Ireland wrote: > >> Greetings! >> >> I would like to expose data I have in a relational database as a data library in Galaxy. I would really like to do this without Galaxy having to make a local copy of the data to the file system. Is this possible and could you point me to any code examples and/or documentation? >> >> I'm sure this must be covered somewhere in the documentation or mailing list, but I haven't been able to find it. >> >> Thanks for your help! >> >> -James >> -- >> J Ireland >> www.5amsolutions.com | Software for Life(TM) >> m: 415 484-DATA (3282) >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ > > Greg Von Kuster > Galaxy Development Team > greg(a)bx.psu.edu > > > > > > > -- > J Ireland > www.5amsolutions.com | Software for Life(TM) > m: 415 484-DATA (3282) Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

4 6

History reports misconfiguration of the Galaxy job running system
by Andrew Warren 09 Nov '11

09 Nov '11

Hello, I recently encountered a problem when trying to run Cufflinks on eight BAM files on our galaxy instance (via the multi-input toggle) and received the error: "Unable to run job due to a misconfiguration of the Galaxy job running system" for some, but not all, of the cufflinks jobs that appear in the history. These particular BAM files were copied over from the history of a larger workflow where they were successfully run through cufflinks. In the case of the problem workflow run, the cufflinks jobs are all successfully submitted to Torque PBS and continue to run and finish but many have this error displayed in the History. The jobs with the error displayed fail to copy files from the working directory to the database directory despite running to completion. We recently updated to galaxy-central changeset 6131:be6c89c33639. I have seen this error multiple times for this workflow, even after restarting galaxy. Does anyone have any ideas of what might be going wrong? Thanks for any help, Andrew Warren

2 3

Tool shed and datatypes
by Duddy, John 08 Nov '11

08 Nov '11

Can we introduce new file types via tools in the tool shed? It seems Galaxy can load them if they are in the datatypes configuration file. Does tool installation automate the editing of that file? John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy(a)illumina.com<mailto:jduddy@illumina.com>

3 14