October 2012 - galaxy-dev - lists.galaxyproject.org

Downloading UCSC complete database
by Perez, Ricardo 07 Oct '12

07 Oct '12

Dear all, I am currently working on downloading the genome data from the UCSC database. I have figured out how to obtain the genome of one species at a time, however this would take a bit of time if I have to type every command by hand. Is there any command that would download the all the data from the UCSC databases? If not, how would I go to start in writing a script that would do so. Thank you for your time. -Ricardo

2 1

Galaxy October 5, 2012 Distribution & News Brief
by Jennifer Jackson 05 Oct '12

05 Oct '12

*Galaxy October 5, 2012 Distribution & News Brief* /http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05/ /* Highlights:* http://wiki.g2.bx.psu.edu/News / ** * *BWA <http://bio-bwa.sourceforge.net/>* tool wrappers /migrated <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#BWA_Migration_to_Tool_Sh…>/ to the *Galaxy Main Tool Shed <http://toolshed.g2.bx.psu.edu/>*. * *Tool Factory* paper has been /E Published <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Galaxy_Tool_Factory_Pape…>/ in PubMed <http://www.ncbi.nlm.nih.gov/pubmed> at *PMID: 23024011 <http://www.ncbi.nlm.nih.gov/pubmed/23024011>*. * The *Tool Shed Wiki <http://wiki.g2.bx.psu.edu/Tool%20Shed>* wiki has been re-written with a new, cleaner presentation. And check out all the /new features <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Tool_Shed>/! * Accessible *tool versions* in the UI and *new exit code features* enable /clearer experimental tracking and ease reproducibility <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Tools>/. * Release also includes *Visualization <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Visualization_framework>* updates, a new admin tool for /PostgreSQL <http://www.postgresql.org>/ *Dataset cleanup <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Dataset_Cleanup>*, and improvements in *API <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#AP>* , *Workflow <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Workflows>*, and *Security <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_05#Security_Fixes>* components. http://getgalaxy.org http://bitbucket.org/galaxy/galaxy-dist * new: * $ hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist * upgrade:* $ hg pull -u -r b5bda7a5c345 ** * Thanks for using Galaxy!* Jennifer Jackson & Galaxy Team http://galaxyproject.org

1 0

"No module named controllers.library_common" error on fresh Galaxy install from 4-Oct-2012
by Anthonius deBoer 05 Oct '12

05 Oct '12

I gave up on figuring out why jobs don't get started for 15 minutes and started a new instance of galaxy, but when I tried to create a new data library, I got this error... Error attempting to display contents of library (TEST DATA): No module named controllers.library_common. Any ideas? Thon

2 1

Display data in browser
by Julian Pradinuk 05 Oct '12

05 Oct '12

Hi, I am proxying with Apache (and using xsendfile). I wrote a wrapper for FastTree (http://www.microbesonline.org/fasttree/) that uses the stderr_wrapper.py method to handle with stderr output. When I click the display data in browser button (the eye) for the tree output (txt file) I am prompted to download the file. It downloads as with a file name and extension like 4HYYCqht.dms.part. Has anyone encountered this issue before? Failing to see anything different in the apache logs.. Cheers, Julian

1 0

Instant crash on large history export
by Bossers, Alex 05 Oct '12

05 Oct '12

Another problem; On our ubuntu x64 server with the latest galaxy_central of Oct 4th... An export of the history sized 2.5Gb was ok. However large exports (like 10Gb) make the server instantly crash (within a few seconds) without anything significant in the galaxy.log. The apache server is still running though... Help! Alex

2 1

Crash on large workflow
by Todd Oakley 05 Oct '12

05 Oct '12

Hello, We have a rather large workflow (120 steps) that works on one of our Galaxy instances, but crashes Galaxy on another instance. All the individual tools work on both instances. The error in paster.log is: TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 This is an older linux machine running CentOS, and there is no queuing system. Is the queuepool limit a Galaxy variable? Or is this a limitation of how we have the machine configured? I've tried searching the Galaxy wiki, but no luck yet. I will keep searching, too. Thanks! Todd -- *************************************** Todd Oakley, Professor Ecology Evolution and Marine Biology University of California, Santa Barbara Santa Barbara, CA 93106 USA ***************************************

1 0

Invitation to use Google Talk
by Google Talk 05 Oct '12

05 Oct '12

----------------------------------------------------------------------- You've been invited by suzan katie to use Google Talk. If you already have a Google account, login to Gmail and accept this chat invitation: http://mail.google.com/mail/b-6dd94008a7-4956633c59-7YVOdLTCcdh4qfgxeie2tbh… To sign up for a Google account and get started with Google Talk, you can visit: http://mail.google.com/mail/a-6dd94008a7-4956633c59-7YVOdLTCcdh4qfgxeie2tbh… Learn more at: http://www.google.com/intl/en/landing/accounts/ Thanks, The Google Team

1 0

Workflow Batching Enhancements
by John Chilton 05 Oct '12

05 Oct '12

Hello Galaxy People, I have put together some batch workflow enhancements that I think could start to address some of the use cases mentioned at the Galaxy conference. This pull request (https://bitbucket.org/galaxy/galaxy-central/pull-request/75/enhanced-batch-…) implements the backend for allowing multiple inputs to be selected on multiple different params when running a workflow (currently Galaxy only allows one input parameter to be batched). The UI piece has not really been addressed but I was hoping to start a conversation on that. The backend currently has two processing "modes". I will illustrate them with examples I heard about at the conference. One thing I heard mentioned is batching processing paired FASTQ files is difficult, and there were some hacks mentioned to address that. If you pull in the above changeset, remove this line from run.mako: $('.multiinput').addClass('disabled'); and ensure that this element is set to "matched" in the form: <input type="hidden" name="multiple_input_mode" value="matched" /> Then you can select multiple inputs on any number of parameters and they will be matched up (based on the order they appear in the history) and the workflow will be run for each match. Right know the UI won't ensure that the same number of items is selected on each so that piece should be added. Someone else had mentioned doing sort of all versus all workflows (in the context of microarray data). This is what I called product mode. Again if you remove the JavaScript line mentioned above and ensure that this element is set to product in the form: <input type="hidden" name="multiple_input_mode" value="product" /> Then you can select multiple inputs on any number of parameters and the workflow will be run on each combination of the inputs. To illustrate the difference - if user selected 5 inputs for one param and 5 inputs for another in matched mode that will be run the workflow 5 times matching each input and in product mode it will run the workflow 25 times with every combination of input pairs. If user selects 6 inputs for one param and 4 for another, in product mode 24 workflows will run and in matched mode the submission will fail. It should be straight forward to pull in the changeset and hack up your Galaxy to do one or the other. I guess though we should allow both and let the user decide :). I was hoping for some ideas on the UI front, what the best way to communicate the differences, can we do this without cluttering up the UI, should there be a separate advanced workflow run mode, how do other workflow applications handle this distinction, etc.... One thought I had was allowing a choice between these two modes on a per workflow basis is a nice step forward from where we are now, but maybe another good step forward would be to allow the decision on a per parameter basis. What I was thinking was maybe when you select the little icon for multiple inputs, a broken link chain icon would appear next to the multi-input icon indicating "product" mode would be enabled for that input (i.e. the inputs are not linked), and then if the user clicked the little broken chain on some parameters and those chains would close (and maybe glow) and those parameters would become "linked" (or matched). Then you could mix and match, for instance you could run n*2 fastq files paired off each against m different parameter files. Mixing and matching would take a little more work, but if that makes the UI more straight forward it might be worth it (I would be happy to implement these extensions). Though I am pretty bad at UI so I assume I am fairly off base on this link idea. Thanks, -John ------------------------------------------------ John Chilton Senior Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223 Bitbucket: https://bitbucket.org/jmchilton Github: https://github.com/jmchilton Web: http://jmchilton.net

1 0

trying to set up local toolshed
by David Hoover 05 Oct '12

05 Oct '12

I'm trying to host a local toolshed, and I've hit a brick wall. It is accessible as a subdirectory under our galaxy installation, uses SQLite as the database, and I've configured it as much as I understand. After I start it up, I can see the main page in the browser, but get this error message: Not Found The resource could not be found. No route for /repository/browse_categories Clicking on 'Repositories' gives this error: This link may not be followed from within Galaxy. Beyond this, can the toolshed use externally authenticated users? David Hoover Helix Systems Staff

2 1

stack trace from a history problem
by Langhorst, Brad 05 Oct '12

05 Oct '12

On one particular history, I see this. I am at 035583d94b11<https://bitbucket.org/galaxy/galaxy-central/changeset/035583d94b11200276aed…> I don't know why this happens on this history… I cloned it, then re-ran one step. I hope this stacktrace is helpful: Exception happened during processing of request from ('127.0.0.1', 56949) Traceback (most recent call last): File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 1053, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__ self.handle() File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 432, in handle BaseHTTPRequestHandler.handle(self) File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle self.handle_one_request() File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 427, in handle_one_request self.wsgi_execute() File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 287, in wsgi_execute self.wsgi_start_response) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/urlmap.py", line 202, in __call__ return app(environ, start_response) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/xforwardedhost.py", line 21, in __call__ return self.app( environ, start_response ) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.6.egg/paste/deploy/config.py", line 164, in __call__ app_iter = self.application(environ, start_response) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/translogger.py", line 68, in __call__ return self.application(environ, replacement_start_response) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py", line 153, in __call__ exc_info) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/translogger.py", line 67, in replacement_start_response return start_response( status, headers, exc_info ) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py", line 143, in __call__ app_iter = self.application(environ, start_response) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py", line 80, in __call__ return self.application(environ, start_response) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py", line 91, in __call__ return self.app( environ, start_response ) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py", line 632, in __call__ return self.application(environ, start_response) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py", line 169, in __call__ return body( environ, start_response ) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py", line 858, in render template.render_context( context ) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg/mako/template.py", line 319, in render_context **kwargs) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg/mako/runtime.py", line 692, in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/Mako-0.4.1-py2.6.egg/mako/runtime.py", line 718, in _exec_template callable_(context, *args, **kwargs) File "/mnt/ngswork/galaxy/galaxy-dist/database/compiled_templates/root/history.mako.py", line 306, in render_body __M_writer(unicode(render_dataset( data, data.hid, show_deleted_on_refresh = show_deleted, for_editing = True ))) File "/mnt/ngswork/galaxy/galaxy-dist/database/compiled_templates/root/history_common.mako.py", line 553, in render_render_dataset visualizations = data.get_visualizations() File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1251, in get_visualizations return self.datatype.get_visualizations( self ) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/datatypes/tabular.py", line 312, in get_visualizations for col_type in dataset.metadata.column_types: TypeError: 'NoneType' object is not iterable brad -- Brad Langhorst langhorst(a)neb.com<mailto:langhorst@neb.com>

2 1