May 2014 - galaxy-dev - lists.galaxyproject.org

Correct toolshed tool config for a .loc file
by Dooley, Damion 07 May '14

07 May '14

I'll try to keep this short! I'm wondering why on a fresh install of galaxy via a setup script into a python virtualenv, my custom toolshed tool's .loc files don't seem to be loading, i.e. the tool's tool_data_table_conf.xml file isn't loading. All the other parts of the tool (and other toolshed tools) do run ok. On the galaxy install where the tool was developed, it runs ok too, and I can't see any universe.wsgi.conf differences that would account for this though I'm sure the dev server has a number of other different config settings. I thought I knew my way around this .loc stuff, but I'm flummoxed. It is as though /usr/local/galaxy/test/galaxy-dist/tool-data/[toolshed domain etc etc]/00d397a381f8/tool_data_table_conf.xml isn't loading into Galaxy's tabular data list?! The only other thing I can think of is that I stuffed the tool-data related loc files into a subfolder of the tool blast_reporting.py blast_reporting.xml tool_data_table_conf.xml.sample ... tool-data/ blast_reporting_fields.loc.sample But the resulting downloaded tool config in the galaxy-dist/tool-data/..... /blast_reporting_fields.loc and tool_data_table_conf.xml files seem just fine!!! Regards, Damion

1 0

drmaa
by Shrum, Donald C 06 May '14

06 May '14

Hi all, I've configured galaxy with the drama python module. This is really more of a drama question... We have no default queue set in moab and I can't seem to find a way to specify a queue in the docs I've been looking at here - http://drmaa-python.readthedocs.org/en/latest/tutorials.html I'd like to be able to specify a queue based on various pieces of logic in my destinations.py script. Any suggestions would be appreciated. Donny FSU Research Computing Center

2 1

Re: [galaxy-dev] Bam upload fails: samtools not found
by Geert Vandeweyer 06 May '14

06 May '14

Peter, After restart of galaxy, the problems was resolved. I've had a similar issue before, where environment settings were not detected after a prolonged uptime of Galaxy. That issue was also resolved after restart. Anyway, problem is fixed for now. thanks for the answers. Geert On 05/05/2014 07:28 PM, Peter Cock wrote: > Sorry - that has gone off list, can we continue this on the mailing list please? > > On Mon, May 5, 2014 at 6:27 PM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: >> On Mon, May 5, 2014 at 6:22 PM, Geert Vandeweyer >> <geert.vandeweyer2(a)uantwerpen.be> wrote: >>> yes, it's on the galaxy users path. I log in using BASH as the shell. path >>> is loaded using .profile in case that matters. >>> >>> geert >> Does the upload job run on the main Galaxy server, or your cluster? >> >> If on the cluster, you will need to check the path settings for jobs - >> which may not include those .profile settings. >> >> Peter -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

1 0

Uploading big data in local installation of galaxy
by Jennifer Jackson 05 May '14

05 May '14

Hi Amit, You may have solved this already, but if not, there are (at least) two good options, with the second better if your data is already on the same computer that you have Galaxy running. A file 2G or over will never load through the browser - this same rule applies to a local and the public Main site (and anywhere else that I am aware of). Start by setting with a basic 'production' configuration https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer 1 - Enable FTP as described in the link above 2 - Load the data into a library through one of these options https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles I am going to post this to the galaxy-dev list to help others that may have the same question. Next time, for the speediest reply, post directly here with local admin questions. You'll benefit from replies from our team and the community (super helpful!). Best, Jen Galaxy team > Dear Jennifer, > > I installed galaxy locally on my linux machine including the tool > shed, but the file upload keeps > showing the "uploading" status forever for big datasets ( for example > CAGE reads > 6GB). > > Could you please help in sorting out the problem ? > > warm regards, > Amit. -- Jennifer Hillman-Jackson http://galaxyproject.org

1 0

permanently deleting histories
by Jennifer Gotenstein 05 May '14

05 May '14

I need to clear space from my quota of usage in my galaxy main user account, but I deleted 2 histories, rather than permanently deleting them. How can I access them again so that I may permanently delete them to clear the space? Thank you, Jennifer

2 1

Portability and versionig in Galaxy
by Golestan (Sally) Radwan 05 May '14

05 May '14

Hi All, I am completely new to galaxy but part of my PhD research is to look at portability issues of workloads across different versions. I've tried looking at the wiki and the various pieces of documentation but no luck so far. Can someone please point me to a document or tutorial that explains how this is currently done in Galaxy and also any issues/gaps that may be worth addressing? I am also generally interested in how Galaxy works "under the hood". Is there a document that explains the architecture and design? Many thanks in advance! Sally.

2 1

Cannot access saved history nor saved datasets
by Joshua David Aaker 05 May '14

05 May '14

I am able to use my current history and run tools but I can't move away nor create new histories. I am currently using the Cistrome instance of Galaxy. Here is the error I receive: Internal Server Error Galaxy was unable to successfully complete your request URL: http://cistrome.org/ap/history/list Module galaxy.web.framework.middleware.error:149 in __call__ >> <http://cistrome.org/ap/history/list#> app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ >> <http://cistrome.org/ap/history/list#> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> <http://cistrome.org/ap/history/list#> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> <http://cistrome.org/ap/history/list#> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request >> <http://cistrome.org/ap/history/list#> body = method( trans, **kwargs ) Module galaxy.web.framework:98 in decorator >> <http://cistrome.org/ap/history/list#> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.history:294 in list >> <http://cistrome.org/ap/history/list#> return self.stored_list_grid( trans, status=status, message=message, **kwargs ) Module galaxy.web.framework.helpers.grids:209 in __call__ >> <http://cistrome.org/ap/history/list#> total_num_rows = query.count() Module sqlalchemy.orm.query:2571 in count Module sqlalchemy.orm.query:2215 in scalar Module sqlalchemy.orm.query:2184 in one Module sqlalchemy.orm.query:2227 in __iter__ Module sqlalchemy.orm.query:2242 in _execute_and_instances Module sqlalchemy.engine.base:1449 in execute Module sqlalchemy.engine.base:1584 in _execute_clauseelement Module sqlalchemy.engine.base:1698 in _execute_context Module sqlalchemy.engine.base:1691 in _execute_context Module sqlalchemy.engine.default:331 in do_execute Module MySQLdb.cursors:173 in execute Module MySQLdb.connections:36 in defaulterrorhandler OperationalError: (OperationalError) (1030, 'Got error 28 from storage engine') 'SELECT count(*) AS count_1 \nFROM (SELECT history.id AS history_id, history.create_time AS history_create_time, history.update_time AS history_update_time, history.user_id AS history_user_id, history.name AS history_name, history.hid_counter AS history_hid_counter, history.deleted AS history_deleted, history.purged AS history_purged, history.importing AS history_importing, history.genome_build AS history_genome_build, history.importable AS history_importable, history.slug AS history_slug, history.published AS history_published \nFROM history \nWHERE history.importing = %s AND %s = history.user_id AND history.deleted = %s ORDER BY history.update_time DESC) AS anon_1' (0, 1889L, 0) Any suggestions or more information you need? -Josh

2 1

Galaxy REST API: multiple params per tool
by Jim McCusker 05 May '14

05 May '14

The current method for supplying parameters to tools using the REST API for workflows is fairly broken. The structure is as follow: "parameters":{ "tool_1": { "param":"param_name", "value":"param value" }, "tool_2": { "param":"param_name", "value":"param value" } This structure means that it is impossible to submit more than 1 parameter per tool within a workflow. Tools themselves, and workflows that use them, have no problems with accepting multiple parameters. We should be supporting something more like the following: "parameters":{ "tool_1": { "param_name1":"param value 1" "param_name2":"param value 2" }, "tool_2": { "tool_2_param_name1":"param value 1" "tool_2_param_name2":"param value 2" } Is there any work towards something like this taking place? I imagine that the REST API could support both formats (if the tool parameters dict has anything in it other than "param" and "value", use the new structure) for backwards compatibility. Thanks, Jim

1 1

Missing pyyaml - raise DistributionNotFound(req) # XXX put more info here
by Peter Cock 05 May '14

05 May '14

Hi all, Recently my TravisCI tests have started failing during the tool functional tests due to what looks like a missing dependency: e.g. https://travis-ci.org/peterjc/pico_galaxy/jobs/24444682 ... galaxy.eggs DEBUG 2014-05-05 11:26:47,387 Fetched http://eggs.galaxyproject.org/boto/boto-2.27.0-py2.7.egg galaxy.eggs DEBUG 2014-05-05 11:26:47,753 Fetched http://eggs.galaxyproject.org/bioblend/bioblend-0.4.2-py2.7.egg Traceback (most recent call last): File "./scripts/functional_tests.py", line 601, in <module> sys.exit( main() ) File "./scripts/functional_tests.py", line 417, in main use_translogger=False, static_enabled=STATIC_ENABLED ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/webapps/galaxy/buildapp.py", line 49, in app_factory webapp.add_ui_controllers( 'galaxy.webapps.galaxy.controllers', app ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/web/framework/__init__.py", line 441, in add_ui_controllers module = import_module( module_name ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/util/backports/importlib/__init__.py", line 37, in import_module __import__(name) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/webapps/galaxy/controllers/cloudlaunch.py", line 20, in <module> eggs.require('pyyaml') File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/eggs/__init__.py", line 412, in require return pkg_resources.working_set.require( req_str ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/pkg_resources.py", line 666, in require needed = self.resolve(parse_requirements(requirements)) File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: pyyaml Is there a missing egg declaration for pyyaml? I found this commit about a PyYAML 3.10 egg., perhaps linked? https://bitbucket.org/galaxy/galaxy-central/commits/c36dc4155beb0ae5e117ca3… Thanks, Peter

2 8

Bam upload fails: samtools not found
by Geert Vandeweyer 05 May '14

05 May '14

Hi, I'm having issues uploading bam files to our galaxy server. It fails for both FTP and browser upload. Upload.py is executed on the local job runner. The error is: Traceback (most recent call last): File "/galaxy/galaxy-dist/tools/data_source/upload.py", line 390, in <module> __main__() File "/galaxy/galaxy-dist/tools/data_source/upload.py", line 379, in __main__ add_file( dataset, registry, json_file, output_path ) File "/galaxy/galaxy-dist/tools/data_source/upload.py", line 318, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 144, in dataset_content_needs_grooming version = self._get_samtools_version() File "/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 128, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib/python2.6/subprocess.py", line 633, in __init__ errread, errwrite) File "/usr/lib/python2.6/subprocess.py", line 1139, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory The logs show 'galaxy.tools.deps WARNING 2014-05-05 15:39:20,676 Failed to resolve dependency on 'samtools', ignoring' I do have samtools in my path (~/bin/samtools). Also, the file is actually uploaded fine (can be downloaded again correctly), but cannot be used in galaxy due to its 'error' state. Any advice on how to solve this? Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1