September 2016 - galaxy-dev - lists.galaxyproject.org

workflow import error
by Tony Schreiner 30 Sep '16

30 Sep '16

We have 2 galaxy instances running one is v15.05-74-gf6b894a the other is v15.10-30-g6a47d2d a user is trying to import a workflow file, it works on the older one, but raises an error on the newer one. URL: http://redacted.edu:8080/workflow/import_workflow <http://prince.bc.edu:8080/workflow/import_workflow> File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/middleware/error.py', line 151 in __call__ app_iter = self.application(environ, sr_checker) File '/cluster/home/galaxy/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/cluster/home/galaxy/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/base.py', line 133 in __call__ return self.handle_request( environ, start_response ) File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/base.py', line 191 in handle_request body = method( trans, **kwargs ) File '/cluster/home/galaxy/galaxy/lib/galaxy/webapps/galaxy/controllers/workflow.py', line 921 in import_workflow workflow, missing_tool_tups = self._workflow_from_dict( trans, data, source=src, add_to_menu=add_to_menu ) File '/cluster/home/galaxy/galaxy/lib/galaxy/web/base/controller.py', line 1111 in _workflow_from_dict publish=publish File '/cluster/home/galaxy/galaxy/lib/galaxy/managers/workflows.py', line 159 in build_workflow_from_dict module, step = self.__module_from_dict( trans, step_dict, secure=False ) File '/cluster/home/galaxy/galaxy/lib/galaxy/managers/workflows.py', line 605 in __module_from_dict module.save_to_step( step ) File '/cluster/home/galaxy/galaxy/lib/galaxy/workflow/modules.py', line 592 in save_to_step step.tool_inputs = self.tool.params_to_strings( self.state.inputs, self.trans.app ) File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/__init__.py', line 1980 in params_to_strings return params_to_strings( self.inputs, params, app ) File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/parameters/__init__.py', line 92 in params_to_strings value = params[ key ].value_to_basic( value, app ) File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/parameters/grouping.py', line 574 in value_to_basic rval[ self.test_param.name ] = self.test_param.value_to_basic( value[ self.test_param.name ], app ) KeyError: 'uncol' i'm not able conclude much from this dump. Can anyone suggest some steps to debug this. Cheers Tony Schreiner

2 1

Problem with conda on galaxy 16.07
by Léo Biscassi 30 Sep '16

30 Sep '16

Hi folks, Today after perform some tests with conda on galaxy 16.07 the follow message was returned in my terminal: "*Conda dependency seemingly installed but failed to build job environment.* " The error occurs when *build_isolated_environment()* function is call. Below some variable values: *is_installed* = *True* *job_directory* = */home/leobiscassi/programs/galaxy/job_work_dir/000/17* *env_path* = */home/leobiscassi/programs/galaxy/job_work_dir/000/17/conda-env* *exit_code* = *127* Analyzing inside of the *build_isolated_environment()*, the *exec_create() *function is executed because *path* variable ( */home/leobiscassi/programs/galaxy/job_work_dir/000/17/conda-env*) don't exists, the command executed by *exec_create* is showed below: *deps/_conda/bin/conda create -y --unknown --offline --prefix /home/leobiscassi/programs/galaxy/job_work_dir/000/17/conda-env --file ./database/tmp/jobdepsIGslWfe2bd2345d2ec1bb4aeb81211183570bae7db28a3bea4f7ff9e8e75074719a7ee/__mgltools(a)1.5.6 > /dev/null* Some important values: *env* = *{'HOME': 'deps/_conda', 'CONDARC': '/home/leobiscassi/programs/galaxy/deps/_condarc'}* The command return is '*/bin/sh: 1: deps/_conda/bin/conda: not found*', but the binary exist. Anyone can help me with this? Thanks in advance. -- Best regards, *Léo Biscassi*

2 1

compilation fails in pyBamTools & pyBamParser dependency packages
by Nikolaos Tur 30 Sep '16

30 Sep '16

Hi, installing tool dependency package_pybamtools_0_0_2 as well as package_pybamparser_0_0_1 cause below mansion errors. Following the tool_dependencies.xml file: <action type="shell_command">hg clone -r ba0341816b15fff5e0a554658d6579a60e1b12b1 https://bitbucket.org/dan/pybamtools pyBamTools</action> <action type="make_directory">$INSTALL_DIR/lib/python</action> <action type="shell_command">export PYTHONPATH=$PYTHONPATH:$INSTALL_DIR/lib/python && python setup.py install --home $INSTALL_DIR</action> I can download this package and manually compile it without errors under Galaxy environments. Question is what are sources of these errors and how to solve them using standard Galaxy installation procedure? After googling this errors there were mentions that it can be due to deprecated package `distribute' in new version of python and it is better to use `setuptools'. It is also described in pybamtools sources and after small modification to use `setuptools' in setup.py script it is also manually compiled without problems. Thanx, Nikolaos Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.38.tar.gz Extracting in /tmp/tmpDIcsTL Now working in /tmp/tmpDIcsTL/distribute-0.6.38 Building a Distribute egg in /home/galaxy/galaxy/database/tmp/tmp-toolshed-mtdFbEf6k/pyBamTools Traceback (most recent call last): File "setup.py", line 248, in <module> scripts = scripts, File "/home/galaxy/python27/python/2.7.9/lib/python2.7/distutils/core.py", line 111, in setup _setup_distribution = dist = klass(attrs) File "/tmp/tmpDIcsTL/distribute-0.6.38/setuptools/dist.py", line 225, in __init__ _Distribution.__init__(self,attrs) File "/home/galaxy/python27/python/2.7.9/lib/python2.7/distutils/dist.py", line 287, in __init__ self.finalize_options() File "/tmp/tmpDIcsTL/distribute-0.6.38/setuptools/dist.py", line 257, in finalize_options ep.require(installer=self.fetch_build_egg) File "/tmp/tmpDIcsTL/distribute-0.6.38/pkg_resources.py", line 2027, in require working_set.resolve(self.dist.requires(self.extras),env,installer)) File "/tmp/tmpDIcsTL/distribute-0.6.38/pkg_resources.py", line 2237, in requires dm = self._dep_map File "/tmp/tmpDIcsTL/distribute-0.6.38/pkg_resources.py", line 2466, in _dep_map self.__dep_map = self._compute_dependencies() File "/tmp/tmpDIcsTL/distribute-0.6.38/pkg_resources.py", line 2499, in _compute_dependencies common = frozenset(reqs_for_extra(None)) File "/tmp/tmpDIcsTL/distribute-0.6.38/pkg_resources.py", line 2496, in reqs_for_extra if req.marker_fn(override={'extra':extra}): File "/tmp/tmpDIcsTL/distribute-0.6.38/_markerlib/markers.py", line 109, in marker_fn return eval(compiled_marker, environment) File "<environment marker>", line 1, in <module> NameError: name 'sys_platform' is not defined /home/galaxy/galaxy/database/tmp/tmp-toolshed-mtdFbEf6k/pyBamTools/distribute-0.6.38-py2.7.egg Traceback (most recent call last): File "setup.py", line 9, in <module> use_setuptools() File "/home/galaxy/galaxy/database/tmp/tmp-toolshed-mtdFbEf6k/pyBamTools/distribute_setup.py", line 152, in use_setuptools return _do_download(version, download_base, to_dir, download_delay) File "/home/galaxy/galaxy/database/tmp/tmp-toolshed-mtdFbEf6k/pyBamTools/distribute_setup.py", line 132, in _do_download _build_egg(egg, tarball, to_dir) File "/home/galaxy/galaxy/database/tmp/tmp-toolshed-mtdFbEf6k/pyBamTools/distribute_setup.py", line 123, in _build_egg raise IOError('Could not build the egg.') IOError: Could not build the egg.

1 0

why modified workflow doesn't save?
by Nikolaos Tur 30 Sep '16

30 Sep '16

Hi, after modification of workflow it does not save in workflow editor. Just shows saving bar for ever. In logs there are not so much info, just mentions that using version of tools are different from specified in this workflow. I work with "CloudMap Hawaiian ..." workflow. Galaxy is 16.04 with version of tools at June/July time. Workflow file is a text file and potentially can be modified by hands. In this case on what I have to pay attention to change it in text editor. Is there any tutorial for this? Thanx, Nikolaos.

2 1

Galaxy cannot restart : python module Crate required but installed with name "crate"
by julie dubois 30 Sep '16

30 Sep '16

Hello, I have a very big problem with a module I've been tried to install for galaxy. I've install Crate with this command pip install Crate But when i verify in sites-packages, and in a python prompt, this module is named with a "c" and not a "C", so Galaxy cannot restart!!!! Please help me . Julie

1 0

Question about Galaxy integration with external access control
by Simon Chang 30 Sep '16

30 Sep '16

Greetings, I am an IT systems administrator at our lab. We maintain some control over our servers but the bulk of the control resides with our university's research computing group. A while ago the admins at research computing created some LDAP-based user accounts for a project that we have, and I had read ( https://wiki.galaxyproject.org/Admin/Config/ExternalUserAuth) that Galaxy has ability to interface with the directory service, although there wasn't much details in the link here. So we have a few questions: 1) Assuming Galaxy can read LDAP directory service information, to what extent is access control enforced? Is it on a file system level? 2) If a researcher logs into Galaxy with his LDAP credentials, runs some analyses and obtains the results, how exactly are these results protected from other researchers who may be prohibited from accessing these results due to institutional policies? Accordingly, if a researcher wants to share the data product with another LDAP user, how is that done exactly apart from simply downloading and emailing it? Depending on the responses I receive from these questions I may have more follow-up questions. Thank you in advance for your consideration. Simon Chang

3 2

CleanUp Datastets ERROR since update from bitbucket to git
by julie dubois 30 Sep '16

30 Sep '16

Hi all, I've recently updated my local instance of Galaxy from bitbucket to git. It seems to work fine. No error when we run galaxy. But I've been tried to cleanup datasets and when I ran this command : python scripts/cleanup_datasets/cleanup_datasets_Redirection.py universe_wsgi.ini -d 0 -1 -r $@ I have error in python module like this : Traceback (most recent call last): File "scripts/cleanup_datasets/cleanup_datasets_Redirection.py", line 23, in <module> import galaxy.model.mapping File "/data/galaxy-dist/lib/galaxy/model/mapping.py", line 19, in <module> from galaxy.model.custom_types import JSONType, MetadataType, TrimmedString, UUIDType File "/data/galaxy-dist/lib/galaxy/model/custom_types.py", line 13, in <module> from galaxy import app File "/data/galaxy-dist/lib/galaxy/app.py", line 16, in <module> from galaxy.visualization.data_providers.registry import DataProviderRegistry File "/data/galaxy-dist/lib/galaxy/visualization/data_providers/registry.py", line 4, in <module> from galaxy.visualization.data_providers import genome File "/data/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 15, in <module> from bx.bbi.bigbed_file import BigBedFile ImportError: No module named bigbed_file I've already reinstall in galaxy virtual env lot of modules required as bx-python , pyyaml, pysam, webhelpers... But it's infinite and I don't understand why all python modules are not retrieve while the galaxy instance works fine and I've had no problem when I've updated the galaxy version. How can I manage this problem and eventually re-install python, python modules in virtual env ? Thanks a lot. Julie

1 0

Re: [galaxy-dev] Modifying a Cloud Cluster Instance
by Lapointe, David 29 Sep '16

29 Sep '16

This is on Amazon I should add. -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research Technology (RT) Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu From: galaxy-dev <galaxy-dev-bounces(a)lists.galaxyproject.org<mailto:galaxy-dev-bounces@lists.galaxyproject.org>> on behalf of David Lapointe <david.lapointe(a)tufts.edu<mailto:david.lapointe@tufts.edu>> Date: Thursday, August 11, 2016 at 1:44 PM To: "galaxy-dev(a)lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>" <galaxy-dev(a)lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>> Subject: [galaxy-dev] Modifying a Cloud Cluster Instance I have a Cloudman instance up and running, but I would like to add SSL ( I have a signed cert and key ) and productionize it. I see several methods documented,but it's not clear which one is preferred: edit in place then persist changes and restart (wiki/Cloudman/CustomizeGalaxyCloud) seems best to me. Any advice? Thanks David -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research Technology (RT) Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu

2 7

GCC2018: Call for Host Proposals
by Dave Clements 27 Sep '16

27 Sep '16

Hello all, The Galaxy Community Conference has been held annually since 2010. *GCC2017* <https://gcc2017.sciencesconf.org/> will be held 26-30 June in Montpellier, France. *We are now seeking proposals <https://wiki.galaxyproject.org/Documents?action=AttachFile&do=view&target=G…> from organizations interested in hosting GCC2018.* GCC alternates every year between meeting in North America, and meeting outside North America. Hosts for GCC2018 need to be located in North America. GCC draws 200+ participants from data-intensive life science research. Participants come from around the world, from all career stages, and do research spanning the tree of life. Universities, hospitals and medical schools, research organizations, and industry are all represented, including some of the largest and most influential research organizations in the world. What do you need to host GCC2018? In approximately decreasing order of importance: - *Enthusiasm to plan and organize several events over several days for more than 200 people. * - Space for 4-6 parallel training sessions, with each space able to accommodate 25 to 75 participants. - Central meeting space with capacity for 250-300 people. - Affordability (or access to sponsorship funds) - Nearby space for breakouts, poster sessions and sponsors. - Nearby space for lunch, coffee breaks. - Good wifi for all events (that's 200+ people and their devices). - Space for hackathons for 2 days - Easy to get to by air. - Nearby, affordable housing, or easy walking distance or easy public transport from lodging options to conference facilities - Close proximity to a pub and other social hubs. Hosts get copious (and enthusiastic) organizational support from Galaxy's Outreach Team (who have been helping to organize GCC since 2011), and even more copious gratitude from the Galaxy Community. You also get to highlight your institution to the Galaxy Community. See the *full call for host proposals <https://wiki.galaxyproject.org/Documents?action=AttachFile&do=view&target=G…>* for more details on what should be covered in proposals. *Proposals are due on November 30, 2016*. If you are interested in possibly hosting GCC2018 then please contact Dave Clements with any questions. Our goal is to announce GCC2018 at GCC2017 this coming June. Hoping to work with you in 2018! Dave Clements and the Galaxy Team -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

Re: [galaxy-dev] [galaxy-iuc] GMAP tool_dependencies.xml package for ToolShed
by Jim Johnson 27 Sep '16

27 Sep '16

I would greatly appreciate the IUC adopting all gmap/gsnap work that I originally worked on. I do not currently have the time to maintain these tools. Administrative access granted to IUC for: https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/ https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/ https://testtoolshed.g2.bx.psu.edu/view/jjohnson/gmap/ https://testtoolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/ These can be copied to the iuc owned repositories and I could deprecate those under my name. On 9/20/16 8:13 AM, Peter Cock wrote: > Thanks JJ :) > > Could you explicitly confirm that we (the IUC) have your permission > to adopt the gmap wrappers and packages (and any of your other > Galaxy work you might want to suggest), to develop and maintain > under the IUC banner, under the MIT license. > > (Ideally I'd suggest you state this in an email CC'ing the galaxy-dev > list as a public record) > > For now this will mean tracking their development in: > https://github.com/galaxyproject/tools-iuc > > Secondly, would you be willing to grant admin/write access > to the IUC account on the (main and test) tool shed for these: > > https://toolshed.g2.bx.psu.edu/view/jjohnson/gmap/ > https://toolshed.g2.bx.psu.edu/view/jjohnson/package_gmap_2013_05_09/ > > Thanks, > > Peter > > On Tue, Sep 20, 2016 at 12:47 PM, Jim Johnson <johns198(a)umn.edu> wrote: >> It would be great to move this to IUC. >> I originally put a lot of effort into this for a researcher at MN; I think >> it was tried once. >> I now use gmap as part a dependency of the deFuse application. >> Thanks, >> JJ -- James E. Johnson Minnesota Supercomputing Institute University of Minnesota

2 3