March 2017 - galaxy-dev - lists.galaxyproject.org

Newline characters for txt files
by Lawrence, Cris W 14 Mar '17

14 Mar '17

I have observed that when I upload a file with CRLF line endings into Galaxy and then later download it, the endings have changed to UNIX (LF only). I also observed that an uploaded file containing CR only line endings in Galaxy downloads to a file again with LF only line endings. These file types are txt. Just want to confirm - does Galaxy scrub line endings? - Cris

2 1

installation error on venv dependencies.xml
by Linda Bakker 14 Mar '17

14 Mar '17

Hi All, I am having some problems with my tool dependencies to setup a python venv. Dependencies.xml looks like this: When i install my tool from the toolshed, it fails to install the venv dependency with this error message: I am not sure what the problem exactly seems to be. os.py should be a default python package right or do i need to specify it in the dependencies file? Can someone point me in the right direction of what the problem might be. Thanks! with regards , Linda Bakker -- Linda Bakker Bioinformatics Research Assistant Plant Research International, Wageningen UR Building 107 (Radix), room W1.Fa.054 Droevendaalsesteeg 1, 6708 PB, Wageningen, The Netherlands tel: +31-317-480756 email: linda.bakker(a)wur.nl

2 1

Search Tool Shed
by Matthias Bernt 09 Mar '17

09 Mar '17

Dear mailing list, What is the expected behavior if I click in the Admin area under "Search Tool Shed" on the "Galaxy Main Tool Shed" button (or any of the Browse .. or Search .. actions that are available via the black triangle submenu)? I remember that a new tab with the corresponding Tool Shed page opened in the browser. This is currently not happening in my installation. What could be the problem? As a work around I can trigger the Browse action via right click and "open link in new tab". As far as I can see this is not a JavaScript problem. Best, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

2 3

Re: [galaxy-dev] Shared Data Library, large number of files
by Hans-Rudolf Hotz 08 Mar '17

08 Mar '17

this sounds more like a browser issue to me which browser do you use? what happens when you use a different browser? and what happens when you just reload the page Hans-Rudolf On 03/01/2017 06:17 PM, D K wrote: > Actually, when I click on the Shared Data-> Data Libraries link I get > only the Galaxy banner and a blank page, so I'm not even seeing the data > libraries themselves. > > If I use the Admin link and then click Data libraries, I can see the > actual libraries. But when I try to enter one of the libraries with lots > of files it hangs. > > > On Wed, Mar 1, 2017 at 9:03 AM, D K <danielfortin86(a)gmail.com > <mailto:danielfortin86@gmail.com>> wrote: > > Hi Hans, > > I'm running 16.10. If I'm not mistaken, the old Data Library > interface is still accessible in the Admin menu isn't it? I have the > same issue when navigating to a Data Library either from the Admin > or Shared Data links. I also tried this in 16.07 with the same > result. If it makes a difference, I'm using PostgreSQL and Apache. > > Dwight > > > > On Tue, Feb 28, 2017 at 11:47 PM, Hans-Rudolf Hotz <hrh(a)fmi.ch > <mailto:hrh@fmi.ch>> wrote: > > Hi Daniel > > Just double checking which version of galaxy are you running? In > other words: are you still using the old Data Library interface? > (if I remember correctly, the old UI was deprecated in 15.10) > We had similar issues with the old UI. > > The new one works fine and is very fast. E.g. we have a 'MISEQ' > folder with more than 100 sub folders and a HISEQ folder with > more than 600 sub folders. Each containing up to 100plus fastq > and text files. > > Regards, Hans-Rudolf > > > > On 03/01/2017 12:13 AM, D K wrote: > > Hi, > > I've noticed that the more files / folders (think hundreds > of folders > each with 20+ fastq files) I add to a Shared Data Library, > the longer > and longer it takes to initially navigate the folder > structure (after > clicking on the specific Shared Data Library). Just opening > up the > Shared Data Library can take several minutes and often the > browser times > out. Is there a way to speed this up? > > If it's relevant, when I add shared data, it's linked rather > than copied > into Galaxy. Any suggestions would be greatly appreciated > > Thanks! > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > <https://lists.galaxyproject.org/> > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > <http://galaxyproject.org/search/mailinglists/> > > >

4 7

Rg: Galaxy Version
by Anamika Singh 08 Mar '17

08 Mar '17

Dear galaxy team, I am using galaxy platform (old version) in Linux OS which was not installed by me. I would like to know 'how to check what is the galaxy version installed currently in my system. Regards -- Anamika

5 5

tool problems (data_manager_fetch_ncbi_taxonomy & ggplot2)
by Matthias Bernt 08 Mar '17

08 Mar '17

Dear mailing list, I'm experimenting with the tools available at the Tool Shed and ran into problems with two of the tools. 1. data_manager_fetch_ncbi_taxonomy If I run the data manager I get an error in the browser: "Uncaught exception in exposed API method" The terminal shows the following stack trace: Traceback (most recent call last): File "/home/berntm/galaxy/lib/galaxy/web/framework/decorators.py", line 282, in decorator rval = func( self, trans, *args, **kwargs ) File "/home/berntm/galaxy/lib/galaxy/webapps/galaxy/api/tools.py", line 98, in build return tool.to_json(trans, kwd.get('inputs', kwd)) File "/home/berntm/galaxy/lib/galaxy/tools/__init__.py", line 1853, in to_json tools = self.app.toolbox.get_loaded_tools_by_lineage( self.id ) File "/home/berntm/galaxy/lib/galaxy/tools/toolbox/base.py", line 492, in get_loaded_tools_by_lineage tool_lineage = self._lineage_map.get( tool_id ) File "/home/berntm/galaxy/lib/galaxy/tools/toolbox/lineages/factory.py", line 38, in get lineage = ToolShedLineage.from_tool_id( self.app, tool_id ) File "/home/berntm/galaxy/lib/galaxy/tools/toolbox/lineages/tool_shed.py", line 70, in from_tool_id return ToolShedLineage( app, tool_version ) File "/home/berntm/galaxy/lib/galaxy/tools/toolbox/lineages/tool_shed.py", line 53, in __init__ self.tool_version_id = tool_version.id File "/home/berntm/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 237, in __get__ return self.impl.get(instance_state(instance), dict_) File "/home/berntm/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 578, in get value = state._load_expired(state, passive) File "/home/berntm/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/state.py", line 474, in _load_expired self.manager.deferred_scalar_loader(self, toload) File "/home/berntm/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", line 610, in load_scalar_attributes (state_str(state))) DetachedInstanceError: Instance <ToolVersion at 0x7fc7180b5310> is not bound to a Session; attribute refresh operation cannot proceed Any ideas whats going on - and how to fix this? 2. The second tool is ggplot2_2_2_1. Here mingchen0919 is listed as owner in the Tool Shed -- so I do not know if this mailing list is the correct address. If not: how do I find contact information to the owner of a tool in the ToolShed? Otherwise, the problem is missing dependencies: In package_fontconfig_2_11_1 (which is a dependency of ggplot) I get """ configure: error: in `/home/berntm/galaxy/database/tmp/tmp-toolshed-mtdBsL0ig/fontconfig-2.11.1': configure: error: The pkg-config script could not be found or is too old. Make sure it is in your PATH or set the PKG_CONFIG environment variable to the full path to pkg-config. Alternatively, you may set the environment variables FREETYPE_CFLAGS and FREETYPE_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. To get pkg-config, see <http://pkg-config.freedesktop.org/>. See `config.log' for more details """ Is pkg-config supposed to be installed on the computer running galaxy? Or should this be installed within galaxy? In addition on the terminal the following error appears after the pkg-config error: """ Exception happened during processing of request from ('127.0.0.1', 42534) Traceback (most recent call last): File "/home/berntm/galaxy/.venv/local/lib/python2.7/site-packages/paste/httpserver.py", line 1085, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.7/SocketServer.py", line 654, in __init__ self.finish() File "/usr/lib/python2.7/SocketServer.py", line 713, in finish self.wfile.close() File "/usr/lib/python2.7/socket.py", line 283, in close self.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 32] Broken pipe """ Thank you very much for your help. Best regards, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

1 0

March 2017 News of the Galaxy
by Dave Clements 07 Mar '17

07 Mar '17

1 0

SAML Authentication
by Keith Suderman 04 Mar '17

04 Mar '17

Is anyone using SAML for authentication? There is a feature request for SAML support on GitHub ( https://github.com/galaxyproject/galaxy/issues/3146) but it doesn't look like any work has been done. We need to add the ability for our users to authenticate with a SAML identity provider (IdP), in particular with InCommon/EduRoam/EduCause et al. Looking at the documentation there appears to be "the old way" (configuring Apache/Nginx to do the authentication) and "the new way" where Galaxy handles the authentication. Is it correct to assume that to use the new way I should implement an external authenticator, something like galaxy/auth/providers/saml.py and then some sort of magic to get a config/auth_conf.xml working? I just want to make sure I am heading down the correct path before investing too much time. - Keith Keith Suderman Research Associate Department of Computer Science Vassar College, Poughkeepsie NY suderman(a)cs.vassar.edu

3 3

Deploy on cloud
by Stavros 0 Sachtouris 03 Mar '17

03 Mar '17

Hi all, we are going to deploy galaxy on ~okeanos cloud ( okeanos.grnet.gr ). We have already developed a working storage backend for our object store system ( https://github.com/galaxyproject/galaxy/pull/3611 ). Now we focus on how to make Galaxy run instances/tasks on our VMs and I would like some advice on how to approach this issue. What we need is a (potentially large) Galaxy deployment with the ability to run workflows of dockerized "tasks". Each task should run on a provisioned VM on our cloud. There are many tools out there to let us automate VM provisioning, managing docker clusters, scaling, as well as monitoring and accounting, and we want to provide all of these features to our partners in European Academia. The question is, how does Galaxy communicate with a cluster manager like Apache Mesos or Docker swarm? Any other tools I should look into, which are known to work with Galaxy? Will we have to drill into Galaxy code base to provide these features? Thank you in advance for you suggestions, Stavros Sachtouris developer @ www.grnet.gr

2 1

Problem with installing tools using Conda
by Ashok Varadharajan 03 Mar '17

03 Mar '17

Dear Team, I have problem with the installation of tools using conda. I am getting following error message when i try to install tools from galaxy. I tried to install these tools manually with the same conda used by the galaxy and it worked. Can anyone help us to solve this issue ? Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,759 Removing failed conda install of samtools, version '1.3.1' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,941 Removing failed conda install of samtools, version 'None' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:50,142 Removing failed conda install of bwa, version '0.7.15' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main We are running galaxy-16.07 behind the proxy using nginx, uwsgi and supervisor in scientific linux 7. Below are the details of galaxy.ini and supervisor configuration. *### galaxy.ini * conda_prefix = /xxxx/xxxxx/xxxx-xxxx/conda/ #conda_exec = conda_debug = False conda_ensure_channels = r,bioconda,iuc #conda_auto_install = False #conda_verbose_install_check=False conda_auto_init = True *### supervisor* [supervisord] [program:galaxy_web] command = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = $GALAXY_PATH umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so numprocs = 1 stoplight = INT startretries = 15 process_name = web%(process_num)s [program:handler] command = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/ >galaxy_handler%(process_num)s.log directory = $GALAXY_PATH process_name = handler%(process_num)s numprocs = 3 umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so startretries = 15 -- *Thanks and Regards,* *Ashok Vardharajan*

4 6