- galaxy-dev - lists.galaxyproject.org

Taking backups of my galaxy instance
by Nilaksha Neththikumara 09 May '14

09 May '14

Hi all, I have installed and currently running a local galaxy instance successfully, customized with desired tools and datasets but suddenly got realized what would I do if anything goes wrong and galaxy stop responding :O . Is there a specific way to take backups of a locally installed galaxy instance with its own settings, so i can perform that task regularly. Thanks in advance :) Sincerely, Nilaksha.

2 1

identify tool based on qstat name.
by Geert Vandeweyer 09 May '14

09 May '14

Hi, I would like to know what package/default tools is creating the jobs on our galaxy server like: 76905_CONVERTER_gff_to_fli_0_XXXXX(a)email.com I see these jobs in the pbs queue, and they use a lot of memory, up to 18Gb of Ram. Since I can't find the job in the tool configuration, I can't set the memory requirements in the job rules :-) Thanks, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1

Restarting toolshed when "run_tool_Shed.sh --daemon stop" doesn't work
by Dooley, Damion 09 May '14

09 May '14

I tried upgrading a local galaxy test site to the stable or default codebases. Most of galaxy seemed to work fine - and I can start and stop it. But the local toolshed I had running under it seems now to be jammed in an error state: Internal Server Error Galaxy was unable to successfully complete your request URL: http://..../toolshed/ Module galaxy.web.framework.middleware.error:149 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ ... >> __M_writer(filters.html_escape(unicode(trans.app.shed_counter.unique_valid_tools ))) AttributeError: 'ShedCounter' object has no attribute 'unique_valid_tools' And there are a handful of admin tab links like 'Manage form definitions' that yield an error like: Error Traceback: View as: Interactive | Text | XML (full) ⇝ AttributeError: 'FormsGrid' object has no attribute 'use_hide_message' URL: http://salk.bccdc.med.ubc.ca/galaxylab/forms/browse_form_definitions Module weberror.evalexception.middleware:364 in respond view ... Though I can stop/start the main galaxy site, I can't seem to stop/start the local toolshed site? (I accidentally started galaxy (and probably the toolshed) in root in the past but I thought I'd recovered from that). Any tips would be appreciated! Regards, Damion

1 0

Re: [galaxy-dev] Missing module in the lwr repository
by John Chilton 08 May '14

08 May '14

Hello Dr. Senturk, This is the second time I have gotten a question like this one in a week so I am going to cc galaxy-dev mailing in the response so there is a public link I can send out about progress in the future. There is some limited caching support so you don't have to re-transfer the same inputs over and over. There is no mechanism for periodically cleaning out this cache however and if the LWR is producing large outputs these need to be transferred back to the Galaxy server and re-uploaded once before they are added to the cache. I have sketched out some plans for improving some of these limitations here - https://trello.com/c/MPlt8DHJ, I consider these important features but I am still not sure how quickly I will be able to get to these issues though. If you are interested in testing out caching, I have added some documentation to Galaxy's sample job_conf.xml file how to enable caching on the client ( https://bitbucket.org/galaxy/galaxy-central/commits/d6826c29e15a1f17553ae2f… ) and on the server side you simply need to specify a directory to store cached files by un-commenting the property file_cache_dir in LWR's server.ini configuration file. As experimental as I consider the feature - it was a big help for some particular and large workflows when I was in my previous position and it allowed us to really scale up what we were doing the LWR and Galaxy. Hope this helps and thanks for your continued interest in the LWR and Galaxy! -John On Thu, May 8, 2014 at 6:54 AM, izzet fatih <ifs005(a)gmail.com> wrote: > Hi John, > I'm new in Galaxy so please correct me if I'm wrong. As far as I > understand, Galaxy dispatches jobs by /lib/galaxy/jobs/handler.py one by > one even though they are part of a workflow. Consequently, LWR receives > them separately. Let's have two consecutive steps to be run by LWR. If the > data size is big, this will cause excessive data transfer burden between > the steps. Do you have any plans to improve this? > > Thanks, > > > On Wed, Apr 30, 2014 at 4:14 PM, John Chilton <jmchilton(a)gmail.com> wrote: > >> Doh - sorry about that. It should be fixed with this commit - >> https://bitbucket.org/jmchilton/lwr/commits/5ebad638537e19b55b7b6c812c49ae9… >> . >> >> Thanks for your interest in the LWR and let me know if there is anything >> else I can do to fix it, improve it, etc... >> >> -John >> >> >> On Wed, Apr 30, 2014 at 1:19 PM, izzet fatih <ifs005(a)gmail.com> wrote: >> >>> Hi John, >>> I've downloaded and configured lwr server and it is great tool thanks >>> for developing it. >>> When I execute run.sh I got the following error >>> >>> File >>> "/home/isenturk/src/jmchilton-lwr-bd8d1cc44e75/lwr/managers/base/directory.py", >>> line 13, in <module> >>> from ..util.env import env_to_statement >>> ImportError: No module named env >>> >>> This exception is thrown from >>> jmchilton-lwr-bd8d1cc44e75/lwr/managers/base/*directory.py* >>> It is trying to use env_to_statement command from ..util.env >>> >>> If I am not mistaken, the module is supposed to be under >>> jmchilton-lwr-bd8d1cc44e75/lwr/managers/util/ >>> However, it is not there. >>> >>> Thanks, >>> >>> >>> -- >>> [image: The Ohio State University] >>> Izzet F Senturk, PhD >>> Post Doctoral Researcher >>> College of Medicine SBS-Biomedical Informatics >>> 310-05 Lincoln Tower, 1800 Cannon Drive, Columbus, OH 43210 >>> 6148250565 Office >>> senturk.2(a)osu.edu http://bmi.osu.edu/hpc/ >>> >> >> > > > -- > [image: The Ohio State University] > Izzet F Senturk, PhD > Post Doctoral Researcher > College of Medicine SBS-Biomedical Informatics > 310-05 Lincoln Tower, 1800 Cannon Drive, Columbus, OH 43210 > 6148250565 Office > senturk.2(a)osu.edu http://bmi.osu.edu/hpc/ >

1 0

user information form
by Sarah Diehl 08 May '14

08 May '14

Hi everyone, is anybody using the user information form feature (https://wiki.galaxyproject.org/Admin/Config/User%20Information)? For me it does not seem to be working. I created a form as explained on the wiki page, but nothing changed in the user registration. It just causes an internal server error when I go to "Manage your information". Best regards, Sarah

1 0

xml file display problem
by Hans-Rudolf Hotz 08 May '14

08 May '14

Hi This is long shot, but maybe someone can help us.... We are in the process of upgrading our production galaxy server from ("release_2013.11.04" to "release_2014.04.14"). Despite some "hiccups" it went very smooth (I might come back with those in a different mail thread next week). However, we are running into a display problem: A former co-worker has written a tool which generates a big xml file. When clicking on the 'eye' icon, we don't wan't display the complete file, but only part of it. For this he has written an xsl file ("qProject.xsl"). This file is placed in ~/galaxy-dist/static/ Correspondingly, the beginning of the xml file looks like: <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="../../../static/qProject.xsl"?> <qProject version="0.1"> etc. And (at least for me by magic), when you now click on the 'eye' icon a nicely formatted output was displayed, instead of the full xml file. This has been the case with the "release_2013.11.04" distribution. Now with the "release_2014.04.14" distribution (and also already with "release_2014.02.10"), this magic trick does not work anymore, and the complete, un-formatted xml file is displayed. I am aware, there has been major changes introduced for the displays with the "release_2014.02.10" distribution. So, can anybody give me some hints how to get this working again. I am happy to provide more details, if required. Thank you very much Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland

2 2

galaxy on ubuntu 14.04: hangs on metadata cleanup
by Jorrit Boekel 08 May '14

08 May '14

Dear all, Has anyone tried running Galaxy on Ubuntu 14.04? I’m trying a test setup on two virtual machines (worker+master) with a SLURM queue. Getting in strange problems when jobs finish, the master hangs, completely unresponsive with CPU at 100% (as reported by virt-manager, not by top). Only drmaa jobs seem to be affected. After hanging, a reboot shows the job is finished (and green in history). It took me some debugging to figure out where things go wrong, but it seems it goes wrong when os.remove is called in lib/galaxy/datatypes/metadata.py in method cleanup_external_metadata. I can reproduce the problem by calling os.remove(metadatafile) by hand (in an interactive python shell) when using pdb to create a breakpoint just before the call. If I comment out the os.remove it runs on until it hits another delete call in lib/galaxy/jobs/__init__.py: self.app.object_store.delete(self.get_job(), base_dir='job_work', entire_dir=True, dir_only=True, extra_dir=str(self.job_id)) It’s in the JobWrapper class in the cleanup() method. I should mention here that my galaxy version is a bit old since I’m running my own fork with local modifications on datatypes. This object_store.delete also leads to a shutil.rmtree and os.remove function. So, remove calls to the filesystem seem to hang the whole thing, but only at this point in time. Rebooting and removing by hand is no problem, pdb-stepping also sometimes fixes it (but if I just press continue it hangs). I don’t know where to go from here with debugging, but has anyone seen anything similar? Right now it feels like it may be caused by timing rather than actual code problems. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden

1 2

Re: [galaxy-dev] Correct toolshed tool config for a .loc file
by Dooley, Damion 08 May '14

08 May '14

One further piece of the mystery I see now: The "View Data tables Registry" shows my tool's loc files, but has odd stuff for "Tool data path" and "Missing index file" columns, for example: blast_reporting_fields [tab] /usr/local/galaxy/test/galaxy-dist/tool-data/salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/blast_reporting/00d397a381f8/blast_reporting_fields.loc [tab] /usr/local/galaxy/test/galaxy-dist/tool-data ./database/tmp/tmp-toolshed-gmfcrDITxyK/blast_reporting_fields.loc [tab] ./database/tmp/tmp-toolshed-gmfcrDITxyK [tab] missing There is no galaxy-dist/database/tmp-toolshed-gmfcrDITxyK folder, if that's where its supposed to be? Is there some setting for database tmp folder cache for tools? Thanks for tips ... Damion

1 2

setup_virtualenv action doesn't update PATH
by Rodrigo Garcia 07 May '14

07 May '14

Hello List, I'm writing a tool_dependencies.xml file for a python package. It defines an action type="setup_virtualenv", and when I install it to my galaxy from the test shed it is actually installed into ~/environments, and the tool definition is set up correctly. This package includes a callable script, when I install the package on a virtualenv using pip, the path to the CLI tool is inserted to the PATH and I can call it anywhere. But when I run the tool from galaxy it cannot find the script. Does the setup_virtualenv action also update the PATH? Thanks! Rodrigo

1 0

Correct toolshed tool config for a .loc file
by Dooley, Damion 07 May '14

07 May '14

I'll try to keep this short! I'm wondering why on a fresh install of galaxy via a setup script into a python virtualenv, my custom toolshed tool's .loc files don't seem to be loading, i.e. the tool's tool_data_table_conf.xml file isn't loading. All the other parts of the tool (and other toolshed tools) do run ok. On the galaxy install where the tool was developed, it runs ok too, and I can't see any universe.wsgi.conf differences that would account for this though I'm sure the dev server has a number of other different config settings. I thought I knew my way around this .loc stuff, but I'm flummoxed. It is as though /usr/local/galaxy/test/galaxy-dist/tool-data/[toolshed domain etc etc]/00d397a381f8/tool_data_table_conf.xml isn't loading into Galaxy's tabular data list?! The only other thing I can think of is that I stuffed the tool-data related loc files into a subfolder of the tool blast_reporting.py blast_reporting.xml tool_data_table_conf.xml.sample ... tool-data/ blast_reporting_fields.loc.sample But the resulting downloaded tool config in the galaxy-dist/tool-data/..... /blast_reporting_fields.loc and tool_data_table_conf.xml files seem just fine!!! Regards, Damion

1 0