February 2016 - galaxy-dev - lists.galaxyproject.org

Restricting tool input to 1 dataset instead of multiple
by Ryan G 24 Feb '16

24 Feb '16

Hi all - I'm creating a tool in Galaxy that takes as input several datasets for different variables. For each variable, I want to restrict the input to a single dataset. The docs, https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax, indicate type="data" needs to be set for a dataset from the current history to be available for input. When I do this, Galaxy displays the icon for Single Dataset, Mulitple Datasets, and Dataset collection. I only want to give the option for Single Dataset. I tried include multiple="false", but that doesn't seem to have any effect. Is there a way to accomplish this?

2 1

Re: [galaxy-dev] cleanup
by Nicola Soranzo 23 Feb '16

23 Feb '16

Hi Philipp, please alway reply to the mailing list. I'm not using Docker, but if Galaxy is in the /galaxy-central/ directory and its version is 16.01, then: $ cd /galaxy-central/ $ . .venv/bin/activate $ python scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d -30 -1 $ deactivate The command ". .venv/bin/activate" will activate the virtualenv installed in /galaxy-central/.venv/ , "deactivate" will deactivate it. Cheers, Nicola On 23/02/16 19:14, Rathert, Philipp, Dr. wrote: > Hi Nicola, > Could you be a bit more specific? > I guess i activate this in the specific container but how exactly? > > Thanks a lot for your help. > > > Cheers, > > Philipp > ---Philipp on the road--- > > Am 23.02.2016 um 16:20 schrieb Nicola Soranzo <nsoranzo(a)tiscali.it > <mailto:nsoranzo@tiscali.it>>: > >> Hi Philipp, >> if you are using Galaxy release 16.01, you should load the virtualenv >> before running the cleanup script, e.g.: >> >> $ cd GALAXY_HOME >> $ . .venv/bin/activate >> $ python scripts/cleanup_datasets/cleanup_datasets.py ... >> >> Cheers, >> Nicola >> >> On 23/02/16 15:07, Hans-Rudolf Hotz wrote: >> > Hi Philipp >> > >> > >> > point 1 : that's as expected - good. >> > >> > point 2 : looks like a docker specific issue? Maybe Bjoern can jump in? >> > >> > >> > Hans-Rudolf >> > >> > >> > >> > >> > On 02/23/2016 12:17 PM, Rathert, Philipp, Dr. wrote: >> >> Dear Hans-Rudolf, >> >> >> >> thanks a lot for your help. >> >> >> >> I did as you suggested: >> >> >> >> 1. when I go into the scripts folder I get the error message you >> >> guessed: >> >> root@galaxy:/galaxy-central/scripts# python >> >> cleanup_datasets/cleanup_datasets.py >> >> /export/galaxy-central/config/galaxy.ini -d -30 -1 >> >> Traceback (most recent call last): >> >> File "cleanup_datasets/cleanup_datasets.py", line 21, in <module> >> >> import galaxy.config >> >> ImportError: No module named galaxy.config >> >> >> >> >> >> 2. from the root directory I recieve the following error message: >> >> root@galaxy:/galaxy-central# python >> >> scripts/cleanup_datasets/cleanup_datasets.py >> >> /export/galaxy-central/config/galaxy.ini -d -60 -1 >> >> Traceback (most recent call last): >> >> File "scripts/cleanup_datasets/cleanup_datasets.py", line 21, in >> >> <module> >> >> import galaxy.config >> >> File "/galaxy-central/lib/galaxy/config.py", line 21, in <module> >> >> from galaxy.util.dbkeys import GenomeBuilds >> >> File "/galaxy-central/lib/galaxy/util/dbkeys.py", line 7, in >> >> <module> >> >> from galaxy.util.object_wrapper import sanitize_lists_to_string >> >> File "/galaxy-central/lib/galaxy/util/object_wrapper.py", line >> >> 45, in >> >> <module> >> >> from six.moves import UserDict >> >> ImportError: cannot import name UserDict >> >> >> >> Any idea what this means? I thought this should be automatically >> >> installed with python... >> >> >> >> Cheers, >> >> >> >> Philipp >> >> >> >> -----Original message----- >> >> *From:* Hans-Rudolf Hotz <hrh(a)fmi.ch <mailto:hrh@fmi.ch>> >> >> *Sent:* Monday 22nd February 2016 17:09 >> >> *To:* Rathert, Philipp, Dr. >> <philipp.rathert(a)ibc.uni-stuttgart.de >> <mailto:philipp.rathert@ibc.uni-stuttgart.de>>; >> >> galaxy-dev(a)lists.galaxyproject.org >> <mailto:galaxy-dev@lists.galaxyproject.org> >> >> *Subject:* Re: [galaxy-dev] cleanup >> >> >> >> Hi Philipp >> >> >> >> > Since the new galaxy version, I am unable to perform a cleanup >> >> anymore. >> >> > >> >> > From the galaxy root directory I want to start the script >> >> > cleanup_datasets.py with the following line: >> >> > >> >> > $ python cleanup_datasets.py >> >> /export/galaxy-central/config/galaxy.ini -d >> >> > -60 -1 >> >> > >> >> > I get the error message: >> >> > >> >> > Can’t open file ‘cleanup_datasets.py’: no such file or >> directory. >> >> >> >> Have you tried: >> >> >> >> $ python scripts/cleanup_datasets/cleanup_datasets.py >> >> /export/galaxy-central/config/galaxy.ini -d -60 -1 >> >> >> >> > But when I go into the folder in which the script is located >> >> it doesn’t >> >> > work as well. >> >> >> >> what is the error this time? Is it "ImportError: No module named >> >> galaxy"? >> >> >> >> >> >> Regards, Hans-Rudolf >> >> >> >> >> >> > For your information: we are running the latest >> >> docker-galaxy-stable >> >> > version. >> >> > >> >> > I hope somebody has an idea. >> >> > >> >> > Cheers, >> >> > >> >> > Philipp >> >> > >> >> > ---------------------------------------------- >> >> > >> >> > Philipp Rathert, PhD >> >> > >> >> > Institute of Biochemistry >> >> > >> >> > University Stuttgart >> >> > >> >> > Pfaffenwaldring 55 >> >> > >> >> > D-70569 Stuttgart >> >> > >> >> > Germany >> >> > >> >> > Tel: +49 711 685 64388 >> >> > >> >> > Fax: +49 711 685 64392 >> >> > >> >> > philipp.rathert(a)ibc.uni-stuttgart.de >> <mailto:philipp.rathert@ibc.uni-stuttgart.de> >> >> > <mailto:philipp.rathert@ibc.uni-stuttgart.de> >> >> > >> >> > http://www.ibc.uni-stuttgart.de/ >> >> > >> >> > >> http://www.ibc.uni-stuttgart.de/mitarbeiter/Rathert/?__locale=en >> >> > >> >> > >> >> > >> >> > ___________________________________________________________ >> >> > Please keep all replies on the list by using "reply all" >> >> > in your mail client. To manage your subscriptions to this >> >> > and other Galaxy lists, please use the interface at: >> >> > https://lists.galaxyproject.org/ >> >> > >> >> > To search Galaxy mailing lists use the unified search at: >> >> > http://galaxyproject.org/search/mailinglists/ >> >> > >> >> >> > >> > ___________________________________________________________ >> > Please keep all replies on the list by using "reply all" >> > in your mail client. To manage your subscriptions to this >> > and other Galaxy lists, please use the interface at: >> > https://lists.galaxyproject.org/ >> > >> > To search Galaxy mailing lists use the unified search at: >> > http://galaxyproject.org/search/mailinglists/ >>

2 1

correctly installing to toolshed
by Xianrong Wong 23 Feb '16

23 Feb '16

Hi, I'm new to publishing tools/repository on the toolshed. I have used planemo and lint and tests as well as serve all worked. I was able to use the tool. However, when I uploaded it onto the test tool shed and imported the tool on a cloud instance, I keep getting this error when running job: Fatal error: Exit code 2 () Can't open perl script "rm_consecutive_sameScore.pl": No such file or directory Please advise! I've been downloading and looking at what others did but I still can't figure this out. I've attached my .xml file and here are some of the commands I used to upload the tool. planemo shed_create --shed_target testtoolshed

3 5

Re: [galaxy-dev] cleanup
by Hans-Rudolf Hotz 23 Feb '16

23 Feb '16

Hi Philipp point 1 : that's as expected - good. point 2 : looks like a docker specific issue? Maybe Bjoern can jump in? Hans-Rudolf On 02/23/2016 12:17 PM, Rathert, Philipp, Dr. wrote: > Dear Hans-Rudolf, > > thanks a lot for your help. > > I did as you suggested: > > 1. when I go into the scripts folder I get the error message you guessed: > root@galaxy:/galaxy-central/scripts# python > cleanup_datasets/cleanup_datasets.py > /export/galaxy-central/config/galaxy.ini -d -30 -1 > Traceback (most recent call last): > File "cleanup_datasets/cleanup_datasets.py", line 21, in <module> > import galaxy.config > ImportError: No module named galaxy.config > > > 2. from the root directory I recieve the following error message: > root@galaxy:/galaxy-central# python > scripts/cleanup_datasets/cleanup_datasets.py > /export/galaxy-central/config/galaxy.ini -d -60 -1 > Traceback (most recent call last): > File "scripts/cleanup_datasets/cleanup_datasets.py", line 21, in <module> > import galaxy.config > File "/galaxy-central/lib/galaxy/config.py", line 21, in <module> > from galaxy.util.dbkeys import GenomeBuilds > File "/galaxy-central/lib/galaxy/util/dbkeys.py", line 7, in <module> > from galaxy.util.object_wrapper import sanitize_lists_to_string > File "/galaxy-central/lib/galaxy/util/object_wrapper.py", line 45, in > <module> > from six.moves import UserDict > ImportError: cannot import name UserDict > > Any idea what this means? I thought this should be automatically > installed with python... > > Cheers, > > Philipp > > -----Original message----- > *From:* Hans-Rudolf Hotz <hrh(a)fmi.ch> > *Sent:* Monday 22nd February 2016 17:09 > *To:* Rathert, Philipp, Dr. <philipp.rathert(a)ibc.uni-stuttgart.de>; > galaxy-dev(a)lists.galaxyproject.org > *Subject:* Re: [galaxy-dev] cleanup > > Hi Philipp > > > Since the new galaxy version, I am unable to perform a cleanup anymore. > > > > From the galaxy root directory I want to start the script > > cleanup_datasets.py with the following line: > > > > $ python cleanup_datasets.py /export/galaxy-central/config/galaxy.ini -d > > -60 -1 > > > > I get the error message: > > > > Can’t open file ‘cleanup_datasets.py’: no such file or directory. > > Have you tried: > > $ python scripts/cleanup_datasets/cleanup_datasets.py > /export/galaxy-central/config/galaxy.ini -d -60 -1 > > > But when I go into the folder in which the script is located it doesn’t > > work as well. > > what is the error this time? Is it "ImportError: No module named galaxy"? > > > Regards, Hans-Rudolf > > > > For your information: we are running the latest docker-galaxy-stable > > version. > > > > I hope somebody has an idea. > > > > Cheers, > > > > Philipp > > > > ---------------------------------------------- > > > > Philipp Rathert, PhD > > > > Institute of Biochemistry > > > > University Stuttgart > > > > Pfaffenwaldring 55 > > > > D-70569 Stuttgart > > > > Germany > > > > Tel: +49 711 685 64388 > > > > Fax: +49 711 685 64392 > > > > philipp.rathert(a)ibc.uni-stuttgart.de > > <mailto:philipp.rathert@ibc.uni-stuttgart.de> > > > > http://www.ibc.uni-stuttgart.de/ > > > > http://www.ibc.uni-stuttgart.de/mitarbeiter/Rathert/?__locale=en > > > > > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > > >

2 1

Galaxy 16.01, eggs vs wheels, and running jobs as the user
by Peter Cock 23 Feb '16

23 Feb '16

Hi all, Last week at the Galaxy Admins video hangout Nate gave an overview of the way Python dependency handling is changing from using egg files to using wheel files instead - slide links at: https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2016_02_18 I understand the Galaxy v16.01 (2016 January feature freeze) release which switches from eggs to wheels is due out shortly - hopefully this week? Recently we've been trying to setup a replacement Galaxy instance where jobs are submitted to our SGE cluster as the linux user requesting the job. This has been a bumpy road, e.g. http://dev.list.galaxyproject.org/Python-on-cluster-for-setting-metadata-af… Right now we've run into a problem with the metadata scripts trying to access ~/.python-eggs and going back over the archives it looks like we may need to set $PYTHON_EGG_CACHE to a temp folder on a per-node per-user basis, see e.g. http://dev.list.galaxyproject.org/python-egg-cache-exists-error-tt4656276.h… Sadly the wiki does not cover this aspect of running jobs as the real user (but the egg settings are about to be obsolete): https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_… Clearly with Galaxy v16.01 the eggs and $PYTHON_EGG_CACHE will go away. What happens with wheels, Python virtual environments, and running jobs as the real user? Thanks, Peter

2 3

Docker and issue with shed_tool_conf.xml
by Rathert, Philipp, Dr. 23 Feb '16

23 Feb '16

Dear All, I have a serious problem with installing new tools to our local Docker-galaxy-stable instance (latest version). All tools seem to install without any issue from the tool shed. When I check under Manage installed tools they all show a green "installed" button. However, the local tool shed is not updated anymore. none of the newly installed tools show up. It seems the shed_tool_conf.xml file is not updated. the tools are all found in the folder shed_tools but not under tool-data, not sure if this means anything. A work-around could be to manually enter these tools to the shed_tool_conf.xml file but to do this I wasn't able to find the guid for each tool. All other parameters are easy. But this would be quite some effort and I am afraid that something is seriously wrong. It would be great to get your thoughts on this issue before I kill everything and start from scratch. Thanks a lot. Cheers, Philipp PS: I am a wet lab scientist please be patient if I don't get everything immediately.

1 0

Plans for workflow & parallelisation work?
by Peter van Heusden 22 Feb '16

22 Feb '16

Hi there I see from the PR landing in Galaxy and the comments on things like issue #1701 (https://github.com/galaxyproject/galaxy/issues/1701) that there's lots of work happening on the workflow side of Galaxy. This is an area of interest at SANBI too, so we'd like to coordinate development efforts as much as possible. To this end: 1) Are there forks to track so we can see what new code is landing? 2) Is there a roadmap for workflow work or perhaps can we have a Hangout to talk about this? 3) Specifically in terms of workflows and parallelisation: are there any plans to work on running workflows as opposed to just generating lots of jobs? I know this is a major change to how Galaxy works - it would mean something like submitting a workflow specification to a job runner that is located on the cluster, and then returning the results of workflow execution. 4) Currently parallelisation in Galaxy is supported using two mechanisms: collections and dataset splitters/tasks. Are there plans on extending and harmonising Galaxy's parallelisation capabilities? Thanks, Peter

3 2

cleanup
by Philipp Rathert 22 Feb '16

22 Feb '16

Dear All, Since the new galaxy version, I am unable to perform a cleanup anymore. >From the galaxy root directory I want to start the script cleanup_datasets.py with the following line: $ python cleanup_datasets.py /export/galaxy-central/config/galaxy.ini -d -60 -1 I get the error message: Can't open file 'cleanup_datasets.py': no such file or directory. But when I go into the folder in which the script is located it doesn't work as well. For your information: we are running the latest docker-galaxy-stable version. I hope somebody has an idea. Cheers, Philipp ---------------------------------------------- Philipp Rathert, PhD Institute of Biochemistry University Stuttgart Pfaffenwaldring 55 D-70569 Stuttgart Germany Tel: +49 711 685 64388 Fax: +49 711 685 64392 <mailto:philipp.rathert@ibc.uni-stuttgart.de> philipp.rathert(a)ibc.uni-stuttgart.de <http://www.ibc.uni-stuttgart.de/> http://www.ibc.uni-stuttgart.de/ <http://www.ibc.uni-stuttgart.de/mitarbeiter/Rathert/?__locale=en> http://www.ibc.uni-stuttgart.de/mitarbeiter/Rathert/?__locale=en

2 1

JBrowse on dev branch: progress
by Peter van Heusden 22 Feb '16

22 Feb '16

Hi there Eric and others I tried to install the JBrowse tool on a dev branch server running on an Ubuntu 14.04 VM. It got stuck at the package_gnu_coreutils_8_22 dependency (revision ac64dfe4b1fb) - this ended up in "Installing tool dependencies" state. If I go look at the tool details it highlights errors in package_perl_5_18 which in turn refers to errors in gnu_coreutils (just "Error" state) and then this more informative failure for perl: "tar (child): local-lib-1.008009.tar.gz: Cannot open: No such file or directory tar (child): Error is not recoverable: exiting now tar: Child returned status 2 tar: Error is not recoverable: exiting now" I wonder if this is a sign of a download error? Digging a bit more I find the error "Shutting down process id 21094 because it generated no output for the defined timeout period of 3600.0 seconds." for gnu_coreutils. I'm trying to do a reinstall of the gnu_coreutils and perl packages to see if I can get further. Peter

2 1

It would be nice to know that labels can only be lowercase in format tags
by rbrown1422＠comcast.net 19 Feb '16

19 Feb '16

Good evening, Having wasted multiple hours it would be helpful to see in the documentation that only LOWERCASE is allowed for defining unique file extensions to link specific files output by one tool as a particular input to another tool. <datatype extension="LOWER CASE ONLY-prog1_output" type="galaxy.datatypes.data:Text" sub display_in_upload="True"/> outgoing tool's xml <datatype format="LOWER CASE ONLY-prog1_output" name="output1" label="xxxx"/> Accepting tool's xml <datatype format="LOWER CASE ONLY-prog1_output" type="data" name="input1" label="xxxx" /> It would be great to know up-front the format can only be LOWER CASE -- no error message, no ignoring upper or lower and treating the same. Just failure when trying to run it.

2 1