November 2014 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] samtools: command not found in bowtie2
by Eric Kuyt 26 Nov '14

26 Nov '14

Hi Anna, Setting your own path variables shouldn't be necessary for toolshed tools, so probably something is wrong with the installation. could you post your tooldependencies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/INSTALLATION.log file? On 26 November 2014 at 13:49, Anna Terry <iseeglass(a)googlemail.com> wrote: > Hi, > I installed bowtie2 from the toolshed, but get the following error when I > run it: > > > Traceback (most recent call last): > File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line > 129, in queue_job > job_wrapper.finish( stdout, stderr, exit_code ) > File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 998, > in finish > dataset.datatype.set_meta( dataset, overwrite=False ) # call > datatype.set_meta directly for the initial set_meta call during dataset > creation > File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line > 251, in set_meta > raise Exception, "Error Setting BAM Metadata: %s" % stderr > Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found > > I'm not sure why this won't work? > samtools is a dependency for the bowtie2 tool and is installed, but cannot > seem to find it. > There must me something slightly wrong on my server as I can't see loads > of errors like this on this mailing list. > > I have added the path from the samtools env file to .bash_profile for > galaxy and this allows this tool to run, but is not the correct solution, > as doesn't work when versions change. > > Could you let me know where I'm going wrong > Thanks > Anna > -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

1 0

How to get cleanup script to run in cron job ?
by James Mullan 26 Nov '14

26 Nov '14

Hello, I am trying to figure out how to run the cleanup scripts in a cron job. The below works on command line: python scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i pysqlite>=2 egg successfully loaded for sqlite dialect ########################################## # 2014-11-19 22:07:32 - Handling stuff older than 30 days # Displaying info only ( --info_only ) Deleted 0 histories Elapsed time: 0.0684289932251 ########################################## but if I try to do this in a cron job as follows: 55 21 * * * python scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i >> /media/Store/galaxy/galaxy-dist/temp 2>&1 python: can't open file 'scripts/cleanup_datasets/cleanup_datasets.py': [Errno 2] No such file or directory or if I try to do this : 00 22 * * * python /media/Store/galaxy/galaxy-dist/scripts/cleanup_datasets/cleanup_datasets.py config/galaxy.ini -d 30 -1 -i >> /media/Store/galaxy/galaxy-dist/temp 2>&1 Traceback (most recent call last): File "/media/Store/galaxy/galaxy-dist/scripts/cleanup_datasets/cleanup_datasets.py", line 13, in <module> from galaxy import eggs ImportError: No module named galaxy Note that I am directing the output to a file only to try to find out why it's not working. Is this a problem with environment? Any suggestions what to try would be much appreciated. Regards James J P Mullan

2 2

Bug with multiple reverse proxies
by Jan Kanis 25 Nov '14

25 Nov '14

Hi, We ran into a problem if galaxy is running behind multiple reverse proxies. Galaxy assumes that the X-Forwarded-Host header only contains a single host to which it redirects, but apache will append comma-separated components to it if it already exists, which is the case of multiple reverse proxies. The result is that galaxy sometimes (i.e. sometimes after user login, depending on browser specifics, and when installing toolshed tools) tries to redirect the browser to "hostname, hostname". Apache docs <http://httpd.apache.org/docs/2.4/mod/mod_proxy.html#x-headers> I have a patch for galaxy in this pull request <https://bitbucket.org/galaxy/galaxy-central/pull-request/560/use-first-comp…>. That change fixes a file that makes the same assumption. The actual fix requires a change in Routes. The Routes PR is here <https://github.com/bbangert/routes/pull/33>, so actually fixing this requires an update of Routes after that pr is merged there. Jan

3 5

Is anyone using composite datatype uploads?
by Aysam Guerler 25 Nov '14

25 Nov '14

Hello everyone, We are considering to disable the deprecated upload tool form which is currently accessible through Tool panel > Get Data > Upload file. The new upload feature (icon at the top of the Tool panel) covers all of its functionality except uploading composite datatypes like e.g. Velvet. Please let us know if you are using the composite file upload functionality of the former tool form. Thanks, Sam

3 3

tool xml data_columns interface
by Jorrit Boekel 25 Nov '14

25 Nov '14

Hi all, I’m playing around with the data_column param in my tool xml. Excellent stuff. I only have a small point. The force_select attribute, can it not instead be called optional (and reverse its behaviour)? That would make it more similar to other params which have optional=true/false as an option (e.g. type=data). cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden

2 1

ymaj error in galaxy
by pj dias 25 Nov '14

25 Nov '14

Hi people, I'm presently trying to use the gmaj tool in galaxy and I have encounter and error when I try to make available the annotation data corresponding to each genome in the alignment. I think my problem results from not knowing how to format the internal structure of the exons file, I'm pointing gmaj directly to each gff3 file I have constructed for these genomes. The error that occurs is: “Warning: No genes found in exons file.” My question is how to format properly the exons file (I think it would be helpful to have available a sample of the internal structure of the exons, underlays, etc files). In the text bellow goes a sample of my gff3 file. Any help will be very welcome, Paulo Ps. A sample of an exons file that someone used with gmaj would be helpful to check the internal structure of the exons file.

1 0

handler processes suddenly consuming all RAM
by Langhorst, Brad 24 Nov '14

24 Nov '14

I recently upgraded galaxy to the 10/06 unannounced galaxy-dist. I typically run with 2 web processes and 2 job handlers on a single machine. It seems that now the job handlers consume all RAM at startup and are killed by the kernel. Anybody seen something like this? Bad job? bug? I’m considering a downgrade to the previous stable release. Brad -- Brad Langhorst, Ph.D. Applications and Product Development Scientist

1 1

Re: [galaxy-dev] rename output dataset in workflow - input dataset variable
by Langhorst, Brad 24 Nov '14

24 Nov '14

Hi Jan: Are you using input dataset tool? I'm not sure it's required - but I habitually use it. I'm attaching an image of a working renaming setup. I hope it helps. brad [cid:8ee56035-e56a-407f-8e02-1d29ef8a6279] ________________________________ From: Jan Hapala <jan(a)hapala.cz> Sent: Wednesday, November 12, 2014 9:29 AM To: Langhorst, Brad Subject: Re: [galaxy-dev] rename output dataset in workflow - input dataset variable Thanks, Brad. This is informative, but I'm still stuck. I have an input dataset which passes data to FASTQ Summary Statistics. (I have tried other tools, as well.) The output file name is: stats on: #{input1} The name of the Input dataset is: input1 I get this result: stats on: Jan 2014-11-12 14:06 GMT+01:00 Langhorst, Brad <Langhorst(a)neb.com<mailto:Langhorst@neb.com>>: For what it's worth (not much ;), I think the dataset naming is the single least intuitive part of the galaxy system for biologists. I think there's a proposal to "fix" this problem - and mainain the connection between input data set and derived data, but It doesn't seem to have gone anywhere yet. Anyway, you have to check the name of the field for each tool - it's sometimes "input" and sometimes something else. You want to use the #{} for naming. The ${} stuff is for parameters I also do this kind of thing: to keep the names from getting too long. #{input_1 | basename}.bam The basename removes everything after the last . character. Brad ________________________________ From: galaxy-dev-bounces(a)lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu> <galaxy-dev-bounces(a)lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu>> on behalf of Jan Hapala <jan(a)hapala.cz<mailto:jan@hapala.cz>> Sent: Wednesday, November 12, 2014 7:49 AM To: galaxy-dev Subject: [galaxy-dev] rename output dataset in workflow - input dataset variable Hello, I want to rename ouput datasets in a workflow in such a way that the name contains the name of the input dataset. I could not find instructions in the manual (and this info is missing in the Workflow editor - would be really helpful there!). I tried #{input}, e.g. "stats on: #{input}" (with no quotes), but I keep getting just "stats on:" The variable is empty. When I use ${input}, I get a variable field in the workflow form (I do not want this). My G. instance: Galaxy changeset: d1e1beeb532239250396dd8fbdc0156508c6744e<https://bitbucket.org/galaxy/galaxy-central/commits/d1e1beeb532239250396dd8…> Can anyone help me, please? Jan

5 5

more control over the "docker" command
by Dan Tenenbaum 24 Nov '14

24 Nov '14

Hi, Thanks for supporting the running of docker containers in Galaxy. I have two requests for more control over the docker command that is run. According to https://github.com/apetkau/galaxy-hackathon-2014 , the docker command that is run when a docker-enabled tool is run might look something like this: command is: sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/aaron/Projects/galaxy-central:/home/aaron/Proje cts/galaxy-central:ro -v /home/aaron/Projects/galaxy-central/tools/docker:/home/aaron/Projects/galaxy-central/tools/docker:ro -v /home/aaron/Projects/galaxy-central/datab ase/job_working_directory/000/6:/home/aaron/Projects/galaxy-central/database/job_working_directory/000/6:rw -v /home/aaron/Projects/galaxy-central/database/files:/home/aa ron/Projects/galaxy-central/database/files:rw -w /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6 --net none busybox:ubuntu-14.04 /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/container.sh; return_code=$?; if [ -f /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/wo rking_file ] ; then cp /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/working_file /home/aaron/Projects/galaxy-central/database/files/000/dataset_10.dat ; fi; sh -c "exit $return_code" I'd like to be able to specify extra flags to be included in the command. In my case I'd like to include "--link server:server" because I want to link this container with another container that contains a long-running server process (I can describe my use case in greater detail if desired). Can there be a way to do this in my tool wrapper? Secondly, rather than giving my galaxy user passwordless sudo, I'd like to add it to the docker group, then I can run docker commands without prepending sudo. This seems a lot safer. Could this be exposed, maybe by a sudo="false" attribute in the tool wrapper? Thanks, Dan

3 4

Re: [galaxy-dev] Some plugin for digital normalization
by Michael R. Crusoe 24 Nov '14

24 Nov '14

> From: Cristian Alejandro Rojas <alejandro.0317(a)gmail.com> > Date: December 26, 2013 2:37 AM > To: galaxy-dev(a)lists.bx.psu.edu > Subject: [galaxy-dev] Some plugin for digital normalization > Hello all, > > I've been searching in tool sheds(main and test) for a plugin to do digital normalization of reads, for example Trinity has one script to do this (./util/normalize_by_kmer_coverage.pl), diginorm is another alternative (http://ged.msu.edu/papers/2012-diginorm/) > > Do you know if there is a galaxy tool ready for this work or should I develop my own? Hello Cristian, I am actively working on a wrapper for normalize-by-median.py and the other tools in the khmer suite. https://github.com/ged-lab/khmer/blob/galaxy-integration/scripts/normalize-… It requires some changes on our end which you can track here: https://github.com/ged-lab/khmer/pull/237 You can install this branch of khmer via pip pip install -e git+git@github.com:ged-lab/khmer.git@output_naming1#egg=khmer or if you don't have a github account: pip install -e git+ https://github.com/ged-lab/khmer.git@output_naming1#egg=khmer (Instructions on installing using a virtualenv or on OS X are at: https://khmer.readthedocs.org/en/latest/install.html just substitute in the pertinent pip line from above) I haven't tested it much; doing so is the top of my to-do list for the rest of this week. If you or anyone else is feeling adventurous I would be happy to have the feedback. Cheers! -- Michael R. Crusoe: Software Engineer and Bioinformatician mcrusoe(a)msu.edu @ the Genomics, Evolution, and Development lab; Michigan State University http://ged.msu.edu/ http://orcid.org/0000-0002-2961-9670 @biocrusoe

3 5