April 2013 - galaxy-dev - lists.galaxyproject.org

Command Line Tool for Creating New Users
by Adam Brenner 27 Apr '13

27 Apr '13

Howdy All, I was wondering if anyone has created a command line tool that will add users to the Galaxy user database? I did look into switching to external user authentication via nginx, however, this will result in us using the campus wide kerberos system and allowing ~30,000 people access to Galaxy... and thus offering us less fine grain control to Galaxy. Anything anyone has to share would be much appreciated, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

2 1

Running Galaxy through Apache
by Jeffrey Long 27 Apr '13

27 Apr '13

Hi folks, I am wanting to run a galaxy instance and serve it at a location other than the webserver root, i.e. at http://mymachine.com/galaxy rather than just at http://mymachine. To do this, I've followed the instructions about proxying Galaxy through Apache at http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy. I'm pretty sure I've followed the instructions exactly. I enabled the necessary apache proxy modules and then I added these lines to my apache.conf file: RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule^/galaxy/static/style/(.*) /myplace/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule^/galaxy/static/scripts/(.*) /myplace/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule^/galaxy/static/(.*) /myplace/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /myplace/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /myplace/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] ...and re-started the apache server without a problem (note: I also tried it commenting out everything but the first, second and last line in case there was an issue with the static stuff). I then made sure the following lines were present and uncommented in my universe_wsgi.ini file [filter:proxy-prefix]use = egg:PasteDeploy#prefixprefix = /galaxy[app:main] filter-with = proxy-prefixcookie_path = /galaxy I start up Galaxy. And still, I can only reach it on http://mymachine:8080, not at http://mymachine/galaxy as I intend. Any help would be appreciated! Thanks! -Jeff

2 1

Environment variables
by Matthias Zytnicki 26 Apr '13

26 Apr '13

Dear everyone, I have spent hours trying to set environment variable in Galaxy. I am managing the "s_mart" repos (you can check it if you wish), and I created the simple files "test.xml" which prints the environment variables: --- <tool id="test" name="test"> <description>This is a test.</description> <requirements> <requirement type="set_environment">PYTHONPATH</requirement> </requirements> <command>env > $outputFile</command> <outputs> <data name="outputFile" format="txt" label="output test file"/> </outputs> <help>This is a test.</help> </tool> --- and "tool_dependencies.xml", which appends something to the PYTHONPATH variable.: --- <?xml version="1.0"?> <tool_dependency> <set_environment version="1.0"> <environment_variable name="PYTHONPATH" action="append_to">$REPOSITORY_INSTALL_DIR</environment_variable> </set_environment> </tool_dependency> --- As you guess, I cannot see anything added in the PYTHONPATH variable. I have tried to append INSTALL_DIR and even "/tmp". None worked. Do you have any suggestion for me? Many thanks to you all, Matthias.

2 12

/bin/sh: 1: filename: not found
by Ramon Tiburski 25 Apr '13

25 Apr '13

I created a new tool in c++, and the "filename" is the name of my executable file. I have the xml and the executable file in the same directory, but I get this error when I try to run the tool. Someone can help me with this error? Thank you

2 4

Speed up the galaxy
by 泽蔡 25 Apr '13

25 Apr '13

Hi all, How can I speed up the galaxy? Like how to use more cores and memeries.

6 15

Re: [galaxy-dev] TestToolShed out of date? (not running unit tests)
by Peter Cock 25 Apr '13

25 Apr '13

On Wed, Apr 24, 2013 at 10:12 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > > ..., I've still finding some other cases where despite having > tests defined, nothing shows up on the Tool Shed test results page. > > Here's an example where I see that two tools (promoter2 and > wolf_psort) have no tests, but the results of all the other tools > in this suite are not mentioned (some expected to fail/be skipped > due to missing dependencies): > > http://toolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp > Revision 13:dc958c2a963a > > Equivalently: > > http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp > Revision: 14:0d8b1d20ce9c > > -------------------------------------------------------------------------------- > > Here's another example where based on what happens locally, > I am expecting a failure: > > http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename > Revision: 1:6ce8634e2035 > > This is failing for me locally, but again I suspect this is due to an > issue in the framework - much like the blastxml_to_top_descr > test is failing due to the datatypes not being updated, here there > seems to be something similar going wrong regarding the > second uploaded input file (which is used for to provide the > potential column parameters). See this thread: > > http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003867.html > http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-April/014261.html > > Regards, > > Peter Hi Dave, Yet another case, http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus Revision: 8:1f546099212f and, http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus Revision: 5:d42346e675c4 These list five or six invalid tools which do not define any unit tests, yet there is no mention of the tools present which do have tests (and if they passed or not). Even if ncbi_blastdbcmd_wrapper doesn't (yet) have a test, I still want to know if the ncbi_blastp_wrapper tests pass ;) Are this further symptoms of whatever was solved here? https://trello.com/card/toolshed-automated-functional-test-bug/506338ce32ae… Or does would a new Trello issue be best? Thanks, Peter

2 4

GalaxyAdmins Group: Future Directions?
by Dave Clements 25 Apr '13

25 Apr '13

Hello all, The GalaxyAdmins group is coming up on it's one year anniversary (coinciding with GCC2013) and this is a good opportunity to discuss what the future of the group should be. Some starting topics for discussion are on the GalaxyAdmins Future Directions page (http://wiki.galaxyproject.org/Community/GalaxyAdmins/Future) These include * What should the group's goals and activities be? * What type of leadership structure should the group have, and how should it be selected? The discussion, however, is wide open to any topic relevant to the group. If you have any opinions or suggestions please reply to the group. Anyone with an interest in the group is encouraged to post. Once the discussion settles, I will summarize the discussion on the wiki page and suggest an action plan for making those things happen. Thanks, Dave C. -- http://galaxyproject.org/GCC2013 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

5 9

Merging files: Error merging filesglobal name 'shutil' is not defined
by Peter Cock 25 Apr '13

25 Apr '13

Hi all, While testing a BLAST search using plain text output (the 'txt' datatype in Galaxy) or subclasses (like 'blastxml') using galaxy-dist with task splitting enabled, I found I new bug when the files are merged: Error merging files global name 'shutil' is not defined This is a side effect of this commit due to a missing import (perhaps the imports changed since Bjoern wrote the patch): https://bitbucket.org/galaxy/galaxy-central/commits/510b063792de89332428ed1… In addition to the missing import in data.py, there is a potential similar issue with binary.py but this is masked by an evil import * line ;) My suggested patch for the import issue below, however right now even though the merge is failing, Galaxy failed to treat the job as a failure - it was shown as a green successful job with "Error merging files" as stdout, and for stderr: "global name 'shutil' is not defined", but return code 0. So there is a deeper problem with the merge error not marking the job as failed. Also there is a minor bug in that the stdout and stderr are combined for the peep text without any white space - presumably most of the time the stdout will have a trailing new line, but not here - thus "filesglobal" [sic], not say "files global" or a newline. There's a second patch below for that minor issue too (although it seems my new lines get turned into spaces by the time they are shown in the history peep text). Regards, Peter -- $ hg diff diff -r 8b9ca63f9128 lib/galaxy/datatypes/binary.py --- a/lib/galaxy/datatypes/binary.py Wed Apr 24 16:24:41 2013 -0400 +++ b/lib/galaxy/datatypes/binary.py Thu Apr 25 11:32:51 2013 +0100 @@ -12,7 +12,7 @@ from bx.seq.twobit import TWOBIT_MAGIC_NUMBER, TWOBIT_MAGIC_NUMBER_SWAP, TWOBIT_MAGIC_SIZE from urllib import urlencode, quote_plus import zipfile, gzip -import os, subprocess, tempfile +import os, subprocess, tempfile, shutil import struct log = logging.getLogger(__name__) diff -r 8b9ca63f9128 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py Wed Apr 24 16:24:41 2013 -0400 +++ b/lib/galaxy/datatypes/data.py Thu Apr 25 11:32:51 2013 +0100 @@ -3,6 +3,7 @@ import mimetypes import os import sys +import shutil import tempfile import zipfile from cgi import escape $ hg diff diff -r 8b9ca63f9128 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Wed Apr 24 16:24:41 2013 -0400 +++ b/lib/galaxy/jobs/__init__.py Thu Apr 25 11:43:30 2013 +0100 @@ -924,7 +924,13 @@ for dataset in dataset_assoc.dataset.dataset.history_associations + dataset_assoc.dataset.dataset.library_associations: #need to update all associated output hdas, i.e. history was shared with job running dataset.blurb = 'done' dataset.peek = 'no peek' - dataset.info = ( dataset.info or '' ) + context['stdout'] + context['stderr'] + dataset.info = (dataset.info or '') + if context['stdout'].strip(): + #Ensure white space between entries + dataset.info = dataset.info.rstrip() + "\n" + context['stdout'].strip() + if context['stderr'].strip(): + #Ensure white space between entries + dataset.info = dataset.info.rstrip() + "\n" + context['stderr'].strip() dataset.tool_version = self.version_string dataset.set_size() if 'uuid' in context:

2 2

traceback - local error?
by Langhorst, Brad 25 Apr '13

25 Apr '13

172.17.130.203 - - [19/Apr/2013:10:31:20 -0400] "GET /api/datasets/dd61915e24aa3fd0?hda_ldda=hda&data_type=converted_datasets_state&chrom=chr1 HTTP/1.1" 200 - "http://galaxy.neb.com/visualization/trackster?dataset_id=b8edc02014d74add&h…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.65 Safari/537.31" galaxy.webapps.galaxy.api.datasets ERROR 2013-04-19 10:31:21,775 Error in dataset API at listing contents: unpack requires a string argument of length 4: unpack requires a string argument of length 4 Traceback (most recent call last): File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 41, in show is_true( kwd.get( 'retry', False ) ) ) File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 96, in _converted_datasets_state if not data_provider.has_data( chrom ): File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 1037, in has_data f, bbi = self._get_dataset() File "/mnt/ngswork/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 1170, in _get_dataset return f, BigWigFile(file=f) File "bigwig_file.pyx", line 127, in bx.bbi.bigwig_file.BigWigFile.__init__ (lib/bx/bbi/bigwig_file.c:3011) File "bbi_file.pyx", line 119, in bx.bbi.bbi_file.BBIFile.__init__ (lib/bx/bbi/bbi_file.c:3508) File "bbi_file.pyx", line 129, in bx.bbi.bbi_file.BBIFile.open (lib/bx/bbi/bbi_file.c:3660) File "/mnt/ngswork/galaxy/galaxy-dist/eggs/bx_python-0.7.1_7b95ff194725-py2.6-linux-x86_64-ucs4.egg/bx/misc/binary_file.py", line 30, in __init__ if struct.unpack( ">I", bytes )[0] == magic: error: unpack requires a string argument of length 4 Is this a bug or have I done something unexpected? I'm trying to use the trackbrowser to visualize some bam files that I aligned to a genome that I had to install manually (Zea_mays_B73_RefGen_v2), maybe i've screwed something up. I was initially able to see some reads, but after reverting to the sample datatypes file and restarting galaxy i can't see anything anymore. I also added a .2bit file to twobit.locs Zea_mays_AGPv2 /mnt/ngswork/galaxy/galaxy-dist/tool-data/genome/Zea_mays_AGPv2/seq/Zea_mays_AGPv2.2bit maybe that's related... I've tried copying the bams to a new history - and they seem to be reindexing as a result, but I still see no reads. Any ideas? Brad -- Brad Langhorst langhorst(a)neb.com<mailto:langhorst@neb.com>

2 1

Blast Errors
by Ganote, Carrie L 25 Apr '13

25 Apr '13

Hi list, I installed the latest toolshed NCBI Blast+ into a new Galaxy and I'm trying to pinpoint these errors:

1 0