February 2016 - galaxy-dev - lists.galaxyproject.org

bx-python issue with intersect and subtract tools
by Poole, Richard 12 Feb '16

12 Feb '16

Hi all, I just used the subtract tool (installed under Operate on Genomic Intervals) on my own Galaxy server for the first time in a very long time and got this error message from the tool: Traceback (most recent call last): File "/Users/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/subtract/c19a2a29c561/subtract/gops_subtract.py<http://toolshed.g2.bx.psu.edu/repos/devteam/subtract/c19a2a29c561/subtract/…>", line 16, in <module> from bx.intervals.io import GenomicInterval, NiceReaderWrapper ImportError: No module named bx.intervals.io So I uninstalled the intersect and subtract tools, plus the bx-python 0.7.1 package plus the galaxy-ops package and reinstalled both tools from the toolshed using the latest revisions of each from devteam. I now get this error message: /Users/galaxy/galaxy-dist/lib/galaxy/__init__.py:63: UserWarning: Module bx was already imported from /Users/galaxy/galaxy-dist/tool-dependencies/bx-python/0.7.1/devteam/package_bx_python_0_7/2d0c08728bca/venv/lib/python2.7/site-packages/bx/__init__.pyc, but /Users/galaxy/galaxy-dist/eggs/bx_python-0.7.2-py2.7-macosx-10.6-intel-ucs2.egg is being added to sys.path self.check_version_conflict() Traceback (most recent call last): File "/Users/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/subtract/c19a2a29c561/subtract/gops_subtract.py<http://toolshed.g2.bx.psu.edu/repos/devteam/subtract/c19a2a29c561/subtract/…>", line 21, in <module> from utils.gff_util import GFFFeature, GFFReaderWrapper, convert_bed_coords_to_gff ImportError: No module named utils.gff_util Does this suggest an incompatibility somehow with the the egg which is installed at startup and is versions 0.7.2 - I tried manually editing the egg file but could not switch that to 0.7.1 I also then tried updating Galaxy to latest release 15.07 and even 15.10 which appeared in the git tag list as I writing this email, but no fix there - my Galaxy is running on a Mac OS X 10.10.5 Yosemite. Any thoughts on how to resolve this problem? Cheers, Rich Richard J Poole PhD Wellcome Trust Fellow Department of Cell and Developmental Biology University College London 518 Rockefeller 21 University Street, London WC1E 6DE Office (518 Rockefeller): +44 20 7679 6577 (int. 46577) Lab (529 Rockefeller): +44 20 7679 6133 (int. 46133)

2 1

Jupyter
by Jasper Koehorst 10 Feb '16

10 Feb '16

I am currently experimenting with jupyter and it finally works as I am able to use the environment on a local machine. However beside python2.7 how do I load a galaxy history file into for example R, Bash, etc? I could not really find any documentation on these languages beside python 2.7. Is there a website for best practices? Kind regards, Jasper

2 3

A Report Calc tool which provides a scripting language and interpreter for text mining and quality control
by Dooley, Damion 09 Feb '16

09 Feb '16

Folks, we're testing out a basic scripting language and interpreter for report writing and quality control that is meant to provide both command-line programmers and Galaxy platform researchers and admins with ways to tweak workflow quality control behaviour without having to be programmers themselves. I wanted to sound-out the community about whether or not there are any basic objections to our approach, described briefly below? Currently the interpreter provides all the built-in Python operator and math functions (as well as some particular named group regular expression functions for text mining) so that users can examine given input log or data datasets for fields that need to be reported or compared with QC metric rules. The Report Calc tool takes in a file of statements (each being a function(parameter1 parameter2 ...) syntax) that look like this: set(4857000 report/contigs/reference_genome_size) set("serovar Typhimurium LT2" report/contigs/reference_genome) set(0.1 report/contigs/good_genome_size_ratio) set( statisticN(report/contigs/contig_lengths 50) report/contigs/N50) if( lt(/N50 200000) set(report/job/status FAIL)) Math is accomplished by python built-in math functions (I.e. Ignore the "/" - that's a namespace syntax character). set( truediv( abs( sub( /sampleGenomeSize /referenceGenomeSize)) /referenceGenomeSize) report/contigs/sample_genome_size_ratio ) And writes any output as desired to a standard tool output folder: writeFile( pageHtml( getHtml(report) "My Report Widget") report.html ) It allows users to build a ruleset (file containing statements like the above), and process text, json, tabular datasets in their history, and it can manipulate variables, arrays and dictionaries in the tools in-memory temporary data structure namespace. It doesn't touch Galaxy's inner workings or interact with a workflow except by way of app exit codes. One can even write little text-mining programs in it: iterate( readFileByName(contigs-all.fasta) if( eq( getitem(iterator/0/value 0) ">") append( regexp(iterator/0/value "length_(?P<value>\\d+)_") report/contigs/contig_lengths ) ) ) I'll release it shortly for review/play, with lots of documentation that describes in detail what it can/can't do, and an argument for why one might want to bother using/learning it. There are one or two irrelevant python built-in functions we might filter out (e.g. isCallable() ); so far we haven't spotted any security issues, and we've limited the flow control to only accomplish loops by iterables so there's no evident way to create infinite loops. One only iterates through files or in-memory arrays. As well suggested functionality for the wish-list now accepted! Regards, Damion

2 4

EMBOSS 6.6.0 + NCBI-BLAST+ integration
by Floreline TOUCHARD 09 Feb '16

09 Feb '16

Hi, I would like to add some tools of the EMBOSS suite (6.6.0) to my Galaxy instance. It seems that the only wrapper available for Galaxy is for the 5.0.0 version ( http://dev.list.galaxyproject.org/Galaxy-with-EMBOSS-5-or-EMBOSS-6-td465804…) I would like to know if someone has already tried (successfully or not) to wrap this tools base on the old wrappers to estimate the workload that awaits me (for prettyseq and water mostly). Could there be an invalid datatype format for example ? I have another question regarding the NCBI BLAST+ suite : is there a way to edit the XML file to enable the selection of multiple databases as proposed by the command lines tool ? Thanks in advance, F.T

4 8

Galaxy broken after update to 16.01
by Matthias De Smet 09 Feb '16

09 Feb '16

Hi all, We were tracking the galaxy master branch on our local server and are looking to migrate to v16.01. The galaxy processes are managed bij supervisor as shown here ( https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling) After updating all dependencies in our venv we tried to restart the server but got the following issue: Loading paste environment: config:/home/galaxy/galaxy-dist/config/galaxy.ini Traceback (most recent call last): File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp return loadobj(APP, uri, name=name, **kw) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj global_conf=global_conf) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext global_conf=global_conf) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 458, in get_context section) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 517, in _context_from_explicit value = import_string(found_expr) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 22, in import_string return pkg_resources.EntryPoint.parse("x=" + s).load(False) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2380, in load return self.resolve() File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2386, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "lib/galaxy/web/buildapp.py", line 5, in <module> from galaxy.webapps.galaxy.buildapp import app_factory File "lib/galaxy/webapps/galaxy/buildapp.py", line 15, in <module> import galaxy.app File "lib/galaxy/app.py", line 9, in <module> import galaxy.queues File "lib/galaxy/queues.py", line 9, in <module> from kombu import Exchange, Queue, Connection File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/__init__.py", line 67, in __getattr__ module = __import__(object_origins[name], None, None, [name]) File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/entity.py", line 10, in <module> from .abstract import MaybeChannelBound, Object File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/abstract.py", line 12, in <module> from .connection import maybe_channel File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/connection.py", line 19, in <module> from kombu import exceptions File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/kombu/exceptions.py", line 12, in <module> from amqp import ChannelError, ConnectionError, ResourceError File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/amqp/__init__.py", line 34, in <module> from .connection import Connection # noqa File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/amqp/connection.py", line 40, in <module> from .transport import create_transport File "/home/galaxy/galaxy-dist/.venv/lib/python2.7/site-packages/amqp/transport.py", line 21, in <module> import ssl File "/Shared/Software/Cellar/python/2.7.11/lib/python2.7/ssl.py", line 97, in <module> import _ssl # if we can't import it, let the error propagate ImportError: /home/galaxy/galaxy-dist/.venv/lib/python2.7/lib-dynload/_ssl.so: undefined symbol: SSL_CTX_set_alpn_protos So far we have tried: - rebuilding the venv - reinstalling python Funny thing is, when we start the configuration with the “run.sh” script instead of through supervisor everything seems to work just fine. We’re at a loss, please help! Thanks! Matthias

2 1

Bioinformaticist @ Johns Hopkins Advanced Physics Laboratory (APL)
by Dave Clements 08 Feb '16

08 Feb '16

[image: Bioinformaticist @ Johns Hopkins APL] <https://jhuapl.taleo.net/careersection/2/jobdetail.ftl?job=11077> Hi All, The Johns Hopkins University Applied Physics Laboratory (APL) <http://www.jhuapl.edu/> is seeking a bioinformatics expert <https://jhuapl.taleo.net/careersection/2/jobdetail.ftl?job=11077> to join the Biological Sciences and Engineering Group (REDD\R1C). They are seeking someone with "demonstrated expertise in the analysis of sequencing data to contribute to innovative research in synthetic biology and infectious disease mitigation." If you are a US citizen with Unix chops, programming experience, analysis skills, and have "*Familiarity with the Galaxy platform for bioinformatic analysis, to include experience administering a local Galaxy instance*" then please give them a shout <https://jhuapl.taleo.net/careersection/2/jobdetail.ftl?job=11077>. Dave Clements <https://wiki.galaxyproject.org/DaveClements> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

Re: [galaxy-dev] Fwd: Rscript run under Tools Directory cannot see Datafile in History smaller email size
by John Chilton 08 Feb '16

08 Feb '16

Forgot to cc the mailing list. -John On Thu, Feb 4, 2016 at 8:11 PM, John Chilton <jmchilton(a)gmail.com> wrote: > Are you sure about the --file when calling Rscript? > > Here is the usage for my local version which does not expect an > argument named --file: > > Usage: /path/to/Rscript [--options] [-e expr] file [args] > > --options accepted are > --help Print usage and exit > --version Print version and exit > --verbose Print information on progress > --default-packages=list > Where 'list' is a comma-separated set > of package names, or 'NULL' > or options to R, in addition to --slave --no-restore, such as > --save Do save workspace at the end of the session > --no-environ Don't read the site and user environment files > --no-site-file Don't read the site-wide Rprofile > --no-init-file Don't read the user R profile > --restore Do restore previously saved objects at startup > --vanilla Combine --no-save, --no-restore, --no-site-file > --no-init-file and --no-environ > > 'file' may contain spaces but not shell metacharacters > > Also, instead of using $GALAXY_ROOT - I would use $__tool_directory__ > if the tool XML file is located next to the RScript, like this: > > $__tool_directory__/Foo.R > > -John > > > On Thu, Feb 4, 2016 at 7:05 PM, <rbrown1422(a)comcast.net> wrote: >> Good morning team, >> I have an Rscript tool that is trying to read a file I uploaded into the >> history beforehand. Galaxy is running with the same user and privileges as >> the dataset directory. >> <command>Rscript --slave --vanilla >> --file=$GALAXY_ROOT_DIR/tools/Foo/Foo.R --args $inputFile1 ..... </command> >> Why would this be? >> >> Error in file(file, "rt") : cannot open the connection >> Calls: performDataOrdering -> read.table -> file >> In addition: Warning message: >> In file(file, "rt") : >> cannot open file >> '--file=/home/rbrown/hmgalaxy/database/files/000/dataset_162.dat': No such >> file or directory >> Execution halted >> >> When I try to run as R versus Rscript - I get a syntax error about >> unexpected '(' on function call that works fine in R window. >> >> Thanks >> Bob >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/

3 4

Removing set_metadata_externally from wiki documentation?
by Peter Cock 06 Feb '16

06 Feb '16

Hello all, We're currently looking at changing our Galaxy setup to link user accounts with Linux user accounts for better cluster integration (running jobs as the actual user on SGE). As part of this, we've tried setting up a fresh installation on a new VM which has thrown up some issues. First of all, what is the status of the old set_metadata_externally setting? This is described on the legacy documentation page as required under "Submitting Jobs as the Real User", https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster/Legacy Unfortunately, the same text is still present on the current page: https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster/ According to the GitHub search, set_metadata_externally no longer exists in the code or default settings files: https://github.com/galaxyproject/galaxy/search?q=set_metadata_externally I eventually tracked this down to the following commit, first included in Galaxy v13.06, back when the config file was universe_wsgi.ini rather than config/galaxy.ini, https://github.com/galaxyproject/galaxy/commit/68065e123800a5a2e700d636db93… So, can we assume that metadata is now always set externally, and should the wiki be updated accordingly? Thanks, Peter

2 2

dramm.py error
by Le-shin Wu 05 Feb '16

05 Feb '16

Hi, I was trying to setup a galaxy server and connect to our UGE cluster. If I setup the default job runner as "local", our galaxy server works properly. But if I change the default job runner as "drmaa", our galaxy server throws out the following error message. It looks like drmaa.py has unicode decoding issue. I wonder does someone have the same problem before and how can I fix it. The python version we are using is 2.7.10. Thank you. galaxy.jobs.runners ERROR 2016-02-04 10:49:44,689 (3) Unhandled exception calling queue_job Traceback (most recent call last): File "/home/c226089/galaxy/lib/galaxy/jobs/runners/__init__.py", line 103, in run_next method(arg) File "/home/c226089/galaxy/lib/galaxy/jobs/runners/drmaa.py", line 135, in queue_job job_name = ''.join( map( lambda x: x if x in ( string.letters + string.digits + '_' ) else '_', job_name ) ) File "/home/c226089/galaxy/lib/galaxy/jobs/runners/drmaa.py", line 135, in <lambda> job_name = ''.join( map( lambda x: x if x in ( string.letters + string.digits + '_' ) else '_', job_name ) ) UnicodeDecodeError: 'ascii' codec can't decode byte 0xaa in position 52: ordinal not in range(128) Best LW

2 2

Upload issue in local installation
by Batsal Devkota 05 Feb '16

05 Feb '16

I installed galaxy locally in a linux server. However, I cannot upload the files (no matter how small, I have tried few kb size fasta files). When I try to upload, the link to the file shows up in the History and gets a new number (purple box). When I click on the link I get 'Dataset is uploading' forever. In the terminal window where I start galaxy, I get the following error report: 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET / HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET /root/tool_menu HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET /history HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:33 -0400] "POST /root/user_get_usage HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:40 -0400] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://redhat:8080/root/tool_menu" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "POST /tool_runner/upload_async_create HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "GET /history HTTP/1.1" 200 - "http://redhat:8080/tool_runner/upload_async_message" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "POST /root/user_get_usage HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:34:04 -0400] "POST /root/history_item_updates HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:34:08 -0400] "POST /root/history_item_updates HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" The last line keeps going for ever... writes new line every 4 secs. I am stuck and don't know where to look. Please help. Batsal.

6 7