January 2014 - galaxy-dev - lists.galaxyproject.org

Nightly testing status on the (Test) Tool Shed
by Peter Cock 07 Mar '14

07 Mar '14

Hi all, Are both the main and test Tool Sheds currently meant to be running nightly tests again? I've not really got back into Galaxy tool testing since Christmas. Most of my tools' test results look sensible, but there seem to be some issues still, e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/mira4_assembler Not tested since 2014-01-04 http://toolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr 2014-01-29 missing blast_datatypes dependency (why?) 2014-01-28 showing failed tests for sambamba_filter (wrong tool) Regards, Peter

4 32

Question on storage configuration for Galaxy datasets
by Lukasse, Pieter 28 Feb '14

28 Feb '14

Hi , We would the location of the datasets our Galaxy stores to be determined dynamically based on a set of rules. For example: for a specific role we would like a datasets to be stored in a folder /storage/no_backup and for another role this should be /storage/with_backup Is there currently a configuration file in Galaxy to set this up? Or perhaps any alternative solutions? Thanks and regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317481122; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

2 2

Difficulty dispatching toolshed tool jobs via job_conf.xml
by Guest, Simon 27 Feb '14

27 Feb '14

I am having trouble getting a toolshed tool to be dispatched to the destination I list in the job_conf.xml file. My job_conf.xml file looks like this: <?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="20"/> <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner" /> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="condor"> <destination id="local" runner="local"/> <destination id="condor" runner="condor">  </destination> </destinations> <tools> <tool id="upload1" destination="local"/>  <tool id="ucsc_table_direct1" destination="local"/>  ... stuff omitted ... <tool id="toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS: infoseq46/5.0.0" destination="local"/> </tools> </job_conf> So, you can see I have a default destination of condor, but I'm trying to run my toolshed EMBOSS infoseq tool on local. However, it is stubbornly running on condor. In lib/galaxy/tools/__init__.py:1132, I see this comment which got me wondering: # In the toolshed context, there is no job config. Is it possible to define tool destinations for toolshed tools? Are there some gotchas that I should know about? Any other ideas why my job is ignoring the config in job_conf.xml? (By the way, I can change say the upload1 tool to run on Condor by setting its destination in that file, so it is doing something.) The other thing I saw in the source code is stuff about old_id and toolshed guids. Do I need to understand this stuff? The paster.log contains the following when I submit the infoseq job: 147.158.130.216 - - [11/Sep/2013:13:53:34 +1300] "GET /tool_runner?tool_id=toolshed-dev.agresearch.co.nz/toolshed/repos/guestsi/emboss_5_native/EMBOSS%3A%20infoseq46/5.0.0 HTTP/1.1" 200 - "http://galaxy-dev.agresearch.co.nz/" "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" 147.158.130.216 - - [11/Sep/2013:13:53:40 +1300] "POST /tool_runner/index HTTP/1.1" 200 - "http://galaxy-dev.agresearch.co.nz/tool_runner?tool_id=toolshed-dev.agresea…" "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" galaxy.jobs DEBUG 2013-09-11 13:53:40,886 (92) Working directory for job is: /home/galaxy-dev/galaxy/database/job_working_directory/000/92 galaxy.tools DEBUG 2013-09-11 13:53:40,886 Tool::get_job_destination: {'runner': 'condor', 'legacy': False, 'params': {}, 'tags': None, 'url': None, 'converted': False, 'id': 'condor'}. galaxy.jobs.handler DEBUG 2013-09-11 13:53:40,894 (92) Dispatching to condor runner (I added the debug output for Tool::get_job_destination to see what was going on.) Any ideas? cheers, Simon ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================

3 8

Blast+: new feature filter/mask... by taxid before job in one task
by Berner, Thomas 27 Feb '14

27 Feb '14

Hi guys, We recently installed NCBI BLAST + to our local Galaxy instance and now we need to provide the possibility to filter/mask ... by taxon id (taxid) using the command line option (-window_masker_taxid) of BLAST (cf. http://www.ncbi.nlm.nih.gov/books/NBK1763/) before the job will be performed in one task. However searching in the documentation and the mailing list, I did not find anything about this. So it would be great to provide the possibility to subselect by taxid (e.g. NCBI BLAST server http://blast.ncbi.nlm.nih.gov/Blast.cgi) by changing the BLAST integration. We believe this could be of common interest. It would be great if anybody could comment on this. Thanks a lot, Thomas

3 4

Galaxy Tool Shed packages for Biopython
by Peter Cock 18 Feb '14

18 Feb '14

Hi all, I've send this to both the Galaxy and Biopython developers lists, and hope this will make sense to both groups. If you've not heard of Galaxy, start here: http://galaxyproject.org - while the easy to guess Biopython website is at http://biopython.org Brad Chapman and I are both Biopython core developers, and are also both on the "IUC" Galaxy Tool Shed committee because we've been quite involved in wrapping and writing tools for use on Galaxy. Fellow committee member Björn Grüning has done a lot of the hands on work defining package definitions for dependencies within the Galaxy Tool Shed ecosystem - including defining them for Biopython, NumPy, SciPy, MatPlotLib, etc. We're very grateful for his hard work - most of which is now available under the IUC group account: http://toolshed.g2.bx.psu.edu/view/iuc/ http://testtoolshed.g2.bx.psu.edu/view/iuc/ The Biopython packages, however, are under a dedicated "biopython" account on the Galaxy Tool Shed to which currently Bjoern, Brad and I have access to: http://toolshed.g2.bx.psu.edu/view/biopython/ http://testtoolshed.g2.bx.psu.edu/view/biopython/ This packaging work was initially tracked in Bjoern's own GitHub repository, https://github.com/bgruening/galaxytools/ We (me, Brad and Bjoern) agreed that a Biopython owned repository would be more sensible in the long term, so I have created this and ported Bjoern's commits to it: https://github.com/biopython/galaxy_packages Currently the "Galaxy packagers" team on GitHub which has read and write access to this new repository is just me, Brad and Bjoern. Regards, Peter

2 3

shed tool dependencies
by Geert Vandeweyer 14 Feb '14

14 Feb '14

Hi, Is there a default notation to specify perl modules (threading modules) in a tool configuration or in a submission to the toolshed? I've got some tools to share that make extensive use of these modules, and they are not default in a perl distribution. Hence, I'd like to inform the users that they need to install these modules (or that they get installed by the the toolshed using cpan ?) Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/pub/geert-vandeweyer/26/457/726

4 3

Several jobs stalled in the queue: Job runner assigned but no external ID recorded
by Derrick Lin 05 Feb '14

05 Feb '14

Hi guys, I have several jobs have been stuck in the queue for long time, I checked the handler log and found the following message: Job runner assigned but no external ID recorded, adding to the job handler queue Is there a way that I can clean these orphan jobs out of the queue? Cheers, Derrick

1 1

Upload via FTP - Assertion Error on Change ownership of working dictory
by Adam Brenner 04 Feb '14

04 Feb '14

Howdy, I am having trouble uploading data on our local instance of Galaxy here. The method is via FTP and here is the error I am seeing in the logs: galaxy.jobs.runners DEBUG 2014-01-30 14:29:31,987 execution of external set_meta for job 1367 finished galaxy.jobs DEBUG 2014-01-30 14:29:32,016 (1367) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /som/galaxy/database/job_working_directory/001/1367 root 0 galaxy.jobs ERROR 2014-01-30 14:29:32,058 (1367) Failed to change ownership of /som/galaxy/database/job_working_directory/001/1367, failing Traceback (most recent call last): File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 870, in finish self.reclaim_ownership() File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 1316, in reclaim_ownership self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) ) File "/data/apps/galaxy/dist/lib/galaxy/jobs/__init__.py", line 1302, in _change_ownership assert p.returncode == 0 AssertionError Any idea on how to fix the issue? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences System Administrator, HPC Cluster Office of Information Technology http://hpc.oit.uci.edu/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

2 3

local galaxy instance can't recognize file format!!
by JU HUCK LEE 01 Feb '14

01 Feb '14

Dear Galaxy team, I have recently made local galaxy instance in my computer (since my labtop is window, I used virtualbox to install linux) to use some workflow made by other people (for finding bacterial non-coding RNA, the name of workflow is DETR'PROK). After I finally installed the local galaxy instance in my labtop(it was not easy because I haven't used linux system before), I uploaded my RNA-seq mapping data(bam file),which I did in galaxy main, through genomespace. However, my local galaxy instance can't recognize file format; it looks like that it can only recognize file format when I uploaded the files from my computer (when I use galaxy main, it is no problem to upload files from genomespace). So, I uploaded gff, fasta and bam files, but I got a error for my bam file. Please let me know what was wrong. I really want to use above workflow to find bacterial non-coding RNA which I can't do in galaxy main. Thank you, Ju Huck Lee ps. since I got this "Mail is not configured for this galaxy instance" when I clicked the report, I attached the screen with my error; please let me know how I can report my errors through my local galaxy instance. one more question; why the manage data in Admin is beta, so I couldn't use it.

1 0

Re: [galaxy-dev] Remote User Logout
by Tim Booth 01 Feb '14

01 Feb '14

Hi, I'm currently using one of those hacks, and it seems to work nicely for the user (Chrome + FF at least) but it does need some messy setting up in Apache and some cunning redirects in place. I've pasted the relevant file fragments below. It's somewhat confounded with my stuff to enable SFTP uploads but hopefully you get the idea and the original explanation on Stackoverflow is pretty good. The remote_user_logout_href is something I got to by trial and error. Cheers, TIM === % cat /usr/share/galaxy-server/logout/.htaccess # HaCk based on http://stackoverflow.com/questions/4163122/http-basic-authentication-log-out # Authname must match the one in ../proxy/.htaccess AuthType Basic AuthName Galaxy_Server AuthUserFile /usr/share/galaxy-server/logout/.htpasswd Require user logout === % cat /usr/share/galaxy-server/logout/.htpasswd #Password is logout. This in not a secret. logout:$apr1$0eB1iURY$kwqa0c8tXksbjPQLYqr6s. === % cat /usr/share/galaxy-server/proxy/.htaccess # Security settings for Galaxy proxied via Apache. Note the actual # proxy config is under /etc/apache2/conf.d/galaxy. If for some # reason you wanted Apache proxy with internal Galaxy authentication # then you could remove this file and Apache would no longer insist on # authentication. AuthBasicProvider external AuthExternal pwauth AuthType Basic AuthName Galaxy_Server #I'd like to do this, but it upsets Firefox. Use ErrorDocument instead. # AuthName "Galaxy Server: \ # Log in with regular username and password. \ # Users need to be in the galaxy system group." ErrorDocument 401 "<html>\ <title>401 Authorization Required</title>\ <h1>Log-in to Galaxy failed</h1>\ <p>You should have been prompted to log into the Galaxy server. \ You need to give your regular system username and password. \ Please reload this page to try again.</p>\ <p>If this fails, check that you are a member of the galaxy system group, by \ running <code>groups</code> on the command line.</p>\ <p>To add a user, eg. user1, to this group, you may use the command:</p>\ <ul><li><code>sudo usermod -aG galaxy user1</code></ul></li>\ </html>" # You may want to comment these 2 lines out or to # change the group required, but users still need to # be in the galaxy group for SFTP uploads to work properly. AuthzUnixgroup on Require group galaxy # This is needed to tell Galaxy about the remote # user. RequestHeader set REMOTE_USER %{RU}e env=RU RequestHeader unset Authorization env=RU === % cat /etc/galaxy-server/universe_wsgi.d/31_apache-proxy.ini # Settings added by debian-galaxy-apache-proxy to switch Galaxy over to # authenticating by real user accounts and also permitting uploads. [app:main] # Other scripts assume that maildomain is localhsot, so you can't just # change the setting below and expect everythig to work. use_remote_user = True remote_user_maildomain = localhost # Users may copy files here directly or upload via SFTP/SCP ftp_upload_dir = /var/lib/galaxy-server/transfer ftp_upload_site = *** Transfer files via SCP or SFTP to /var/lib/galaxy-server/transfer/... *** # There is no neat way to log out a user with Basic Auth, but here is a non-neat way. # Not yet tested on IE. remote_user_logout_href = javascript:var r=new XMLHttpRequest();r.onreadystatechange=function(){if(r.readyState==4)window.location.replace('logout.html')};r.open('get','logout.html',true,'logout','logout');r.send(); === -- Tim Booth <tbooth(a)ceh.ac.uk> NERC Environmental Bioinformatics Centre Centre for Ecology and Hydrology Maclean Bldg, Benson Lane Crowmarsh Gifford Wallingford, England OX10 8BB http://nebc.nerc.ac.uk +44 1491 69 2705

3 4