October 2014 - galaxy-dev - lists.galaxyproject.org

Role of suite_config.xml in current Tool Shed?
by Peter Cock 28 Oct '14

28 Oct '14

Hi all, I have a suite_config.xml file in one of my Tool Shed packages, but I am unclear if this is still used, or simply a legacy from the old pre-hg-based Tool Shed? e.g. https://github.com/peterjc/pico_galaxy/blob/master/tools/protein_analysis/s… My understanding was this allowed the tool author some control over the appearance/order that the tools will be shown in the Galaxy left hand pane. Thanks, Peter

1 0

Database Error When Bootstrapping Tool Shed
by Bruno Grande 28 Oct '14

28 Oct '14

I initially posted this question on Galaxy Biostars <https://biostar.usegalaxy.org/p/9389/>, but realized that galaxy-dev is probably more appropriate. I'm attempting to host a local instance of the Galaxy Tool Shed for development purposes. I've already gotten the Galaxy server up and running. I don't know where the Galaxy version number is located, but I have revision 78c9a97be4b6+ of galaxy-dist repository. I did manage to get the Tool Shed server working, but upon reading this blog post <http://gregvonkuster.org/galaxy-tool-shed-introduction/> by Greg Von Kuster (see section "Hosting Your Own Tool Shed for Developing Galaxy Tools and Utilities"), I wanted to bootstrap from the main Galaxy Tool Shed ( http://toolshed.g2.bx.psu.edu) First of all, I'm trying to make use of PostgreSQL as the database backend. I have enabled the database_connection option in my tool_shed.ini configuration file using the appropriate postgresql://connection string. However, the default SQLite database file in database/community.sqlite is still created, even with the database_file = database/community.sqlite line commented out in the tool_shed.ini config file. Is this normal? I can see a Galaxy-like schema in both the SQLite and PostgreSQL databases. However, it seems that the SQLite is the one being used. For instance, in order to re-run the bootstrapping command, the SQLite database file must be cleared. Otherwise, I get the error below. The same isn't true with the PostgreSQL database. Unable to bootstrap tool shed. The database configured for this Tool Shed is not new, so bootstrapping is not allowed. Create a new database that has not been migrated before attempting to bootstrap. Second of all, while running the bootstrapping command (command and output <https://gist.github.com/brunogrande/4277625d8be45e6a4840>), I run into a SQLAlchemy error claiming that the "repository" table doesn't exist. It's unclear which database it's referring to, but that table exists in both the SQLite and PostgreSQL databases. Therefore, I'm left wondering if it's referring to the public Tool Shed I'm bootstrapping from. Also, while the following steps say that they're "done", neither databases are populated with categories, nor with users. When I look into the tool_shed_bootstrap.log file (see attached), one thing I notice that that one of the GET HTTP requests fails with a 404 status ("GET /api/authenticate/baseauth/ HTTP/1.1" 404 - "-" "curl/7.35.0"), followed by many POST requests ("POST /api/users?key=a HTTP/1.1" 200 - "-" "Python-urllib/2.7"). Let me know if you could use any additional information to troubleshoot these issues. Best regards, Bruno

1 0

Dependencies not working on toolshed?
by Lukasse, Pieter 27 Oct '14

27 Oct '14

Hi , I am trying to get a dependency to work in practice (when running a tool after the installation). But I keep getting the following message when running the tool: [sshexec] galaxy.tools.deps DEBUG 2014-10-24 16:04:15,514 Building dependency shell command for dependency 'R_bioc_metams' [sshexec] galaxy.tools.deps WARNING 2014-10-24 16:04:15,514 Failed to resolve dependency on 'R_bioc_metams', ignoring This is the tool requirements section: <requirements> <requirement type="package" version="3.1.1">R_bioc_metams</requirement> </requirements> This is the repository_dependencies.xml : <repositories description="Required metaMS dependencies."> <repository toolshed="http://testtoolshed.g2.bx.psu.edu" name="prims_metabolomics_dependencies" owner="pieterlukasse" changeset_revision="71356c62e5cd" /> </repositories> This is the package name of the package that is being referred: <tool_dependency> <package name="R_bioc_metams" version="3.1.1"> <install version="1.0"> <actions_group> .... I can't find the problem. Any ideas? I have updated to the latest Galaxy version (today) and removed both items (with option to remove files) and installed them again. Am I facing a bug here? Thanks, Pieter Lukasse Wageningen UR, Plant Research International Department of Bioinformatics (Bioscience) Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands T: +31-317481122; M: +31-628189540; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

4 5

My CloudMan EC2 instance won't start Galaxy
by Jim White 27 Oct '14

27 Oct '14

When I start CloudMan following the instructions at (https://wiki.galaxyproject.org/CloudMan) (ami-a7dbf6ce, Galaxy CloudMan 2.3) I get pretty far in the process but then Galaxy won't start because the EBS volumes are never mounted. The Volume console shows that the volumes get created based on the public snapshots but they never go to "in use" which suggest that the call to mount them isn't getting done for some reason. I probably will end up building my own CloudMan AMI (I'm mostly interested in HTCondor DAGman rather than Galaxy) but I figured you'd want to know about this issue. Thanks, Jim Admin Console:https://dl.dropboxusercontent.com/u/73840666/CloudMan%20Trouble/Adm… EC2 Volumes:https://dl.dropboxusercontent.com/u/73840666/CloudMan%20Trouble/Vol… Logs: Cluster status log 13:22:17 - Master starting 13:22:18 - Successfully retrieved root user's public key from file. 13:22:18 - Completed the initial cluster startup process. This is a new cluster; waiting to configure the type. 13:22:22 - Migration service prerequisites OK; starting the service 13:22:22 - SGE service prerequisites OK; starting the service 13:22:28 - Setting up SGE... 13:22:36 - Initializing 'Galaxy' cluster type with storage type 'volume'. Please wait... 13:22:40 - For filesystem galaxy, unknown kind: galaxy 13:22:40 - For filesystem galaxyIndices, unknown kind: galaxyIndices 13:22:45 - PostgreSQL data directory '/mnt/galaxy/db' does not exist (yet?) 13:22:46 - Adding volume vol-9fe10c87 (FS object for galaxy)... 13:22:46 - Could not determine next available device from frozenset([]) 13:22:46 - Error adding file system service FS object for galaxy: 'NoneType' object is not iterable 13:22:46 - Adding volume vol-6ae10c72 (FS object for galaxyIndices)... 13:23:32 - Could not determine next available device from frozenset([]) 13:23:32 - Error adding file system service FS object for galaxyIndices: 'NoneType' object is not iterable 13:23:36 - Empty disk usage for FS transient_nfs 13:23:36 - STATUS CHECK: File system named 'galaxy' is not mounted. Error code 0 13:23:36 - STATUS CHECK: File system named 'galaxyIndices' is not mounted. Error code 0 13:23:36 - PostgreSQL data directory '/mnt/galaxy/db' does not exist (yet?) 13:23:49 - Empty disk usage for FS transient_nfs 13:23:49 - STATUS CHECK: File system named 'galaxy' is not mounted. Error code 0 13:23:49 - STATUS CHECK: File system named 'galaxyIndices' is not mounted. Error code 0 13:23:49 - PostgreSQL data directory '/mnt/galaxy/db' does not exist (yet?) 13:24:01 - Empty disk usage for FS transient_nfs 13:24:01 - STATUS CHECK: File system named 'galaxy' is not mounted. Error code 0 13:24:01 - STATUS CHECK: File system named 'galaxyIndices' is not mounted. Error code 0 13:24:01 - PostgreSQL data directory '/mnt/galaxy/db' does not exist (yet?) 13:24:14 - Empty disk usage for FS transient_nfs 13:24:14 - STATUS CHECK: File system named 'galaxy' is not mounted. Error code 0 13:24:14 - STATUS CHECK: File system named 'galaxyIndices' is not mounted. Error code 0 Admin Console log 2014-10-25 13:27:42,910 DEBUG master:2445 Monitor adding service 'Galaxy' 2014-10-25 13:27:42,910 DEBUG __init__:300 Galaxy service prerequisites are not yet satisfied, waiting for: [, , , , ]. Setting Galaxy service state to 'Unstarted' 2014-10-25 13:27:42,911 DEBUG master:2445 Monitor adding service 'GalaxyReports' 2014-10-25 13:27:42,911 DEBUG __init__:300 GalaxyReports service prerequisites are not yet satisfied, waiting for: []. Setting GalaxyReports service state to 'Unstarted' 2014-10-25 13:27:47,307 DEBUG sge:552 qstat: ['all.q(a)ip-10-158-92-92.ec2.inte BIP 0/0/1 0.33 lx24-amd64 '] 2014-10-25 13:27:47,344 WARNING filesystem:518 Empty disk usage for FS transient_nfs 2014-10-25 13:27:47,366 ERROR filesystem:582 STATUS CHECK: File system named 'galaxy' is not mounted. Error code 0 2014-10-25 13:27:47,390 ERROR filesystem:582 STATUS CHECK: File system named 'galaxyIndices' is not mounted. Error code 0 2014-10-25 13:27:47,391 WARNING postgres:199 PostgreSQL data directory '/mnt/galaxy/db' does not exist (yet?) 2014-10-25 13:27:47,404 DEBUG master:2534 S&S: Migration..Completed; SGE..OK; FS object for transient_nfs..OK; PSS..Unstarted; FS object for galaxy..Error; FS object for galaxyIndices..Error; Postgres..Unstarted; ProFTPd..Unstarted; Galaxy..Unstarted; GalaxyReports..Unstarted; 2014-10-25 13:27:47,404 DEBUG master:2445 Monitor adding service 'PSS' 2014-10-25 13:27:47,404 DEBUG pss:87 FS object for galaxy not running (Error), PSS service prerequisites not met afterall,not starting the service yet 2014-10-25 13:27:47,404 DEBUG master:2445 Monitor adding service 'Postgres' 2014-10-25 13:27:47,405 DEBUG __init__:300 Postgres service prerequisites are not yet satisfied, waiting for: []. Setting Postgres service state to 'Unstarted' 2014-10-25 13:27:47,405 DEBUG master:2445 Monitor adding service 'ProFTPd' 2014-10-25 13:27:47,405 DEBUG __init__:300 ProFTPd service prerequisites are not yet satisfied, waiting for: [, ]. Setting ProFTPd service state to 'Unstarted' 2014-10-25 13:27:47,405 DEBUG master:2445 Monitor adding service 'Galaxy' 2014-10-25 13:27:47,405 DEBUG __init__:300 Galaxy service prerequisites are not yet satisfied, waiting for: [, , , , ]. Setting Galaxy service state to 'Unstarted' 2014-10-25 13:27:47,405 DEBUG master:2445 Monitor adding service 'GalaxyReports' 2014-10-25 13:27:47,406 DEBUG __init__:300 GalaxyReports service prerequisites are not yet satisfied, waiting for: []. Setting GalaxyReports service state to 'Unstarted' 2014-10-25 13:27:51,407 DEBUG master:2445 Monitor adding service 'PSS' 2014-10-25 13:27:51,408 DEBUG pss:87 FS object for galaxy not running (Error), PSS service prerequisites not met afterall,not starting the service yet

2 2

Is the "new tool repositories summary" in the monthly newsletter useful?
by Dave Clements 26 Oct '14

26 Oct '14

Hi All, The October Galaxy newsletter went out a week ago. Buried at the bottom is this - 36 new ToolShed repos <https://wiki.galaxyproject.org/GalaxyUpdates/2014_10#ToolShed_Contributions> which lists repositories that have been published in the Galaxy Project ToolShed in the previous month. I have two questions about this: *1. How useful is this summary?* Compiling it is a manual process and it's kind of mind-numbing. Most months it takes around 2 hours (I think). *2. If we keep the summary, should we put it in the Dev News Briefs instead?* I'm kinda thinking this summary is a better match for the Dev News Briefs (every release), then it is for the general newsletter (every month). Since we are on a survey binge, I've created a 2 question Google form for you to vote at: http://bit.ly/1qdfs5t (It has a birthday cat theme!) If you care, please vote. Discussion (in this thread) is also welcome. And many thanks to IUC for suggesting I send this email. Thanks, Dave C -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

3 5

Can't change working directory during tool dependency installation
by Bruno Grande 25 Oct '14

25 Oct '14

I originally posted this question <https://biostar.usegalaxy.org/p/9401/> on the Galaxy Biostars, but later realized that galaxy-dev is probably more appropriate. So, here it goes. I'm trying to automate the installation of a tool's dependency with a tool dependency definition repository. During the package installation process, after moving the contents of a downloaded archive to the installation directory (*i.e.* $INSTALL_DIR), I simply want to change to that target directory away from the initial temporary working directory. However, I can't seem to change directory, either with a change_directory action or a cd shell_command action. Running pwd before and after attempting to change directory confirms this (see attached for installation log). I even made sure that $INSTALL_DIR is different from my initial working directory. Best regards, Bruno

2 2

Citations tag not working
by James Mullan 25 Oct '14

25 Oct '14

Hello, I downloaded Galaxy a few months ago but I just noticed that the citations tag doesn't work on my download. The info on the hg pull I have is: hg tip changeset: 13771:7a4d321c0e38 branch: stable tag: tip user: Nate Coraor <nate(a)bx.psu.edu> date: Wed Jul 30 11:13:20 2014 -0400 summary: Updated tag latest_2014.06.02 for changeset 8c30e91bc9ae What would I need to do if I wanted to include this functionality ? Regards jamesp J ames Mullan

3 2

SLURM timeouts
by Sytchev, Ilya 25 Oct '14

25 Oct '14

Hi, I'm running a fork of galaxy-central latest_2014.08.11. The instance is configured to run jobs on a SLURM cluster. The problem is that the SLURM controller sometimes becomes too busy which results in errors like: galaxy.jobs.runners.drmaa INFO 2014-10-23 21:10:47,768 (1813/22896754) job left DRM queue with following message: code 1: slurm_load_jobs error: Socket timed out on send/recv operation,job_id: 22896754 This causes Galaxy to assume that the job has failed: galaxy.jobs.runners ERROR 2014-10-23 21:10:47,881 (1813/22896754) Job output not returned from cluster: [Errno 2] No such file or directory: '/n/regal/stemcellcommons/galaxy-stage/job_working_directory/001/1813/galax y_1813.o' This happens with both galaxy.jobs.runners.drmaa:DRMAAJobRunner and galaxy.jobs.runners.slurm:SlurmJobRunner. Is there any way to handle this condition in Galaxy? Thanks, Ilya

1 0

filtering a <param> of type 'data' so only one type is available
by Dan Tenenbaum 24 Oct '14

24 Oct '14

Hi, I have a tool wrapper with a <param> of type="data". Currently, this renders as a text box with a drop down list that has 52 items in it (the number of things I have in my history, I guess). I'd like to filter this list so that only some items in the history (in my case any item whose name ends with '.rda') are shown in this dropdown. It seems like maybe the 'format' parameter to the 'param' tag is what I want, but: 1) I tried format="rda" and that didn't seem to change anything. 2) "rda" is not in the datatypes.conf file which the documentation suggests is required? (.rda is an extension used for serialized R objects.) The tool I'm working on only accepts rda files as input and the reason I'm asking for this is that it is all too easy to accidentally feed it a file of some other type. If the dropdown could be filtered so that only items which will work with the tool are shown, that would be great. Is there a way to do that? Thanks, Dan

4 5

Determining galaxy api path automatically from a tool via dynamic_options?
by Dooley, Damion 23 Oct '14

23 Oct '14

After uploading my beta status tool to our test toolshed, I tried installing it on another of our galaxy servers but it wasn't able to find the api endpoint url. Is there a way to automatically do this within tool forms? I was hoping to avoid hardcoding this as part of the tool installation process. I'd thought I had it automated: In tool's form, a dynamic_options param <param name="api_url" display="radio" type="drill_down" label="For user with Galaxy API Key" dynamic_options="vdb_init_tool_user(__trans__)" /> was able to pass "trans" galaxy structure so I got as far as this python: self.api_url = 'http://' + trans.request.host ... but this yeilds http://acme.com/ rather than what's needed, the path to a particular installation: http://acme.com/galaxy/ I couldn't find a way to read the /galaxylab/ setting from the (slightly dated) universe_wsgi.ini config file - without say reading the file and parsing it. Is there an __app__.config.prefix or somesuch variable where this info is? Thanks! Damion

2 2