February 2015 - galaxy-dev - lists.galaxyproject.org

Galaxy CloudMan instance - missing link 'display at UCSC' for bed files
by Anne Pajon 16 Feb '15

16 Feb '15

Dear All, I’ve installed a Galaxy CloudMan instance on amazon for a training course we will be giving at the end of the month, as a re-run of a course we did last September. I am facing a problem as I am unable to get the link 'display at UCSC’ for our bed files, and it was not the case back in September. So far, I’ve edited /mnt/galaxy/galaxy-app/datatypes_conf.xml in our cloud instance and uncommented <display file="ucsc/interval_as_bed.xml" /> but it did not change anything after a re-start. Any ideas what could have changed? Or how I could turn this back on? Any help would be greatly appreciated otherwise I will have to rewrite part of the training materials… Many thanks in advance. Best wishes, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk | +44 (0)1223 769 631

2 3

R lib dependencies in the toolshed
by Nikos Sidiropoulos 14 Feb '15

14 Feb '15

Hi all, I have a package repository that installs an R package and another repository with a set of tools that use that package repository. Tools that use an Rscript command interpreter load that module successfully, e.g.: "normalize.xml" <requirements> <requirement type="package" version="3.1.1">R_3_1_1</requirement> <requirement type="R-module">RNAprobR</requirement> <requirement type="package" version="1.0.0">RNAprobR</requirement> <requirement type="set_environment">RNA_PROBING_SCRIPT_PATH</requirement> </requirements> <command interpreter="Rscript"> normalize.R .... </command> "normalize.R" suppressMessages(require(RNAprobR)) ... But there is a tool which executes an Rscript indirectly, meaning: "summarize_unique_barcodes.sh" <requirements> <requirement type="package" version="4.1.0">gnu_awk</requirement> <requirement type="package" version="0.1.19">samtools</requirement> <requirement type="package" version="3.1.1">R_3_1_1</requirement> <requirement type="R-module">RNAprobR</requirement> <requirement type="package" version="1.0.0">RNAprobR</requirement> <requirement type="set_environment">RNA_PROBING_SCRIPT_PATH</requirement> </requirements> <command interpreter="bash"> summarize_unique_barcodes.sh ... -r \$RNA_PROBING_SCRIPT_PATH </command> "summarize_unique_barcodes.sh" .... #Produce k2n file if [ "$k2n" == "True" ]; then Rscript $R_SCRIPT_PATH/k2n.R merged_temp.gz $output_dir/unique_barcodes.txt $output_dir/k2n.txt fi "k2n.R" suppressMessages(require(RNAprobR)) I always get the following error: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, : there is no package called 'RNAprobR' A solution to the problem could be to pass the R library path of the RNAprobR module or the installation folder of package "RNAprobR" as a parameter to the bash wrapper and then the k2n.R script but I can't seem to be able to do it. Do you have any suggestions? Regards, Nikos

2 2

Re: [galaxy-dev] Tool test failure
by Dooley, Damion 13 Feb '15

13 Feb '15

I tried using the default config/job_conf.xml but that didn't make a difference. I'll just try testing on a different solo galaxy instance, and will report back. I'm guessing it will work elsewhere. Message: 3 ... Subject: Re: [galaxy-dev] Tool test failure - I could use some help... ... Thanks for the tips, Peter. Hmm. No amount of playing with my galaxy-dist/tool_conf.xml.sample file is enabling the tool to be found by run_tests.sh . In the run_functional_tests.html I see:

1 0

restricting new accounts
by Michael Thon 13 Feb '15

13 Feb '15

I'm installing Galaxy for the first time on my server, which will be used only for our own research and possibly for teaching. I want to restrict new user signups. Is there any way to do that? Thanks!

2 1

Tool test failure - I could use some help...
by Dooley, Damion 12 Feb '15

12 Feb '15

I was trying to track down why an attempt to test my tool is eventually failing with: "ValueError: No such test TestForTool_ffp_phylogeny" In my case the tool code is located in a Galaxy's tool shed folder, so I'm guessing that might have something to do with it? Has someone come up with a thorough checklist on issues around this? Tool xml file header: <tool id="ffp_phylogeny" name="Feature Frequency Profile Phylogeny" version="0.1.00"> Using John Chilton's ready made sqlite db (though our live server uses a postgres one) > export GALAXY_TEST_DB_TEMPLATE=tmp/db_gx_rev_0120.sqlite Command line (recent Update tag latest_2015.01.13 for changeset 0cb6cec4ee57): > run_tests.sh -id ffp_phylogeny begins the process alright. *********************************** My galaxy-dist/tool_shed.xml.sample where I understand references to the tool are supposed to go (and I confirmed the tool file absolute path): <?xml version="1.0"?> <toolbox tool_path="../shed_tools"> <section id="bccdc_tools" name="BCCDC Tools" version=""> <tool file="/usr/local/galaxy/production1/shed_tools/salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/ffp_phylogeny/eb1924fbd3ba/ffp_phylogeny/ffp_phylogeny.xml" guid="salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/ffp_phylogeny/ffp_phylogeny/0.1.00"> <tool_shed>salk.bccdc.med.ubc.ca/toolshed</tool_shed> <repository_name>ffp_phylogeny</repository_name> <repository_owner>ddooley</repository_owner> <installed_changeset_revision>eb1924fbd3ba</installed_changeset_revision> <id>salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/ffp_phylogeny/ffp_phylogeny/0.1.00</id> <version>0.1.00</version> </tool> <tool file="salk.bccdc.med.ubc.ca/toolshed/repos/damion/versioned_data/e9ebe2959300/versioned_data/versioned_data.xml" guid="salk.bccdc.med.ubc.ca/toolshed/repos/damion/versioned_data/versioned_data/0.1.00"> <tool_shed>salk.bccdc.med.ubc.ca/toolshed</tool_shed> <repository_name>versioned_data</repository_name> <repository_owner>damion</repository_owner> <installed_changeset_revision>e9ebe2959300</installed_changeset_revision> <id>salk.bccdc.med.ubc.ca/toolshed/repos/damion/versioned_data/versioned_data/0.1.00</id> <version>0.1.00</version> </tool> </toolbox> Thanks for any pointers! Soon as tests fly, I'll be contributing the tools up to the test tooshed. Regards, Damion Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

2 2

tool xml to fetch choices from a URL
by Langhorst, Brad 12 Feb '15

12 Feb '15

Hi: I need users to specify a project when they run a tool… I would like this to be a select list whose set of options comes from a remote url. What’s the best way to achieve that UI? Should I try to add a from_url attribute to the options tag? Thanks, Brad -- Brad Langhorst, Ph.D. Applications and Product Development Scientist

2 1

Question Regarding fetch_eggs &c. vs pip
by Stephen Rosen 12 Feb '15

12 Feb '15

Hi Galaxy Dev, I've been looking at the setup scripts for Galaxy to try to understand a problem I recently had provisioning a Galaxy server. I will readily admit that I have not read all of the relevant code top-to-bottom, but I have at least skimmed all of it and read much of it. Sorry if these questions are answered somewhere in Trello, the Wiki, or somewhere else, but I was not able to find answers in any public locations. As a small bit of probably irrelevant context: I'm working with the Globus Genomics group on the DevOps side of things. We're using Chef. I've only just started working with the group in the past couple of weeks (so my expertise with Galaxy itself is limited to nonexistent). First, to describe the problem: We want to provision a server running Galaxy without explicitly wrapping it in a virtualenv. Unless I missed something, that means that it's using system python. When we use pip to install a package from a git repository before running the setup scripts, fetch_eggs fails saying it failed to fetch WebError 0.8a If we install the same package from git with `pip install --egg ...` we get a hunky-dory system where everything seems to work. As far as I can tell, there is no reason that this should be the case. Sure, putting the git source directly into site-packages might cause issues upon installation, but EggNotFetchable exceptions should only be thrown if the egg actually can't be pulled down from eggs.g2.bx.psu.edu , right? I don't feel comfortable trying to make further progress on my provisioning scripts without knowing why this is happening. I'd hate to be bitten by this later on in the process. Yes, the package in question may have poor behavior (likely it does), but that doesn't change the fact that the error is totally misleading. Furthermore, it doesn't appear that this poor behavior impedes me from doing a pip install of the WebError package or any other packages from PyPI. In case someone else wants to test to replicate, this is the command being used: pip install --egg git+ https://github.com/globusonline/python-nexus-client@599f04edef6b72569b7a5b2… problems occur if you omit `--egg`. Second, a question about the rationale for Galaxy's egg handling: Why is all of this wrapped up in these scripts in the first place? I understand that pip might not be present on every platform, and I don't mean to question a decision to support systems without it. However, as detailed below, Galaxy does not support any platforms which are incapable of running pip. Furthermore, pip is being pushed by the Python maintainers over easy_install, so it's not like there isn't a clear choice in terms of which one to support. Perhaps most importantly, there don't appear to be any clear-cut options to do the following, which I would consider a more ordinary workflow: - Run a galaxy script (like check_eggs) to generate a list of packages from eggs.g2.bx.psu.edu for platform (redirect output to requirements.txt or similar) - `pip install -r requirements.txt` The above could be part of a galaxy provisioning script, rather than exposed to the administrator. That also makes it significantly easier to control and manage the virtualenv in which we run Galaxy, since we don't have to worry about egg-related logic that we don't control and we know that the virtualenv's bin dir will be earlier in the PATH than the system pip's dir. Yes, I said above that our setup is presently using system python -- switching to a virtualenv is one of the many items on my to-do list. In fact, I would expect that the default, desired setup for Galaxy would be to put it in a virtualenv, rather than using system python, and to use pip, rather than fetch_eggs.py and company. When I look at the logic being used here, especially at https://bitbucket.org/galaxy/galaxy-central/src/f0ae870b22e9/lib/galaxy/egg… , it looks like a solution built exclusively for platforms on which pip is not installed. According to the wiki, Galaxy support only goes back as far as 2.6, and get-pip.py supports 2.6, so there is no way of building a Galaxy server on a platform that can't also have pip installed. Adding pip to the requirements for Galaxy would not be particularly onerous, and may simplify things significantly (no need to bundle get-pip.py or similar). As a last note about the misleading error from the fetch_eggs script, telling me that WebError is "NotFetchable". I probably wouldn't have much of a complaint about this if the error had been more on target and I hadn't felt the need to do things like patch lib/galaxy/eggs to print a stacktrace. For example, if the script detected that there was a source installed package which was getting underfoot, it should have alerted me or even suggested installing my various packages in egg format. That kind of error detection is hard to maintain and hard to keep accurate since the Galaxy team's priority is to build Galaxy, not a package manager. This, I suppose, circles back to an earlier question: why isn't Galaxy using a python package manager to... manage its packages? At the very least something along the lines of `./scripts/common_setup.sh --use-pip` should be added. It doesn't seem like it would be that hard to implement -- but I feel like my lack of knowledge of Galaxy disqualifies me from building the changeset reliably. Of course, if no one objects, I will readily do so anyway (when I have the time) as a proof of concept. Barring even that improvement, the Wiki page at https://wiki.galaxyproject.org/Admin/Config/Eggs should definitely be updated to include some note on why Galaxy has this complex logic for fetching eggs instead of using pip. A quick tl;dr and summary: - I don't have extensive experience with Galaxy, so I may not know what I'm talking about. - fetch_eggs.py can be made to raise EggNotFetchable by doing a pip install from a VCS without using `--egg`. This is a bug in fetch_eggs.py - All supported platforms for Galaxy support pip - Galaxy should have an option to use pip to download its packages over https from eggs.g2.bx.psu.edu - Galaxy should probably default to using pip when it's available, since its failure modes are significantly better than a home-brewed package manager -- this also leads to good behavior in virtualenvs. Best regards, and many thanks for your time and attention, -Stephen

5 9

BWA alignment problem in Galaxy
by Xiong, Min, 12 Feb '15

12 Feb '15

Hi, When I use BWA alignment, FASTQ file can't be detected. What reason cause it? The attached file contains figure from galaxy, you will see I have good fastq file which can be reported by FastQC. Thanks Min Xiong, Ph.D. Post-doctoral Fellow Shui Q. Ye Lab Group Division of Experimental and Translational Genetics Department of Pediatrics Children's Mercy Hospital 2401 Gillham Road Kansas City, MO 64108 ________________________________ Electronic mail from Children's Mercy Hospitals and Clinics. This communication is intended only for the use of the addressee. It may contain information that is privileged or confidential under applicable law. If you are not the intended recipient or the agent of the recipient, you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited. If you have received this communication in error, please immediately forward the message to Children's Mercy Hospital's Information Security Officer via return electronic mail at informationsecurityofficer(a)cmh.edu and expunge this communication without making any copies. Thank you for your cooperation.

3 2

Adding options to a tool
by Scott Szakonyi 12 Feb '15

12 Feb '15

Hello all, I am a new Galaxy admin/developer, so if this is as stupid question please be kind. I am trying to add the ability for a user to choose additional parameters for Tophat for Illumina. After analyzing the Python and XML files in /galaxy-dist/tools/ngs_rna, I came to the conclusion that modifying these files would give me the desired results. However, no changes that I make to these files seem to have any effect whatsoever on the user experience inside the Galaxy server. I can not find any other likely candidates inside galaxy-dist. Am I looking in the wrong place? Is it even possible to add parameters to a tool interface? Thanks in advance for any help you can offer. -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu

3 2

Accessing Specific Entries in Data Table
by Jeremy Liu 10 Feb '15

10 Feb '15

Galaxy Devs, I have been following the documentation located at ( https://wiki.galaxyproject.org/Admin/Tools/Data%20Tables) to configure my tools to use files installed into data tables via a data manager. I have configured my tool region_motif_intersect.r to provide users the options drawn from the data table with: <param name="bgz_file" type="select" label="Using motif database BGZ file"> <options from_data_table="motif_databases"/> </param> However, is there a way to directly pass the file paths of elements in the data table? I would like to implement a drop down menu so that users select which database to reference, and the tool config selects the 2-3 corresponding files from the data table (as well as checks if they exist/have been downloaded). I have attached a screenshot of what the data table looks like. In addition, is there a way to filter the options returned by <options from_data_table> so that only a subset of the files in the data table are returned? For reference, the tools and the data manager can be found, respectively here: https://testtoolshed.g2.bx.psu.edu/view/jeremyjliu/region_motif_enrichment https://testtoolshed.g2.bx.psu.edu/view/jeremyjliu/region_motif_data_manager Thank you for your help. Jeremy Liu jeremy.liu(a)yale.edu

1 0