- galaxy-dev - lists.galaxyproject.org

Tool test failure - I could use some help...
by Dooley, Damion 13 Feb '15

13 Feb '15

I was trying to track down why an attempt to test my tool is eventually failing with: "ValueError: No such test TestForTool_ffp_phylogeny" In my case the tool code is located in a Galaxy's tool shed folder, so I'm guessing that might have something to do with it? Has someone come up with a thorough checklist on issues around this? Tool xml file header: <tool id="ffp_phylogeny" name="Feature Frequency Profile Phylogeny" version="0.1.00"> Using John Chilton's ready made sqlite db (though our live server uses a postgres one) > export GALAXY_TEST_DB_TEMPLATE=tmp/db_gx_rev_0120.sqlite Command line (recent Update tag latest_2015.01.13 for changeset 0cb6cec4ee57): > run_tests.sh -id ffp_phylogeny begins the process alright. *********************************** My galaxy-dist/tool_shed.xml.sample where I understand references to the tool are supposed to go (and I confirmed the tool file absolute path): <?xml version="1.0"?> <toolbox tool_path="../shed_tools"> <section id="bccdc_tools" name="BCCDC Tools" version=""> <tool file="/usr/local/galaxy/production1/shed_tools/salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/ffp_phylogeny/eb1924fbd3ba/ffp_phylogeny/ffp_phylogeny.xml" guid="salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/ffp_phylogeny/ffp_phylogeny/0.1.00"> <tool_shed>salk.bccdc.med.ubc.ca/toolshed</tool_shed> <repository_name>ffp_phylogeny</repository_name> <repository_owner>ddooley</repository_owner> <installed_changeset_revision>eb1924fbd3ba</installed_changeset_revision> <id>salk.bccdc.med.ubc.ca/toolshed/repos/ddooley/ffp_phylogeny/ffp_phylogeny/0.1.00</id> <version>0.1.00</version> </tool> <tool file="salk.bccdc.med.ubc.ca/toolshed/repos/damion/versioned_data/e9ebe2959300/versioned_data/versioned_data.xml" guid="salk.bccdc.med.ubc.ca/toolshed/repos/damion/versioned_data/versioned_data/0.1.00"> <tool_shed>salk.bccdc.med.ubc.ca/toolshed</tool_shed> <repository_name>versioned_data</repository_name> <repository_owner>damion</repository_owner> <installed_changeset_revision>e9ebe2959300</installed_changeset_revision> <id>salk.bccdc.med.ubc.ca/toolshed/repos/damion/versioned_data/versioned_data/0.1.00</id> <version>0.1.00</version> </tool> </toolbox> Thanks for any pointers! Soon as tests fly, I'll be contributing the tools up to the test tooshed. Regards, Damion Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

2 2

tool xml to fetch choices from a URL
by Langhorst, Brad 12 Feb '15

12 Feb '15

Hi: I need users to specify a project when they run a tool… I would like this to be a select list whose set of options comes from a remote url. What’s the best way to achieve that UI? Should I try to add a from_url attribute to the options tag? Thanks, Brad -- Brad Langhorst, Ph.D. Applications and Product Development Scientist

2 1

Question Regarding fetch_eggs &c. vs pip
by Stephen Rosen 12 Feb '15

12 Feb '15

Hi Galaxy Dev, I've been looking at the setup scripts for Galaxy to try to understand a problem I recently had provisioning a Galaxy server. I will readily admit that I have not read all of the relevant code top-to-bottom, but I have at least skimmed all of it and read much of it. Sorry if these questions are answered somewhere in Trello, the Wiki, or somewhere else, but I was not able to find answers in any public locations. As a small bit of probably irrelevant context: I'm working with the Globus Genomics group on the DevOps side of things. We're using Chef. I've only just started working with the group in the past couple of weeks (so my expertise with Galaxy itself is limited to nonexistent). First, to describe the problem: We want to provision a server running Galaxy without explicitly wrapping it in a virtualenv. Unless I missed something, that means that it's using system python. When we use pip to install a package from a git repository before running the setup scripts, fetch_eggs fails saying it failed to fetch WebError 0.8a If we install the same package from git with `pip install --egg ...` we get a hunky-dory system where everything seems to work. As far as I can tell, there is no reason that this should be the case. Sure, putting the git source directly into site-packages might cause issues upon installation, but EggNotFetchable exceptions should only be thrown if the egg actually can't be pulled down from eggs.g2.bx.psu.edu , right? I don't feel comfortable trying to make further progress on my provisioning scripts without knowing why this is happening. I'd hate to be bitten by this later on in the process. Yes, the package in question may have poor behavior (likely it does), but that doesn't change the fact that the error is totally misleading. Furthermore, it doesn't appear that this poor behavior impedes me from doing a pip install of the WebError package or any other packages from PyPI. In case someone else wants to test to replicate, this is the command being used: pip install --egg git+ https://github.com/globusonline/python-nexus-client@599f04edef6b72569b7a5b2… problems occur if you omit `--egg`. Second, a question about the rationale for Galaxy's egg handling: Why is all of this wrapped up in these scripts in the first place? I understand that pip might not be present on every platform, and I don't mean to question a decision to support systems without it. However, as detailed below, Galaxy does not support any platforms which are incapable of running pip. Furthermore, pip is being pushed by the Python maintainers over easy_install, so it's not like there isn't a clear choice in terms of which one to support. Perhaps most importantly, there don't appear to be any clear-cut options to do the following, which I would consider a more ordinary workflow: - Run a galaxy script (like check_eggs) to generate a list of packages from eggs.g2.bx.psu.edu for platform (redirect output to requirements.txt or similar) - `pip install -r requirements.txt` The above could be part of a galaxy provisioning script, rather than exposed to the administrator. That also makes it significantly easier to control and manage the virtualenv in which we run Galaxy, since we don't have to worry about egg-related logic that we don't control and we know that the virtualenv's bin dir will be earlier in the PATH than the system pip's dir. Yes, I said above that our setup is presently using system python -- switching to a virtualenv is one of the many items on my to-do list. In fact, I would expect that the default, desired setup for Galaxy would be to put it in a virtualenv, rather than using system python, and to use pip, rather than fetch_eggs.py and company. When I look at the logic being used here, especially at https://bitbucket.org/galaxy/galaxy-central/src/f0ae870b22e9/lib/galaxy/egg… , it looks like a solution built exclusively for platforms on which pip is not installed. According to the wiki, Galaxy support only goes back as far as 2.6, and get-pip.py supports 2.6, so there is no way of building a Galaxy server on a platform that can't also have pip installed. Adding pip to the requirements for Galaxy would not be particularly onerous, and may simplify things significantly (no need to bundle get-pip.py or similar). As a last note about the misleading error from the fetch_eggs script, telling me that WebError is "NotFetchable". I probably wouldn't have much of a complaint about this if the error had been more on target and I hadn't felt the need to do things like patch lib/galaxy/eggs to print a stacktrace. For example, if the script detected that there was a source installed package which was getting underfoot, it should have alerted me or even suggested installing my various packages in egg format. That kind of error detection is hard to maintain and hard to keep accurate since the Galaxy team's priority is to build Galaxy, not a package manager. This, I suppose, circles back to an earlier question: why isn't Galaxy using a python package manager to... manage its packages? At the very least something along the lines of `./scripts/common_setup.sh --use-pip` should be added. It doesn't seem like it would be that hard to implement -- but I feel like my lack of knowledge of Galaxy disqualifies me from building the changeset reliably. Of course, if no one objects, I will readily do so anyway (when I have the time) as a proof of concept. Barring even that improvement, the Wiki page at https://wiki.galaxyproject.org/Admin/Config/Eggs should definitely be updated to include some note on why Galaxy has this complex logic for fetching eggs instead of using pip. A quick tl;dr and summary: - I don't have extensive experience with Galaxy, so I may not know what I'm talking about. - fetch_eggs.py can be made to raise EggNotFetchable by doing a pip install from a VCS without using `--egg`. This is a bug in fetch_eggs.py - All supported platforms for Galaxy support pip - Galaxy should have an option to use pip to download its packages over https from eggs.g2.bx.psu.edu - Galaxy should probably default to using pip when it's available, since its failure modes are significantly better than a home-brewed package manager -- this also leads to good behavior in virtualenvs. Best regards, and many thanks for your time and attention, -Stephen

5 9

BWA alignment problem in Galaxy
by Xiong, Min, 12 Feb '15

12 Feb '15

Hi, When I use BWA alignment, FASTQ file can't be detected. What reason cause it? The attached file contains figure from galaxy, you will see I have good fastq file which can be reported by FastQC. Thanks Min Xiong, Ph.D. Post-doctoral Fellow Shui Q. Ye Lab Group Division of Experimental and Translational Genetics Department of Pediatrics Children's Mercy Hospital 2401 Gillham Road Kansas City, MO 64108 ________________________________ Electronic mail from Children's Mercy Hospitals and Clinics. This communication is intended only for the use of the addressee. It may contain information that is privileged or confidential under applicable law. If you are not the intended recipient or the agent of the recipient, you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited. If you have received this communication in error, please immediately forward the message to Children's Mercy Hospital's Information Security Officer via return electronic mail at informationsecurityofficer(a)cmh.edu and expunge this communication without making any copies. Thank you for your cooperation.

3 2

Adding options to a tool
by Scott Szakonyi 12 Feb '15

12 Feb '15

Hello all, I am a new Galaxy admin/developer, so if this is as stupid question please be kind. I am trying to add the ability for a user to choose additional parameters for Tophat for Illumina. After analyzing the Python and XML files in /galaxy-dist/tools/ngs_rna, I came to the conclusion that modifying these files would give me the desired results. However, no changes that I make to these files seem to have any effect whatsoever on the user experience inside the Galaxy server. I can not find any other likely candidates inside galaxy-dist. Am I looking in the wrong place? Is it even possible to add parameters to a tool interface? Thanks in advance for any help you can offer. -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu

3 2

Accessing Specific Entries in Data Table
by Jeremy Liu 10 Feb '15

10 Feb '15

Galaxy Devs, I have been following the documentation located at ( https://wiki.galaxyproject.org/Admin/Tools/Data%20Tables) to configure my tools to use files installed into data tables via a data manager. I have configured my tool region_motif_intersect.r to provide users the options drawn from the data table with: <param name="bgz_file" type="select" label="Using motif database BGZ file"> <options from_data_table="motif_databases"/> </param> However, is there a way to directly pass the file paths of elements in the data table? I would like to implement a drop down menu so that users select which database to reference, and the tool config selects the 2-3 corresponding files from the data table (as well as checks if they exist/have been downloaded). I have attached a screenshot of what the data table looks like. In addition, is there a way to filter the options returned by <options from_data_table> so that only a subset of the files in the data table are returned? For reference, the tools and the data manager can be found, respectively here: https://testtoolshed.g2.bx.psu.edu/view/jeremyjliu/region_motif_enrichment https://testtoolshed.g2.bx.psu.edu/view/jeremyjliu/region_motif_data_manager Thank you for your help. Jeremy Liu jeremy.liu(a)yale.edu

1 0

gui and command line trouble
by Edlund, Anna 10 Feb '15

10 Feb '15

Hi there. I am trying to run MGTAXA to generate taxonomic predictions on assembled contigs in fasta format. I try to use both the gui at http://mgtaxa.jcvi.org/ with username aedlund@jcvi and 6926edlund(a)gmail.com. Both of these jobs have been running for days (they are yellow). When I try and use the standard command via the terminal nothing happens either. This command was working earlier for bacteria classification. Do you have any ideas of what I can do to use this tool? Thank you for your awesome help. Best, Anna

1 0

"This job is waiting to run"
by Ryan G 06 Feb '15

06 Feb '15

I tried setting up Galaxy to use Sun Grid Engine, but failed miserably. I removed my config/job_conf.xml hoping Galaxy would revert to its default state, but jobs are no longer being dispatched. I don't see any errors in the log. How can I get Galaxy to 1) use SGE, or 2) revert to its default state for running jobs. The page, https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster isn't 100% clear on how to set up a simply configuration for SGE. I'd be happy to contribute mine as an example, if I can get it running.

2 2

backtrace -> bug?
by Langhorst, Brad 06 Feb '15

06 Feb '15

Is this a bug you want in trello? got this back trace when attempting to get updates to the picard tool 172.17.130.111 - - [06/Feb/2015:13:45:55 -0400] "POST /library_common/library_item_updates HTTP/1.0" 200 - "http://galaxy.neb.com/library_common/browse_library?sort=name&f-description…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36" galaxy.tools.data WARNING 2015-02-06 13:45:57,640 Cannot find index file '/mnt/galaxy/data/galaxy/tmp/tmp-toolshed-gmfcr_ys2eZ/picard_index.loc' for tool data table 'picard_indexes' galaxy.tools.data DEBUG 2015-02-06 13:45:57,641 Loading another instance of data table 'picard_indexes', attempting to merge content. galaxy.tools.data DEBUG 2015-02-06 13:45:58,724 Loading another instance of data table 'picard_indexes', attempting to merge content. galaxy.tools.data DEBUG 2015-02-06 13:45:59,161 Loading another instance of data table 'picard_indexes', attempting to merge content. galaxy.tools.data DEBUG 2015-02-06 13:45:59,307 Loading another instance of data table 'picard_indexes', attempting to merge content. galaxy.tools.data DEBUG 2015-02-06 13:45:59,814 Loading another instance of data table 'picard_indexes', attempting to merge content. galaxy.tools.data DEBUG 2015-02-06 13:45:59,981 Loading another instance of data table 'picard_indexes', attempting to merge content. 172.17.130.111 - - [06/Feb/2015:13:46:00 -0400] "POST /library_common/library_item_updates HTTP/1.0" 200 - "http://galaxy.neb.com/library_common/browse_library?sort=name&f-description…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36" galaxy.tools.data DEBUG 2015-02-06 13:46:01,205 Loading another instance of data table 'picard_indexes', attempting to merge content. galaxy.tools.data DEBUG 2015-02-06 13:46:01,380 Loading another instance of data table 'picard_indexes', attempting to merge content. 172.17.130.190 - - [06/Feb/2015:13:45:50 -0400] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=picard&owner=devteam&changeset_revision=e0232cbac965&latest_changeset_revision=bf1c3f9f8282&latest_ctx_rev=3 HTTP/1.0" 500 - "http://galaxy.neb.com/admin_toolshed/browse_repositories" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36" Error - <type 'exceptions.AttributeError'>: 'list' object has no attribute 'iteritems' URL: http://galaxy.neb.com/admin_toolshed/update_to_changeset_revision?tool_shed… File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/remoteuser.py', line 107 in __call__ return self.app( environ, start_response ) File '/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1954 in update_to_changeset_revision irmm.generate_metadata_for_changeset_revision() File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 443 in generate_metadata_for_changeset_revision metadata_dict ) File '/mnt/galaxy/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 648 in generate_tool_metadata for param_name, values in ttb.inputs.iteritems(): AttributeError: 'list' object has no attribute 'iteritems'

1 0

galaxy of dm6
by Jian Lu 06 Feb '15

06 Feb '15

Hi folks, I wonder when you guys will include the newly updated genome assembly of Drosophila melanogaster (dm6) and the corresponding multiple alignments into the galaxy webserver. Best, Jian Lu -- Jian Lu, Ph. D. Principal Investigator Center for Life Sciences (CLS) Center of Bioinformatics State Key Laboratory of Protein and Plant Gene Research School of Life Sciences Peking University Beijing, China, 100871 http://evolution-pku.org/

1 0