- galaxy-dev - lists.galaxyproject.org

Missing configuration and location files automatic creation
by Julien Seiler 22 Sep '14

22 Sep '14

Hi all, I have notice that the run.sh script is creating automatically all missing configuration and location files to make sure Galaxy can work properly at first launch. However, the script is not taking care of the actual settings of an existing universe_wsgi.ini file. As a result, if the universe_wsgi.ini file says that job_conf.xml is located at foo/bar location, the run.sh will create a job_conf.xml file at the root of the galaxy source code anyway. I'm thinking of improving the run.sh behavior to respect file path declared in an existing universe_wsgi.ini file in order not to create unused config or location files. I have currently started to implement this behavior in a small Python script (more handy than pure sh). What do you think about this concern ? Should I go for a pull request at any point ? Regards, -- Julien @julozi

2 3

Workflow param error leads to template error
by Paul-Michael Agapow 19 Sep '14

19 Sep '14

Looking for pointers on what might be causing this problem: A user had a moderately complicated workflow that when run under certain parameter values results in a traceback that ends: Module workflow_run_mako:232 in render_body >> for ic in oc.input_step.module.get_data_inputs(): AttributeError: 'WorkflowStep' object has no attribute 'module' After some puzzling, I realized one of the parameters was outside the allowed range (a max on an integer param) and that Galaxy was trying to render "run.mako" flagging up the error but erroring out. Why Galaxy tries to call this non-existent member is unclear to me. Any insight or places I should start exploring? Galaxy version: various (started with an 6-month old one, upgraded to latest production to see if it would fix error, it didn't) Hosting OS: CentOS 6 ------ Paul Agapow (paul-michael.agapow(a)hpa.org.uk) Bioinformatics, Health Protection Agency (UK) ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

3 3

find bam index files
by Ulf Schaefer 18 Sep '14

18 Sep '14

Dear all For each bam file in my history I can download the associated bai (bam index) file. I assume these files are stored somewhere under /mount/galaxy/database/files/_metadata_files. Correct? Is there an easy way to find the bam index file for a bam file, given only the internal file name of the bam (e.g. /mont/galaxy/database/files/089/dataset_89231.dat)? I am asking because I would like to use the files_to_ftp tool to automatically download bams together with their associated indices. Thanks Ulf ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

2 1

Question about uploading a large application to the tool-shed
by George Weingart 17 Sep '14

17 Sep '14

Hello, I have an application that is working perfectly in our local Galaxy (Not using tool-shed) I would like to upload it to the tool-shed (I have uploaded other applications to the tool-shed before) The size of the directory (//usr/local/galaxy-dist/tools/metaphlan2) is 2.1 GB and after I tar it it is 1.1 GB. The question: Is it too big to be uploaded to the tool-shed? Thanks! George Weingart

3 2

Problem writing functional tool tests with 'repeat' & 'conditional' tags
by Peter Briggs 17 Sep '14

17 Sep '14

Hello all I'm experiencing problems with specifying some tests for a Galaxy tool I've written, which are related to setting the value of a 'conditional' parameter that is defined within a 'repeat' tag. The bones of the tool XML look like: ... <repeat name="operations" title="..." min="1"> <conditional name="operation"> <param name="name" type="select" label="..."> <option selected="true" value="SLIDINGWINDOW">Sliding window trimming...</option>  </param> <when value="SLIDINGWINDOW"> ... When I run the functional tests I get: base.twilltestcase: ERROR: In submit_form, continuing, but caught exception. Traceback (most recent call last): File "XXXXX/galaxy-dist/test/base/twilltestcase.py", line 1910, in submit_form for value in kwd[ control.name ]: KeyError: 'operations_0|operation|name' Is there a way to specify the the value in the tests? I've tried both <param name="operations_0.operation.name' value="..." /> and <param name="operations_0|operation|name' value="..." /> in my <test>s, but neither form seem to change the exception from the tests. Any suggestions are greatly appreciated! Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

Problem with tool dependencies when running functional tool tests
by Peter Briggs 17 Sep '14

17 Sep '14

Hello again As an addendum to my previous message, I'm also having general problems running the functional tests for any tool, due to unresolved tool dependencies. For example if I clone and configure a fresh galaxy-dist instance and install the FastQC tool from the main toolshed, and then run e.g.: $ export GALAXY_TOOL_DEPENDENCY_DIR=/home/pjb/galaxies/kellerman/tool_dependencies ; sh run_functional_tests.sh -installed In the stdout I see output like: ... galaxy.tools.deps: DEBUG: Building dependency shell command for dependency 'FastQC' galaxy.tools.deps: WARNING: Failed to resolve dependency on 'FastQC', ignoring ... and in the run_functional_tests.html report I see: Traceback (most recent call last): File "/home/pjb/galaxies/kellerman/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/e28c965eeed4/fastqc/rgFastQC.py", line 204, in <module> assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##rgFastQC.py error - cannot find executable /fastqc As far as I can make out I'm correctly following the instructions at https://wiki.galaxyproject.org/TestingInstalledTools so I'm wondering if anyone can point out what it is that I'm missing. No doubt it's blindingly obvious, but I just can't see it. Thanks again for any help Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

1 0

Dependencies within dependencies
by Christophe Antoniewski 16 Sep '14

16 Sep '14

I am trying to move the package_r_3_0_3 repository owned by devteam from the main toolshed to our local toolshed. This does not work as some dependencies required for package_r_3_0_3 are not exported (export capsule) and not included in the archive. For instance, package_freetype_2_5_2 and package_libpng_1_6_7 are present in the capsule and thus moved to the local toolshed, but not package_readline_6_2, package_cairo_1_12_14 and package_pixman_0_32_4. Is it a bug (I would expect a sort of recursivity in the export process) or did I miss some logics behind this ? And do we have to manually export-import the missing dependencies to the local toolshed ? Cheers Chris Christophe Antoniewski Drosophila Genetics and Epigenetics Laboratoire de Biologie du Développement 9, Quai St Bernard, Boîte courrier 24 75252 Paris Cedex 05 christophe.antoniewski(a)upmc.fr http://bio-dev.snv.jussieu.fr/ Tel +33 1 44 27 34 39 Fax +33 1 44 27 34 45 Mobile +33 6 68 60 51 50 http://drosophile.org http://www.sciencesenmarche.org/

2 1

Main tool shed installation issues
by Yves Gagnon 16 Sep '14

16 Sep '14

Hello fellow Galaxy-Dev users, We are having an issue trying to install tools from the main tool shed. Trying to install bowtie2 (or any other tool for that matter) gives us the following error message : Error Traceback: ⇝ URLError: <urlopen error [Errno -2] Name or service not known> Looking further in the error traceback, the url not found in question is this one (in the case of bowtie2): https://toolshed.g2.bx.psu.edu/repository/get_repository_information?reposi… Indeed, copy pasting this URL into a browser gives me the Internal server error page. Anyone knows if this is a widespread issue? Thanks for your feedback! Yves

2 3

New version of vcflib wrappers
by Anton Nekrutenko 15 Sep '14

15 Sep '14

The new version is based on vcflib 386afcaafd. GitHub -> https://github.com/nekrut/vcflib-tools ToolShed -> https://toolshed.g2.bx.psu.edu/view/anton/suite_vcflib_tools_3_0 These will be on test and main instances shortly. Any feedback is appreciated. anton -- Anton Nekrutenko Professor of Biochemistry and Molecular Biology http://nekrut.bx.psu.edu (814) 826-3051

1 0

This looks like a Galaxy bug in installation of tools with a complex repository dependency.
by Melissa Cline 15 Sep '14

15 Sep '14

Hi Folks, I've done a bit more work on this problem that I posted about on Friday. I'm working on streamlining the installation of two tools, one of which has a complex repository dependency on the other. FWIW, both tools are in the test tool shed, under visualization: xena_import has a complex repository dependency on start_xena. I know that it should not be necessary to specify the toolshed and changeset revision in tool_dependencies.xml when indicating the complex repository dependency. But here is what I observed. When I specified the changeset_revision and toolshed, the installation worked fine. When I removed the changeset_revision tag and reinstalled the tool, I experienced the bug detailed below, where a Google Chrome pop-up window told me that the repository installation failed, and after I clicked OK on the pop-up window, Galaxy seemed to go into an infinite state of indicating that the tool was "Cloning" and showing the same repeated POST commands in paster.log. When I put the changeset_revision tag back again, the tool installation worked just fine. Just to make sure that this wasn't all from some cruft hiding somewhere in my Galaxy installation, I started with a clean install today (including an 'hg update stable') and a clean shed_tools subdirectory. I hope this helps. In the meantime, thanks, everyone for all of your help! Melissa On Fri, Sep 12, 2014 at 12:02 PM, Melissa Cline <cline(a)soe.ucsc.edu> wrote: > Thanks, folks! Everything you've said has clarified many things, and I'm > working on putting them into practice. But in the meantime, I've got a new > error condition, with the same repositories (and complex repository > dependencies), and could use some guidance. > > I started to see this error condition when I removed the toolshed and > changeset_revision tags in the tool_dependencies.xml for the dependent > repository xena_install. It now reads: > > <?xml version="1.0"?> > > <tool_dependency> > > <package name="installXena" version="1.0"> > > <repository name="start_xena" owner="melissacline"/> > > </package> > > </tool_dependency> > > Short and simple. But lately, when I try to install this repository from > the test toolshed, I get a pop-up window from Chrome (my web browser) that > says "The page at 127.0.0.1:8080 says:", "Initializing repository > installation failed", with an OK button. There is no error message visible > in paster.log. Here is what I do see: > > 127.0.0.1 - - [12/Sep/2014:11:45:57 -0700] "GET > /admin_toolshed/prepare_for_ins\ > > tall?tool_shed_url= > https://testtoolshed.g2.bx.psu.edu/&repository_ids=4ab1d3705\ > > ad0de2d&changeset_revisions=fe95cf1ae864 HTTP/1.1" 200 - "-" "Mozilla/5.0 > (Maci\ > > ntosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) > Chrome/37.\ > > 0.2062.94 Safari/537.36" > > tool_shed.galaxy_install.repository_dependencies.repository_dependency_manager > \ > > DEBUG 2014-09-12 11:46:05,937 Creating repository dependency objects... > > tool_shed.util.shed_util_common DEBUG 2014-09-12 11:46:07,214 Updating an > exist\ > > ing row for repository 'xena_import' in the tool_shed_repository table, > status \ > > set to 'New'. > > tool_shed.galaxy_install.repository_dependencies.repository_dependency_manager > \ > > DEBUG 2014-09-12 11:46:07,241 Skipping installation of revision > 75c7d80df9c1 of\ > > repository 'start_xena' because it was installed with the (possibly > updated) r\ > > evision 75c7d80df9c1 and its current installation status is 'Installed'. > > tool_shed.galaxy_install.repository_dependencies.repository_dependency_manager > \ > > DEBUG 2014-09-12 11:46:07,241 Building repository dependency > relationships... > > 127.0.0.1 - - [12/Sep/2014:11:46:05 -0700] "POST > /admin_toolshed/prepare_for_in\ > > stall HTTP/1.1" 200 - " > http://127.0.0.1:8080/admin_toolshed/prepare_for_install\ > > ?tool_shed_url= > https://testtoolshed.g2.bx.psu.edu/&repository_ids=4ab1d3705ad0d\ > > e2d&changeset_revisions=fe95cf1ae864" "Mozilla/5.0 (Macintosh; Intel Mac > OS X 1\ > > 0_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 > Safari/537.36\ > > " > > warning: testtoolshed.g2.bx.psu.edu certificate with fingerprint > 79:94:82:1f:74\ > > :e6:f8:5c:1c:b8:3e:8d:89:f0:0e:88:9a:ac:2b:61 not verified (check > hostfingerpri\ > > nts or web.cacerts config setting) > > 127.0.0.1 - - [12/Sep/2014:11:46:07 -0700] "POST > /admin_toolshed/manage_reposit\ > > ories HTTP/1.1" 500 - " > http://127.0.0.1:8080/admin_toolshed/prepare_for_install\ > > " "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 > (KHTML, li\ > > ke Gecko) Chrome/37.0.2062.94 Safari/537.36" > > Debug at: http://127.0.0.1:8080/_debug/view/1410547505 > > > Then when I click OK, the status of the tool changes to "Cloning", and > stays in that state indefinitely while paster.log keeps logging a repeat of > this message: > > 127.0.0.1 - - [12/Sep/2014:11:55:28 -0700] "POST > /admin_toolshed/repository_ins\ > > tallation_status_updates HTTP/1.1" 200 - " > http://127.0.0.1:8080/admin_toolshed/\ > > prepare_for_install" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) > AppleWebKi\ > > t/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 Safari/537.36" > > Just in case this is from some cruft from a previous installation, I tried > starting over with a brand new galaxy-dist directory. It doesn't look like > that changed anything. > > > Any ideas? Many thanks in advance! > > > Melissa > > > > > > On Fri, Sep 12, 2014 at 10:30 AM, Dave Bouvier <dave(a)bx.psu.edu> wrote: > >> Melissa, >> >> The revision recorded in the filesystem path can seem odd at first >> glance. My inline responses should hopefully clarify that. >> >> --Dave B. >> >> On 09/10/2014 05:49 PM, Melissa Cline wrote: >> >>> Hi Folks, >>> >>> Okay, I really need debugging ideas on this one. >>> >>> I have three repositories I'm developing under the test toolshed. >>> They're named start_xena, xena_import and xena_find_datasets (they're >>> all under visualization). start_xena contains a simple tool dependency >>> to a package named installXena. xena_import and xena_find_datasets >>> contain complex dependencies to installXena. I've installed these tools >>> on two computers. On one, everything works great - amazingly well, in >>> fact! Kudos, Development Team! On the other computer, the env.sh files >>> associated with the complex dependencies keep referencing the wrong >>> version of start_xena/installXena. >>> >>> Here are the gory details. >>> >>> The start_xena tool is installed in >>> testtoolshed.g2.bx.psu.edu/repos/melissacline/start_xena/ >>> 82755b0ee5a5/start_xena >>> <http://testtoolshed.g2.bx.psu.edu/repos/melissacline/ >>> start_xena/82755b0ee5a5/start_xena>. >>> I'm a bit confused about the revision part of the path, because the >>> latest revision is different (75c7d80df9c1), and I have uninstalled and >>> re-installed the tool since its last update. Its package, installXena, >>> is installed under tool_dependencies at >>> tool_dependencies/installXena/1.0/melissacline/start_xena/82755b0ee5a5/. >>> That directory contains all the files I'd expect to be there, and its >>> env.sh file contains all the environment variables I'd intended to set. >>> So far, so good. >>> >>> xena_import is installed on my computer at >>> testtoolshed.g2.bx.psu.edu/repos/melissacline/xena_ >>> import/dc42b6bbc22b/xena_import >>> <http://testtoolshed.g2.bx.psu.edu/repos/melissacline/ >>> xena_import/dc42b6bbc22b/xena_import>. >>> Its tool_dependencies.xml reference installXena under start_xena as: >>> >>> <?xml version="1.0"?> >>> >>> <tool_dependency> >>> >>> <package name="installXena" version="1.0"> >>> >>> <repository toolshed="http://testtoolshed.g2.bx.psu.edu" >>> name="start_xena" owner="melissacline" changeset_revision=" >>> 82755b0ee5a5"/> >>> >>> </package> >>> >>> </tool_dependency> >>> >>> (Aside: is it necessary to have the changeset_revision in there? I'm a >>> bit worried about the maintenance task of updating it for all of my >>> dependent tools every time I update start_xena, but as I'll explain, >>> it's not clear that this revision info is getting used). >>> >> >> It is not necessary to populate the changeset_revision or toolshed >> attributes. If you omit them, they will be automatically filled in based on >> the most recent matching repository. >> >> >>> The tool dependency dir for xena_import is at >>> tool_dependencies/installXena/1.0/melissacline/xena_import/dc42b6bbc22b/. >>> Its >>> env.sh reads: >>> >>> if [ -f >>> /Users/melissacline/src/galaxy-dist/tool_dependencies/ >>> installXena/1.0/melissacline/start_xena/0676a227dbc6/env.sh >>> ] ; then . >>> /Users/melissacline/src/galaxy-dist/tool_dependencies/ >>> installXena/1.0/melissacline/start_xena/0676a227dbc6/env.sh >>> ; fi >>> >>> Notice that the wrong revision number is in there. The 06 revision was >>> an older one. Needless to say, the files and environment variables I >>> need aren't in the 06 directory. >>> >>> Where does this incorrect revision number come from? I've deleted all >>> the 06 directories, repeatedly. I've deleted and reinstalled the tools, >>> and verified that after deletion, the directories in question were in >>> fact gone. I've tried resetting the metadata for these tools under the >>> Galaxy Admin menu. Now I need new ideas. As I mentioned, all of this >>> works beautifully on a second computer. It's as if the first Galaxy >>> installation, on the first computer, got stuck with some outdated >>> information, and I haven't figured out how to clear it out. >>> >> >> The 0676 revision number is based on the revision that was first >> installed, and should never change. When you update the repository, the >> database will record the new revision in a separate field, but the original >> revision id needs to be recorded in order to find the env.sh file on the >> filesystem. >> >> >> >>> Thanks! >>> >>> Melissa >>> >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >>> >

1 0