- galaxy-dev - lists.galaxyproject.org

Test ToolShed misfiling missing tool tests
by Peter Cock 03 Mar '14

03 Mar '14

Hello all, Currently the Test Tool Shed seems not to be flagging all tools with missing tests. For example, the following currently lack tests yet are not listed under "Latest revision: missing tool tests" but instead under "Latest revision: all tool tests pass": http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/9fc8b8a639c9 http://testtoolshed.g2.bx.psu.edu/view/peterjc/mira4_assembler/4c6e4d583b8f Viewing these repositories they are shown to have tools without tests. Note that some tools are correctly listed under "Latest revision: missing tool tests", e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/clc_assembly_cell http://testtoolshed.g2.bx.psu.edu/view/peterjc/mira_assembler Regards, Peter

2 1

Re: [galaxy-dev] tool trouble
by Briand, Sheldon 28 Feb '14

28 Feb '14

Hi, So those are the ones that work. One of the ones that doesn't work is bowtie2: tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 16:03:14,476 Adding an entry for revision 017a00c265f1 of repository package_bowtie2_2_1_0 owned by devteam to repository_dependencies_of_installed_repositories. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 16:03:14,490 Adding an entry for revision 017a00c265f1 of reposito ry package_bowtie2_2_1_0 owned by devteam to installed_repository_dependencies_of_installed_repositories. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 16:03:14,502 Adding an entry for revision 017a00c265f1 of reposito ry package_bowtie2_2_1_0 owned by devteam to tool_dependencies_of_installed_repositories. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 16:03:14,516 Adding an entry for revision 017a00c265f1 of reposito ry package_bowtie2_2_1_0 owned by devteam to installed_tool_dependencies_of_installed_repositories. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 16:03:45,046 Adding an entry for version 2.1.0 of package bowtie2 to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. galaxy.tools.data DEBUG 2014-02-28 16:04:39,615 Loaded tool data table 'bowtie2_indexes' No tool ID loaded (loaded from the .xml file in the shed_tools dir?). And no file .xml fileor .py file in the shed_tools directory. But a successful install in the admin interface. I appreciate you looking into this for me. -Sheldon From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, February 28, 2014 5:51 PM To: Briand, Sheldon Subject: Re: [galaxy-dev] tool trouble Hi Sheldon, I'm not sure what to advise you to do here as all of the information looks ok. It's not clear at all why the tools are not displayed in your tool panel. Are they loaded but simply hidden (set to not be visible)? Did you make any changes to your integrated_tool_panel.xml file? If not, try deleting it an restartign your Galaxy server (the file gets automatically recreated if missing). I can't think of anything else to take a look at on this. Sorry I'm not able to be very helpful here. On Feb 28, 2014, at 4:24 PM, "Briand, Sheldon" <Sheldon.Briand(a)SSC-SPC.gc.ca<mailto:Sheldon.Briand@SSC-SPC.gc.ca>> wrote: Hi, Yes for deseq and sam2count: <section id="ngs:rnaseq" name="NGS:RNAseq" version=""> <tool file="toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e…<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…>" guid="too lshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/deseq/1.0.0<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/deseq/1.0.0>"> <tool_shed>toolshed.g2.bx.psu.edu<http://bx.psu.edu></tool_shed> <repository_name>deseq_and_sam2counts</repository_name> <repository_owner>nikhil-joshi</repository_owner> <installed_changeset_revision>a49aff09553e</installed_changeset_revision> <id>toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/deseq/1.0.0<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/deseq/1.0.0></id> <version>1.0.0</version> </tool> <tool file="toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e…<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…>" gu id="toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/sam2counts/1…<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/sam2counts/1.0.0>"> <tool_shed>toolshed.g2.bx.psu.edu<http://bx.psu.edu></tool_shed> <repository_name>deseq_and_sam2counts</repository_name> <repository_owner>nikhil-joshi</repository_owner> <installed_changeset_revision>a49aff09553e</installed_changeset_revision> <id>toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/sam2counts/1…<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/sam2counts/1.0.0></id> <version>1.0.0</version> </tool> </section> galaxy.tools DEBUG 2014-02-28 16:10:18,298 Loaded tool id: toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/deseq/1.0.0<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/deseq/1.0.0>, ve rsion: 1.0.0 into tool panel. galaxy.tools DEBUG 2014-02-28 16:10:18,310 Loaded tool id: toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/sam2counts/1…<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/sam2counts/1.0>. 0, version: 1.0.0 into tool panel. -Sheldon From: Greg Von Kuster [mailto:greg@bx.psu.edu<http://bx.psu.edu>] Sent: Friday, February 28, 2014 5:19 PM To: Briand, Sheldon Cc: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] tool trouble I assume you have a proper entry for the deseq_and_sam2count repository in your shed_tool_conf.xml file (please confirm). What does the relevant section of your Galaxy paster log show when attempts are made to load the tools contained in this repository at server startup? On Feb 28, 2014, at 4:02 PM, "Briand, Sheldon" <Sheldon.Briand(a)SSC-SPC.gc.ca<mailto:Sheldon.Briand@SSC-SPC.gc.ca>> wrote: Hi, Yep that's what I have. Here is deseq: find . -name "*deseq*" ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq/deseq._r.sample.i<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq/deseq.xml.i<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq/deseq<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq/deseq/deseq.xml.i<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq/deseq/deseq._r.i<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/.hg/store/data/deseq/deseq._r.i<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/deseq<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/deseq/deseq.R<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/deseq_and_sam2counts/deseq/deseq.xml<http://bx.psu.edu/repos/nikhil-joshi/deseq_and_sam2counts/a49aff09553e/dese…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/package_deseq2_1_0_17<http://bx.psu.edu/repos/iuc/package_deseq2_1_0_17> ./shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/package_deseq2_1_0_17/0e80b1827773/package_deseq2_1_0_17<http://bx.psu.edu/repos/iuc/package_deseq2_1_0_17/0e80b1827773/package_dese…> ./galaxy-dist/dependancies/R_3_0_1/3.0.1/iuc/package_deseq2_1_0_17 ./galaxy-dist/dependancies/deseq2 ./galaxy-dist/dependancies/deseq2/1.0.17/iuc/package_deseq2_1_0_17 ./galaxy-dist/dependancies/deseq2/1.0.17/iuc/package_deseq2_1_0_17/0e80b1827773/deseq2 -Sheldon From: Greg Von Kuster [mailto:greg@bx.psu.edu<http://bx.psu.edu>] Sent: Friday, February 28, 2014 4:59 PM To: Briand, Sheldon Cc: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] tool trouble Hello Sheldon, What is the configuration setting you are using for the location to install repositories? This is defined by the tool_path attribute of the <toolbox> tag in your shed_tools_conf.xml file. The default is: <toolbox tool_path="../shed_tools"> On Feb 28, 2014, at 3:47 PM, "Briand, Sheldon" <Sheldon.Briand(a)SSC-SPC.gc.ca<mailto:Sheldon.Briand@SSC-SPC.gc.ca>> wrote: Hi, No joy on the migration strategy. Deseq is example of a tool that I installed successfully yesterday. Do you mean that you installed the deseq repository from the main tool shed: http://toolshed.g2.bx.psu.edu/view/nikhil-joshi/deseq_and_sam2counts and that its contained deseq tool "DE Seq (version 1.0.0)" was properly displayed in your Galaxy tool panel? If so, what is now happening? Bowtie2 is an example of a tool that isn't showing up even though the web interface indicates that it is successfully installed. Can you uninstall the repsository and reinstall it? If so, what happens? Here is my directory structure relating to bowtie2: $ find . -name "*bowtie2*" ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2<http://bx.psu.edu/repos/devteam/bowtie2> ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/package_bowtie2_2_1_0<http://bx.psu.edu/repos/devteam/package_bowtie2_2_1_0> ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/package_bowtie2_2_1_0/017a00c265f1/package_bowtie2_2_1_0<http://bx.psu.edu/repos/devteam/package_bowtie2_2_1_0/017a00c265f1/package_…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/tophat2/.hg/store/data/tool-data/bowtie2__indices.loc.sample.i<http://bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/tophat2/.hg/store/data…> ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/tophat2/tool-data/bowtie2_indices.loc.sample<http://bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/tophat2/tool-data/bowt…> ./galaxy-dist/.hg/store/data/tool-data/bowtie2__indices.loc.sample.i ./galaxy-dist/.hg/store/data/tools/sr__mapping/bowtie2__wrapper.xml.i ./galaxy-dist/.hg/store/data/tools/sr__mapping/bowtie2__wrapper.py.i ./galaxy-dist/.hg/store/data/test-data/bowtie2 ./galaxy-dist/tool-data/bowtie2_indices.loc.sample ./galaxy-dist/tool-data/bowtie2_indices.loc ./galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/bowtie2_indices.loc<http://bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/bowtie2_indices.loc> ./galaxy-dist/dependancies/samtools/0.1.18/devteam/bowtie2 ./galaxy-dist/dependancies/samtools/0.1.19/iuc/package_samtools_0_1_19/00e17a794a2e/bin/bowtie2sam.pl ./galaxy-dist/dependancies/bowtie2 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/bowtie2 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-align ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-build-debug ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-inspect-debug ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-inspect ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-align-debug ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-build [galaxy@eugene galaxy]$ cd shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/<http://bx.psu.edu/repos/devteam/bowtie2/> I seem to be missing the .py and .xml files in the shed_tools directory. -Sheldon From: Briand, Sheldon Sent: Friday, February 28, 2014 2:11 PM To: 'Greg Von Kuster' Subject: RE: [galaxy-dev] tool trouble Hi, Thanks for getting back to me. I didn't get your first reply (email trouble here) when I sent my second email. I have not played with the sql tables. I think my problem is that I need to run tool migration scripts. The reinstall log I sent was for tophat 2. It didn't show up in shed_tool_conf.xml after the update so I thought it had once again failed. After I sent the email I noticed that tophat2 had shown up in the tool menu (not where I wanted it but it was there). So I looked around and there was an entry in migrated_tools_conf.xml. I think the tools I am missing are because I likely missed the migration step. I'll try that first and report back. Cheers, -Sheldon From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, February 28, 2014 11:04 AM To: Briand, Sheldon Cc: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] tool trouble Hello Briand, This may take some back-and-forth with you to understand the steps you have taken so far. Successfully installing a repository that contains tools usually results in the tools being displayed in the Galaxy tool panel. However, there may be issues that result in the tools not properly loading. This often occurs when you stop and restart your Galaxy instance. Your Galaxy pasxter log will include information about any tool that does not properly load. Uninstalling and reinstalling the repository will usually not have any affect on this. On Feb 28, 2014, at 9:17 AM, "Briand, Sheldon" <Sheldon.Briand(a)ssc-spc.gc.ca<mailto:Sheldon.Briand@ssc-spc.gc.ca>> wrote: Hi, I've been poking around a bit more. It looks as though galaxy isn't actually downloading and installing the tools. The tool status changes to installed in the web interface and postgres but the tool is not present on disk. What is present on disk? If you install a repository with tools, the repository revision's contents should be on disk. However, if the repository is uninstalled, disk files are deleted. Do you have repositories that are installed but do not have any files on disk? The log below shows no download and make of the tool and dependancies. Hopefully the log gives a bit more information. Sorry for the second email. tool_shed.util.shed_util_common DEBUG 2014-02-28 09:58:48,567 Updating an existing row for repository 'tophat2' in the tool_shed_repository table, status set to 'New'. tool_shed.util.repository_dependency_util DEBUG 2014-02-28 09:58:48,714 Creating repository dependency objects... The log looks ok at first inspection, but it's not useful without more information. From: galaxy-dev-bounces(a)lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu> [mailto:galaxy-dev-bounces@lists.bx.psu.edu<mailto:dev-bounces@lists.bx.psu.edu>] On Behalf Of Briand, Sheldon Sent: Thursday, February 27, 2014 8:18 PM To: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: [galaxy-dev] tool trouble Hi, I'm having trouble with some of my tools not appearing in the Analyze Data menu. I've successfully installed the tools via the manage installed tool shed repositories(so they are installed to disk). I have tried uninstalling and installing again. The above should not be necessary. When you install, are you checking the optional checkboxes to install repository dependencies and tool dependencies? If so, are they installing as well? The tool installs but does not appear in the menu. Do you have an example of a repository that contains tools that installls corruptly, but the tools do not load into the tool panel? Other tools install the menu updates just fine. Do you have an example of a repsoitory that contains tools that do show up in the tool panel after successfule repository installation? This is probably due to me corrupting the misbehaving tools installation somehow. I'm guessing that I need to uninstall the tool and wipe some postgres entries. I strongly advise not manually changing inaything in your database suing sql commands. If you have done this, it could be the cause of the behavor you are describing. The Galaxy UI or API should be used. Greg Von Kuster My shed tool path: <toolbox tool_path="../shed_tools"> My current version: parent: 12441:dc067a95261d Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable Could someone point me in the right direction? Thanks, -Sheldon Sheldon Briand NRC Research Computing Support Analyst Research Computing Support / Soutien Informartique a la Recherche Operations, Science Portfolio / Operations, Portefeuil des sciences SSC-NRC / SPC-CNRC Rm 329A, 1411 Oxford Street / Piece 329A, 1411 Rue Oxford Halifax, NS B3H 3Z1 902 426-1677 sheldon.briand(a)ssc-spc.gc.ca<mailto:sheldon.briand@ssc-spc.gc.ca> ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

1 1

Re: [galaxy-dev] tool trouble
by Briand, Sheldon 28 Feb '14

28 Feb '14

Hi, No joy on the migration strategy. Deseq is example of a tool that I installed successfully yesterday. Bowtie2 is an example of a tool that isn't showing up even though the web interface indicates that it is successfully installed. Here is my directory structure relating to bowtie2: $ find . -name "*bowtie2*" ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2 ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/package_bowtie2_2_1_0 ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/package_bowtie2_2_1_0/017a00c265f1/package_bowtie2_2_1_0 ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/tophat2/.hg/store/data/tool-data/bowtie2__indices.loc.sample.i ./shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/tophat2/tool-data/bowtie2_indices.loc.sample ./galaxy-dist/.hg/store/data/tool-data/bowtie2__indices.loc.sample.i ./galaxy-dist/.hg/store/data/tools/sr__mapping/bowtie2__wrapper.xml.i ./galaxy-dist/.hg/store/data/tools/sr__mapping/bowtie2__wrapper.py.i ./galaxy-dist/.hg/store/data/test-data/bowtie2 ./galaxy-dist/tool-data/bowtie2_indices.loc.sample ./galaxy-dist/tool-data/bowtie2_indices.loc ./galaxy-dist/tool-data/toolshed.g2.bx.psu.edu/repos/devteam/tophat2/ffa30bedbee3/bowtie2_indices.loc ./galaxy-dist/dependancies/samtools/0.1.18/devteam/bowtie2 ./galaxy-dist/dependancies/samtools/0.1.19/iuc/package_samtools_0_1_19/00e17a794a2e/bin/bowtie2sam.pl ./galaxy-dist/dependancies/bowtie2 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/bowtie2 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-align ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-build-debug ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-inspect-debug ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2 ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-inspect ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-align-debug ./galaxy-dist/dependancies/bowtie2/2.1.0/devteam/package_bowtie2_2_1_0/017a00c265f1/bowtie2-build [galaxy@eugene galaxy]$ cd shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/ I seem to be missing the .py and .xml files in the shed_tools directory. -Sheldon From: Briand, Sheldon Sent: Friday, February 28, 2014 2:11 PM To: 'Greg Von Kuster' Subject: RE: [galaxy-dev] tool trouble Hi, Thanks for getting back to me. I didn't get your first reply (email trouble here) when I sent my second email. I have not played with the sql tables. I think my problem is that I need to run tool migration scripts. The reinstall log I sent was for tophat 2. It didn't show up in shed_tool_conf.xml after the update so I thought it had once again failed. After I sent the email I noticed that tophat2 had shown up in the tool menu (not where I wanted it but it was there). So I looked around and there was an entry in migrated_tools_conf.xml. I think the tools I am missing are because I likely missed the migration step. I'll try that first and report back. Cheers, -Sheldon From: Greg Von Kuster [mailto:greg@bx.psu.edu] Sent: Friday, February 28, 2014 11:04 AM To: Briand, Sheldon Cc: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] tool trouble Hello Briand, This may take some back-and-forth with you to understand the steps you have taken so far. Successfully installing a repository that contains tools usually results in the tools being displayed in the Galaxy tool panel. However, there may be issues that result in the tools not properly loading. This often occurs when you stop and restart your Galaxy instance. Your Galaxy pasxter log will include information about any tool that does not properly load. Uninstalling and reinstalling the repository will usually not have any affect on this. On Feb 28, 2014, at 9:17 AM, "Briand, Sheldon" <Sheldon.Briand(a)ssc-spc.gc.ca<mailto:Sheldon.Briand@ssc-spc.gc.ca>> wrote: Hi, I've been poking around a bit more. It looks as though galaxy isn't actually downloading and installing the tools. The tool status changes to installed in the web interface and postgres but the tool is not present on disk. What is present on disk? If you install a repository with tools, the repository revision's contents should be on disk. However, if the repository is uninstalled, disk files are deleted. Do you have repositories that are installed but do not have any files on disk? The log below shows no download and make of the tool and dependancies. Hopefully the log gives a bit more information. Sorry for the second email. tool_shed.util.shed_util_common DEBUG 2014-02-28 09:58:48,567 Updating an existing row for repository 'tophat2' in the tool_shed_repository table, status set to 'New'. tool_shed.util.repository_dependency_util DEBUG 2014-02-28 09:58:48,714 Creating repository dependency objects... The log looks ok at first inspection, but it's not useful without more information. From: galaxy-dev-bounces(a)lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu> [mailto:galaxy-dev-bounces@lists.bx.psu.edu<mailto:dev-bounces@lists.bx.psu.edu>] On Behalf Of Briand, Sheldon Sent: Thursday, February 27, 2014 8:18 PM To: galaxy-dev(a)lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: [galaxy-dev] tool trouble Hi, I'm having trouble with some of my tools not appearing in the Analyze Data menu. I've successfully installed the tools via the manage installed tool shed repositories(so they are installed to disk). I have tried uninstalling and installing again. The above should not be necessary. When you install, are you checking the optional checkboxes to install repository dependencies and tool dependencies? If so, are they installing as well? The tool installs but does not appear in the menu. Do you have an example of a repository that contains tools that installls corruptly, but the tools do not load into the tool panel? Other tools install the menu updates just fine. Do you have an example of a repsoitory that contains tools that do show up in the tool panel after successfule repository installation? This is probably due to me corrupting the misbehaving tools installation somehow. I'm guessing that I need to uninstall the tool and wipe some postgres entries. I strongly advise not manually changing inaything in your database suing sql commands. If you have done this, it could be the cause of the behavor you are describing. The Galaxy UI or API should be used. Greg Von Kuster My shed tool path: <toolbox tool_path="../shed_tools"> My current version: parent: 12441:dc067a95261d Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable Could someone point me in the right direction? Thanks, -Sheldon Sheldon Briand NRC Research Computing Support Analyst Research Computing Support / Soutien Informartique a la Recherche Operations, Science Portfolio / Operations, Portefeuil des sciences SSC-NRC / SPC-CNRC Rm 329A, 1411 Oxford Street / Piece 329A, 1411 Rue Oxford Halifax, NS B3H 3Z1 902 426-1677 sheldon.briand(a)ssc-spc.gc.ca<mailto:sheldon.briand@ssc-spc.gc.ca> ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

2 3

March 2014 Galaxy Update Newsletter
by Dave Clements 28 Feb '14

28 Feb '14

1 0

Galaxy command line API information
by Benoit Meilhac 28 Feb '14

28 Feb '14

Dear Sir or Madam, I am working for the University of Westminster on the ER-Flow project (http://www.erflow.eu/home) I'm currently investigating a way to use the galaxy workflow engine with a command line interface in order to support it within our project. The idea would be to develop a plugin that could make a link between our repository, where a galaxy workflow can be specified (workflow and inputs stored and output declared), and a galaxy server. We have a specific way our existing plugins work. It would be something like: ./executable -w workflow <list of inputs> And then, outputs are retrieved and returned (they don't need to be given as command line parameters). I had a quick look at the provided API within the galaxy project and I have some questions. I saw this is possible to import the workflow to the server, then to execute it. In my plugin, I would have to import the workflow, then run it with inputs listed in the executable command line, and after a successful execution, I would have to retrieve the outputs generated, then delete the created workflow and any files related that exist. Is it currently possible for me, in a short time, to develop such plugin performing operations previously described using the existing API? If yes, could you please provide me more information on the current API or some help on the different tools I should use? Many thanks and kind regards, Benoît Meilhac The University of Westminster is a charity and a company limited by guarantee. Registration number: 977818 England. Registered Office: 309 Regent Street, London W1B 2UW. This message and its attachments are private and confidential. If you have received this message in error, please notify the sender and remove it and its attachments from your system.

3 2

Markup safe egg problem/fetching the wrong eggs?
by Joseph Ward 28 Feb '14

28 Feb '14

Hi, I've got a strange problem with python paths/eggs. When I run any tool that produces html/text output I get the error: WARNING:galaxy.eggs:Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched. So if I close galaxy, delete the eggs folder, run fetch_eggs.py (on the cluster, sourcing the same files as galaxy does so the python path should be identical) it gathers all the eggs it needs for python 2.7. Running a job then ends with the proper results, no errors, but not with python 2.6 eggs present in the eggs folder alongside the 2.7 eggs. Re-running the same job, or running any other that produces html output, not throws up the error mentioned above. I can't see anywhere where I've missed a python path or anywhere where python 2.6 is mentioned. Any idea what I'm missing here? Cheers, Joe

2 2

tool trouble
by Briand, Sheldon 28 Feb '14

28 Feb '14

Hi, I'm having trouble with some of my tools not appearing in the Analyze Data menu. I've successfully installed the tools via the manage installed tool shed repositories(so they are installed to disk). I have tried uninstalling and installing again. The tool installs but does not appear in the menu. Other tools install the menu updates just fine. This is probably due to me corrupting the misbehaving tools installation somehow. I'm guessing that I need to uninstall the tool and wipe some postgres entries. My shed tool path: <toolbox tool_path="../shed_tools"> My current version: parent: 12441:dc067a95261d Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable Could someone point me in the right direction? Thanks, -Sheldon Sheldon Briand NRC Research Computing Support Analyst Research Computing Support / Soutien Informartique a la Recherche Operations, Science Portfolio / Operations, Portefeuil des sciences SSC-NRC / SPC-CNRC Rm 329A, 1411 Oxford Street / Piece 329A, 1411 Rue Oxford Halifax, NS B3H 3Z1 902 426-1677 sheldon.briand(a)ssc-spc.gc.ca

2 2

Torque.py Bug
by ngsflow＠hygenomics.com 28 Feb '14

28 Feb '14

Hi, base: os:centos6.4 torque:4.2.7 I built a glaxy cluster today,and then I found the job completion status of torque queue is 'c',but the need state of galalxy is "ok".So I modified the torque.py file and solved this error. I do, right? diff -r 8411a9f30feb lib/galaxy/jobs/runners/cli_job/torque.py --- a/lib/galaxy/jobs/runners/cli_job/torque.py Thu Dec 19 14:38:03 2013 -0500 +++ b/lib/galaxy/jobs/runners/cli_job/torque.py Fri Feb 28 13:02:48 2014 +0800 @@ -117,6 +117,8 @@ if id in job_ids: state = job.find('job_state').text # map PBS job states to Galaxy job states. + if state == u'C' : + state = 'ok' rval[id] = self.__get_job_state(state) return rval ngsflow(a)hygenomics.com

2 1

Question on storage configuration for Galaxy datasets
by Lukasse, Pieter 28 Feb '14

28 Feb '14

Hi , We would the location of the datasets our Galaxy stores to be determined dynamically based on a set of rules. For example: for a specific role we would like a datasets to be stored in a folder /storage/no_backup and for another role this should be /storage/with_backup Is there currently a configuration file in Galaxy to set this up? Or perhaps any alternative solutions? Thanks and regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317481122; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

2 2

universe_wsgi.ini question for Cistrome
by ruiwang.sz 28 Feb '14

28 Feb '14

Hi Guys, Are there anyone who is using Cistrome? I tried to merge Cistrome's setting into our own galaxy instance. A new entry is # Path to the static library files for assembly, ceaslib, chromLen, conservation, liftOver and MAT-lib # Then in other tool configuration xml file, we can use 'from galaxy import config' then '$config.Configuration().cistrome_static_library_path' # default would be in tool-data/ folder cistrome_static_library_path = /home/bioinfoadmin/app/cistrome-apps/cistrome_library However, I got error when I tried CEAS: Enrichment on chromosome and annotation: ***************************************************************************** Dataset 23: CEAS: Enrichment on chromosome and annotation on data 16 and data 13 The Galaxy framework encountered the following error while attempting to run the tool: Traceback (most recent call last): File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 121, in prepare_job job_wrapper.prepare() File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/jobs/__init__.py", line 707, in prepare config_filenames = self.tool.build_config_files( param_dict, self.working_directory ) File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 2609, in build_config_files f.write( fill_template( template_text, context=param_dict ) ) File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/home/bioinfoadmin/app/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1391037616_15_43258.py", line 274, in respond NotFound: cannot find 'cistrome_static_library_path' while searching for '__app__.config.cistrome_static_library_path' Tool execution generated the following error message: failure preparing job ******************************************************************************** I'm quite confused since '__app__.config.cistrome_static_library_path' should be defined? or right now __app__ is dropped as Galaxy evolves? Besides, what does the comment mean by # Then in other tool configuration xml file, we can use 'from galaxy import config' then '$config.Configuration().cistrome_static_library_path' Does that mean that we should not be using __app__.config...but how to use this? I tried to insert 'from galaxy import config' into the xml file but only got error... I'll appreciate any input. Thanks, Rui

2 1