Thanks to Marius and others, I now have a working Galaxy server, including CVMFS support, as documented in the complete blog post: http://pvh.wp.sanbi.ac.za/2018/03/18/galaxy-18-01-install/

The problem I was encountering with installations was related to incorrect environment configuration, which led to the `hg` binary not being found in the PATH and thus the install failing. Another fault led to this failure being a silent failure.

I've linked to the working configuration in the blog, but here is a direct link:

https://gist.github.com/pvanheus/589f84507a26ffa49e3bdf3bd938f25c

It is rather too unwieldy to paste here.

Note that while the says [program:web], there is only one section in the supervisord configuration. The server is running in a uWSGI + Mules configuration.

Thanks!
Peter



On Mon, 19 Mar 2018 at 11:17 Peter van Heusden <pvh@sanbi.ac.za> wrote:
First to answer your questions:

I'm on release_18.01 branch, commit c58ca749c26bbb3c4ca8750f2c37fd1bb6bd7999. 

Thanks for the note about nginx-extras, I've installed that and it fixes the upload 
directives problem. 

Then on to the main problem. I used "Get Data" as the default location because I'm really just testing the install. The install goes from "New" status to "Error" status pretty much immediately:

image.png

This is not a bowtie2-specific problem, I get the same behaviour when trying to install fastqc.

Thanks for your reply,
Peter
P.S. running already-installed tools works

On Mon, 19 Mar 2018 at 09:44 Marius van den Beek <m.vandenbeek@gmail.com> wrote:
Hi Peter,

does the tool show up in the tool panel (Looks like you've installed it into "Get Data")?
What commit are you on ? There were some last minute bugfixes to the release_18.01
prior to the release, so if the checkout is a few days old an upgrade (and an uninstall/install cycle) might help.

Also you mentioned the lack of the upload_store in nginx in nginx, you need to install nginx-extras, not nginx.

Hope that helps,
Marius

On 18 March 2018 at 21:17, Peter van Heusden <pvh@sanbi.ac.za> wrote:
Hi there

My newly installed Galaxy 18.01 server seems to be unable to install bowtie (the devteam version). I'm documenting my installation progress at:


As noted in the last paragraph:

"My next test was to try and install the bowtie2 tool from the toolshed. This failed, with the tool status going from New to Error with not much interesting left in the logs." 

The log during the time when I'm trying to install bowtie2 is:

  1. [pid: 8856|app: 0|req: 35/48] 10.8.0.2 () {48 vars in 1108 bytes} [Sun Mar 18 19:14:55 2018] GET /admin_toolshed/manage_repository?id=f597429621d6eb2b&operation=install => generated 650 bytes in 26 msecs (HTTP/1.1 302) 3headers in 269 bytes (1 switches on core 1)
  2. 10.8.0.2 - - [18/Mar/2018:19:14:57 +0000] "GET /admin_toolshed/manage_repository?id=f597429621d6eb2b&operation=install HTTP/1.1" 302 - "https://galaxy.sanbi.ac.za/admin_toolshed/manage_repository?id=f597429621d6eb2b" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  3. [pid: 8856|app: 0|req: 37/49] 10.8.0.2 () {48 vars in 1108 bytes} [Sun Mar 18 19:14:57 2018] GET /admin_toolshed/manage_repository?id=f597429621d6eb2b&operation=install => generated 650 bytes in 24 msecs (HTTP/1.1 302) 3headers in 269 bytes (1 switches on core 0)
  4. galaxy.tools.deps DEBUG 2018-03-18 19:14:59,375 [p:8856,w:2,m:0] [uWSGIWorker2Core2] Dependency bowtie2 not found.
  5. galaxy.tools.deps DEBUG 2018-03-18 19:14:59,376 [p:8856,w:2,m:0] [uWSGIWorker2Core2] Dependency samtools not found.
  6. galaxy.tools.deps DEBUG 2018-03-18 19:15:00,028 [p:8856,w:2,m:0] [uWSGIWorker2Core3] Dependency bowtie2 not found.
  7. galaxy.tools.deps DEBUG 2018-03-18 19:15:00,028 [p:8856,w:2,m:0] [uWSGIWorker2Core3] Dependency samtools not found.
  8. 10.8.0.2 - - [18/Mar/2018:19:14:55 +0000] "GET /admin_toolshed/prepare_for_install?changeset_revisions=dc1639b66f12&repository_ids=126c0918b5459666&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/manage_repository?id=f597429621d6eb2b" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  9. [pid: 8856|app: 0|req: 38/50] 10.8.0.2 () {48 vars in 1274 bytes} [Sun Mar 18 19:14:55 2018] GET /admin_toolshed/prepare_for_install?changeset_revisions=dc1639b66f12&repository_ids=126c0918b5459666&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F => generated 19062 bytes in 7001 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 2)
  10. 10.8.0.2 - - [18/Mar/2018:19:14:57 +0000] "GET /admin_toolshed/prepare_for_install?changeset_revisions=dc1639b66f12&repository_ids=126c0918b5459666&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/manage_repository?id=f597429621d6eb2b" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  11. [pid: 8856|app: 0|req: 38/51] 10.8.0.2 () {48 vars in 1274 bytes} [Sun Mar 18 19:14:57 2018] GET /admin_toolshed/prepare_for_install?changeset_revisions=dc1639b66f12&repository_ids=126c0918b5459666&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F => generated 19062 bytes in 5987 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 3)
  12. galaxy.tools.toolbox.base DEBUG 2018-03-18 19:15:14,128 [p:8846,w:1,m:0] [uWSGIWorker1Core1] Appending to tool panel section: Get Data
  13. galaxy.tools.toolbox.base DEBUG 2018-03-18 19:15:14,143 [p:8846,w:1,m:0] [uWSGIWorker1Core1] Appending to tool panel section: Get Data
  14. 10.8.0.2 - - [18/Mar/2018:19:15:11 +0000] "POST /admin_toolshed/prepare_for_install HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/prepare_for_install?changeset_revisions=dc1639b66f12&repository_ids=126c0918b5459666&tool_shed_url=https%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146Safari/537.36"
  15. [pid: 8846|app: 0|req: 14/52] 10.8.0.2 () {56 vars in 1329 bytes} [Sun Mar 18 19:15:11 2018] POST /admin_toolshed/prepare_for_install => generated 11133 bytes in 2509 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
  16. galaxy.tools.toolbox.base DEBUG 2018-03-18 19:15:15,296 [p:8856,w:2,m:0] [uWSGIWorker2Core1] Appending to tool panel section: Get Data
  17. 10.8.0.2 - - [18/Mar/2018:19:15:15 +0000] "POST /admin_toolshed/install_repositories HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36(KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  18. [pid: 8856|app: 0|req: 39/53] 10.8.0.2 () {54 vars in 1152 bytes} [Sun Mar 18 19:15:15 2018] POST /admin_toolshed/install_repositories => generated 20 bytes in 206 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
  19. 10.8.0.2 - - [18/Mar/2018:19:15:18 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  20. [pid: 8856|app: 0|req: 40/54] 10.8.0.2 () {54 vars in 1208 bytes} [Sun Mar 18 19:15:18 2018] POST /admin_toolshed/repository_installation_status_updates => generated 174 bytes in 17 msecs (HTTP/1.1 200) 4 headers in 110bytes (1 switches on core 2)
  21. 10.8.0.2 - - [18/Mar/2018:19:15:21 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  22. [pid: 8856|app: 0|req: 41/55] 10.8.0.2 () {54 vars in 1208 bytes} [Sun Mar 18 19:15:21 2018] POST /admin_toolshed/repository_installation_status_updates => generated 2 bytes in 14 msecs (HTTP/1.1 200) 4 headers in 110 bytes (1 switches on core 3)
  23. 10.8.0.2 - - [18/Mar/2018:19:15:24 +0000] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "https://galaxy.sanbi.ac.za/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.146 Safari/537.36"
  24. [pid: 8846|app: 0|req: 15/56] 10.8.0.2 () {54 vars in 1208 bytes} [Sun Mar 18 19:15:24 2018] POST /admin_toolshed/repository_installation_status_updates => generated 2 bytes in 27 msecs (HTTP/1.1 200) 4 headers in 110 bytes (1 switches on core 3) 
which doesn't seem terribly informative.

Any ideas?
Thanks!
Peter

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/