Relaoding tools and tool menu in multiprocess setup
Hi; We have a multiprocess setup (job runner and multi web runners) and are frequently customising tools/adding new ones. When we were single threading at first, we could reload tools from the admin menu but that does not update the job runner version when multi thread. Is there any way to; - Reload tools without stopping galaxy when in multi thread - Update the tool menu This is critical because if you have a long running job, stopping the server will kill it - we have seq jobs that take days to complete and our setup is very fluent at the moment. I dont mind hacking if it works. Cheers DG
Dennis Gascoigne wrote:
Hi;
We have a multiprocess setup (job runner and multi web runners) and are frequently customising tools/adding new ones. When we were single threading at first, we could reload tools from the admin menu but that does not update the job runner version when multi thread. Is there any way to;
* Reload tools without stopping galaxy when in multi thread * Update the tool menu
Hi Dennis, Currently, the only way to do it is to load each application's admin interface and reload the tool separately. If your Galaxy servers are only listening on localhost this means initiating the connection from the same server. Since you're already proxying, you can set up individual hostnames that proxy to each application (including the job runner) so this can be done. The approach is tedious, but until we implement the API for administrative functions, there's no way around it.
This is critical because if you have a long running job, stopping the server will kill it - we have seq jobs that take days to complete and our setup is very fluent at the moment.
You can restart servers without interrupting jobs if you use PBS or SGE. --nate
I dont mind hacking if it works.
Cheers DG
------------------------------------------------------------------------
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Hello Dennis, Nate Coraor wrote, On 05/26/2010 09:06 AM:
Dennis Gascoigne wrote:
We have a multiprocess setup (job runner and multi web runners) and are frequently customising tools/adding new ones. When we were single threading at first, we could reload tools from the admin menu but that does not update the job runner version when multi thread. Is there any way to;
* Reload tools without stopping galaxy when in multi thread * Update the tool menu
I dont mind hacking if it works.
We use the following shell script to force reload on one of our tools. ports 8080/1/2 are the TCP ports for our multiple python processes. TOOLID is the tool identifier from the XML file. the "REMOTE_USER" header fakes an authenticated user (you don't need that if your galaxy doesn't require authentication). ================= #Reload the Tools to update the file lists TOOLID='cshl_import_local_files' # We add REMOTE_USER to simulate successful authentication for PORT in 8080 8081 8082; do STEP2=$(curl -d "tool_id=${TOOLID}" --header "REMOTE_USER: gordon" -s http://localhost:$PORT/admin/tool_reload | grep "Reloaded" ) if [ -z "$STEP2" ]; then echo "Error: failed to reload tool on port $PORT " >&2 fi done =============== It's ugly but it works. -gordon
participants (3)
-
Assaf Gordon -
Dennis Gascoigne -
Nate Coraor