Nate, Thanks! We've taken your suggestion and implemented in in our instance. Thanks again, Dave On 10/25/10 9:15 AM, "Nate Coraor" <nate@bx.psu.edu> wrote:
Dave Walton wrote:
Thanks very much Nate.
I do have a question while I have you.
As recommended, we are trying to off-load all work from Galaxy that we can.
Uploads and downloads going through nginx, most jobs running on the cluster.
There is a catch however. We'd like the URL based uploads to get off-loaded to the cluster. However, for importing our NGS data, we are using the mechanism built in to used the shared libraries.
We are doing this because the NGS data generated by our Sequencing Core, is sitting on a share that is visible to our Galaxy server, but not visible to our cluster. While the directory where the galaxy data is stored *is* visible to our cluster. What this means is that to import this data, it has to be a "local" import, not run on the cluster.
Is there anyway to separate the shared library imports from "upload1", so that these can be local, but the URL based uploads can go through our cluster?
Hi Dave,
There's a way, but it's a hack. If you feel comfortable modifying the code, then here are the instructions:
1. Copy tools/data_source/upload.xml to something else like tools/data_source/upload_history.xml 2. In the original upload.xml, add the following tags somewhere inside the root <tool> element: <hidden>True</hidden> 3. In the new upload_history.xml, change the tool id to something else like "upload_history" 4. Add the upload_history.xml file to tool_conf.xml
Now you'll have two different upload tools with two different tool IDs that you can use to specify differing tool runners, but only one of them will appear in the tool menu.
Finally, to make sure the new upload tool isn't rerun in workflows, you'd need to modify the following method in lib/galaxy/tools/__init__.py:
def check_workflow_compatible( self ): ... # HACK: upload is (as always) a special case becuase file parameters # can't be persisted. if self.id == "upload1": return False
The test on the tool id should become:
if self.id in [ "upload1", "upload_history" ]:
I'm sorry there isn't a simpler way. In the future we'd like to have something that's less of a hack.
Thanks, --nate
Thanks,
Dave
On 10/21/10 3:26 PM, "Nate Coraor" <nate@bx.psu.edu> wrote:
Dave Walton wrote:
The problem has been solved.
Hooray!
I'm going to own-up to misunderstanding what "file uploads" the proxy server could off load.
I should have been clear in my first email wrt what kind of a file upload we were trying. We were doing an upload by URL.
Ah, that could be explained in the documentation. I'll try and clear this up.
If you have any other questions or issues please let us know.
Thanks, --nate
It turns out that the file upload from the user's desktop was always working (going through the proxy).
I've reconfigured our upload1 to go through our cluster and now everything is working great.
Thanks for your time.
Dave
On 10/21/10 2:32 PM, "Nate Coraor" <nate@bx.psu.edu> wrote:
Dave Walton wrote:
Cut and paste from a view source of my upload form:
<form id="tool_form" name="tool_form" action="/_upload?nginx_redir=/tool_runner/index" enctype="multipart/form-data" target="galaxy_main" method="post">
When I point my browser at /_upload on my galaxy server I get a: 405 Not allowed Nginx/0.8.53
Could you check the nginx error log for anything interesting?
Thanks, --nate
Dave
On 10/21/10 11:45 AM, "Nate Coraor" <nate@bx.psu.edu> wrote:
> Hi Dave, > > Your config looks good. Can you verify that the generated upload form's > action > is to '/_upload', e.g.: > > <form id="tool_form" name="tool_form" > action="/_upload?nginx_redir=/tool_runner/index" > enctype="multipart/form-data" > target="galaxy_main" method="post"> > > Can you access /_upload in your browser? > > --nate > > Dave Walton wrote: >> Good Morning, >> >> I have a galaxy instance at our institution, and we are trying to make >> the >> file upload with nginx work. >> >> It appears that when I do a file download, it is getting handled by >> nginx >> (running top I see an nginx process appear, do work and disappear when >> the >> file is done downloading). However, when I try to do an upload, I see >> additional python processes appear in top, and I see the upload.py >> script >> go >> by in my runner0.log file, but I don't see an nginx process appear in >> top. >> Also, when I go to run several large uploads in a row, and then attempt >> to >> run some other job (like a sort) it blocks while waiting for the files >> to >> upload. This leads me to believe the upload is running in galaxy, not >> nginx. >> >> I am also running "watch ls -ltr upload_store" from >> galaxy-dist/database/tmp >> while the upload is occurring and I never see anything written to the >> upload >> temp directory. >> >> We are running galaxy on a 4 cpu server, which is running SUSE Linux >> Enterprise Server 11 (x86_64) for an os. >> >> I have galaxy configured to run 3 webapps, and 1 job runner. >> >> We have nginx installed and built with the nginx_upload_module (detail >> below) >> >> jaxgalaxydev01:~ # /usr/local/nginx/sbin/nginx -V >> >> nginx version: nginx/0.8.53 >> built by gcc 4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux) >> configure arguments: --prefix=/usr/local/nginx-0.8.53 >> --add-module=/usr/local/src/nginx_upload_module-2.2.0 >> >> In both my universe_wsgi.webapp.ini and universe_wsgi.runner.ini I >> have >> the >> following lines defined in my [app:main] section: >> >> # Directive for using nginx to do file downloads >> nginx_x_accel_redirect_base = /_x_accel_redirect >> >> # Directives to allow nginx to deal with file uploads >> nginx_upload_store = database/tmp/upload_store >> nginx_upload_path = /_upload >> >> Below is the entire contents of my nginx.conf file: >> >> worker_processes 1; >> events { >> worker_connections 1024; >> } >> >> user galaxy; >> http { >> include mime.types; >> default_type application/octet-stream; >> >> # Added for compression and caching. According to Galaxy wiki >> page: >> # >> http://bitbucket.org/galaxy/galaxy-central/wiki/Config/nginxProxy >> # This will decrease downlod and page load times for clients >> gzip on; >> gzip_http_version 1.1; >> gzip_vary on; >> gzip_comp_level 4; >> gzip_proxied any; >> gzip_types text/plain text/css application/x-javascript text/xml >> application/xml text/javascript application/json; >> gzip_buffers 16 8k; >> gzip_disable "MSIE [1-6].(?!.*SV1)"; >> >> sendfile on; >> >> keepalive_timeout 65; >> >> >> # Added to support proxying to galaxy server >> upstream galaxy_app { >> server localhost:8080; >> server localhost:8081; >> server localhost:8082; >> } >> >> server { >> listen 80; >> server_name localhost; >> # Line added for galaxy support >> client_max_body_size 10G; >> >> >> # Added to allow nginx to handle file uploads >> location /_upload { >> upload_store >> /galaxydata/galaxy-setup/galaxy-dist/database/tmp/upload_store; >> upload_pass_form_field ""; >> upload_set_form_field >> "__${upload_field_name}__is_composite" >> "true"; >> upload_set_form_field "__${upload_field_name}__keys" "name >> path"; >> upload_set_form_field "${upload_field_name}_name" >> "$upload_file_name"; >> upload_set_form_field "${upload_field_name}_path" >> "$upload_tmp_path"; >> upload_pass_args on; >> upload_pass /_upload_done; >> } >> location /_upload_done { >> set $dst /tool_runner/index; >> if ($args ~ nginx_redir=([^&]+)) { >> set $dst $1; >> } >> rewrite "" $dst; >> } >> >> # added to allow nginx to handle file downloads >> location /_x_accel_redirect/ { >> internal; >> alias /; >> } >> >> # The following series of "locations" are for off-loading static >> # content from the galaxy server. >> location /static { >> alias /galaxydata/galaxy-setup/galaxy-dist/static; >> # This will decrease downlod and page load times for >> clients >> expires 24h; >> } >> location /static/style { >> alias >> /galaxydata/galaxy-setup/galaxy-dist/static/june_2007_style/blue; >> } >> location /static/scripts { >> alias >> /galaxydata/galaxy-setup/galaxy-dist/static/scripts/packed; >> # This will decrease downlod and page load times for >> clients >> expires 24h; >> } >> location /favicon.ico { >> alias >> /galaxydata/galaxy-setup/galaxy-dist/static/favicon.ico; >> } >> location /robots.txt { >> alias >> /galaxydata/galaxy-setup/galaxy-dist/static/robots.txt; >> } >> # End of static content >> >> location / { >> proxy_pass http://galaxy_app; >> proxy_set_header X-Forwarded-Host $host; >> proxy_set_header X-Forwarded-For >> $proxy_add_x_forwarded_for; >> } >> >> # redirect server error pages to the static page /50x.html >> # >> error_page 500 502 503 504 /50x.html; >> location = /50x.html { >> root html; >> } >> } >> } >> >> I have restarted both my galaxy server and my nginx server to confirm >> that >> the configurations above have been picked up. >> >> I would be interested in: >> 1) any other suggestions to confirm where the upload is occurring >> (assuming >> I'm mistaken about it running in galaxy). >> 2) any insight as to how I might have my server(s) misconfigured, in >> order >> to >> get upload working in nginx. >> >> Any assistance or suggestions you could provide would be greatly >> appreciated. >> >> Thanks, >> >> Dave > >> _______________________________________________ >> galaxy-dev mailing list >> galaxy-dev@lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-dev >