download dataset from the proxy-ed Galaxy failed with broken link
Hi guys, On our Galaxy is proxy-ed via Apache according to the wiki guide, the normal download dataset via browser is working fine. However, when I tried download from command line via wget or curl -O (I copied the dataset URL by right clicking the floppy disk icon). And here is the wget output: pwbcad@gamma01:tmp$ wget http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... Resolving galaxy.demo.edu.au... 1xx.x4.1xx.xx5 Connecting to galaxy.demo.edu.au|1xx.x4.1xx.xx5|:80... connected. HTTP request sent, awaiting response... 302 Found Location: /galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None [following] --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None Connecting to galaxy.demo.edu.au|129.94.136.205|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: "root?dataset_id=ccc5aff28d2c8616&filename=None.1" [ <=> ] 28,476 --.-K/s in 0.001s 2012-10-25 10:57:42 (31.4 MB/s) - "root?dataset_id=ccc5aff28d2c8616&filename=None.1" saved [28476] It seems like the Apache returned another URL (returned 302, probably due to proxy) and the new URL is broken. Thanks in advance. Cheers, Derrick
On Oct 24, 2012, at 11:15 PM, Derrick Lin wrote:
Hi guys,
On our Galaxy is proxy-ed via Apache according to the wiki guide, the normal download dataset via browser is working fine.
However, when I tried download from command line via wget or curl -O (I copied the dataset URL by right clicking the floppy disk icon).
And here is the wget output:
pwbcad@gamma01:tmp$ wget http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... Resolving galaxy.demo.edu.au... 1xx.x4.1xx.xx5 Connecting to galaxy.demo.edu.au|1xx.x4.1xx.xx5|:80... connected. HTTP request sent, awaiting response... 302 Found Location: /galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None [following] --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None Connecting to galaxy.demo.edu.au|129.94.136.205|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: "root?dataset_id=ccc5aff28d2c8616&filename=None.1"
[ <=> ] 28,476 --.-K/s in 0.001s
2012-10-25 10:57:42 (31.4 MB/s) - "root?dataset_id=ccc5aff28d2c8616&filename=None.1" saved [28476]
It seems like the Apache returned another URL (returned 302, probably due to proxy) and the new URL is broken.
Hi Derrick, What are the contents of the file that wget saved? This should give you a clue as to what's going on. --nate
Thanks in advance.
Cheers, Derrick ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Nate, I checked and what downloaded is a webpage, I opened it on browser, and screenshot attached. Basically it displays error saying File not found. But dataset download works fine via browser. Cheers, Derrick On Fri, Oct 26, 2012 at 11:44 PM, Nate Coraor <nate@bx.psu.edu> wrote:
On Oct 24, 2012, at 11:15 PM, Derrick Lin wrote:
Hi guys,
On our Galaxy is proxy-ed via Apache according to the wiki guide, the normal download dataset via browser is working fine.
However, when I tried download from command line via wget or curl -O (I copied the dataset URL by right clicking the floppy disk icon).
And here is the wget output:
pwbcad@gamma01:tmp$ wget http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... Resolving galaxy.demo.edu.au... 1xx.x4.1xx.xx5 Connecting to galaxy.demo.edu.au|1xx.x4.1xx.xx5|:80... connected. HTTP request sent, awaiting response... 302 Found Location: /galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None [following] --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None Connecting to galaxy.demo.edu.au|129.94.136.205|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: "root?dataset_id=ccc5aff28d2c8616&filename=None.1"
[ <=>
] 28,476 --.-K/s in 0.001s
2012-10-25 10:57:42 (31.4 MB/s) -
"root?dataset_id=ccc5aff28d2c8616&filename=None.1" saved [28476]
It seems like the Apache returned another URL (returned 302, probably
due to proxy) and the new URL is broken.
Hi Derrick,
What are the contents of the file that wget saved? This should give you a clue as to what's going on.
--nate
Thanks in advance.
Cheers, Derrick ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
If I downloaded with curl -i, the page it returned was: pwbcad@gamma01:tmp$ curl -i http://galaxy-qa.garvan.unsw.edu.au/datasets/dc097fa4e7ad53f6/display?to_ext... HTTP/1.1 302 Found Date: Tue, 30 Oct 2012 04:25:21 GMT Server: PasteWSGIServer/0.5 Python/2.6.6 content-type: text/html location: /root?dataset_id=dc097fa4e7ad53f6&filename=None Set-Cookie: galaxysession=c6ca0ddb55be603a4107d5e4793b9ce28e93ce1e646977c209d1af46a242147b57953de37776fcc2; expires=Mon, 28-Jan-2013 15:25:21 GMT; httponly; Max-Age=7776000; Path=/; Version=1 Vary: Accept-Encoding Transfer-Encoding: chunked <html> <head><title>Found</title></head> <body> <h1>Found</h1> <p>The resource was found at <a href="/root?dataset_id=dc097fa4e7ad53f6&filename=None">/root?dataset_id=dc097fa4e7ad53f6&filename=None</a>; you should be redirected automatically. <!-- --></p> <hr noshade> <div align="right">WSGI Server</div> </body> </html> Don't know why the apache returned 302 only when access via wget or curl. Derrick On Sat, Oct 27, 2012 at 7:44 AM, Derrick Lin <klin938@gmail.com> wrote:
Hi Nate,
I checked and what downloaded is a webpage, I opened it on browser, and screenshot attached.
Basically it displays error saying File not found. But dataset download works fine via browser.
Cheers, Derrick
On Fri, Oct 26, 2012 at 11:44 PM, Nate Coraor <nate@bx.psu.edu> wrote:
On Oct 24, 2012, at 11:15 PM, Derrick Lin wrote:
Hi guys,
On our Galaxy is proxy-ed via Apache according to the wiki guide, the normal download dataset via browser is working fine.
However, when I tried download from command line via wget or curl -O (I copied the dataset URL by right clicking the floppy disk icon).
And here is the wget output:
pwbcad@gamma01:tmp$ wget http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=fa... Resolving galaxy.demo.edu.au... 1xx.x4.1xx.xx5 Connecting to galaxy.demo.edu.au|1xx.x4.1xx.xx5|:80... connected. HTTP request sent, awaiting response... 302 Found Location: /galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None [following] --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None Connecting to galaxy.demo.edu.au|129.94.136.205|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: "root?dataset_id=ccc5aff28d2c8616&filename=None.1"
[ <=>
] 28,476 --.-K/s in 0.001s
2012-10-25 10:57:42 (31.4 MB/s) -
"root?dataset_id=ccc5aff28d2c8616&filename=None.1" saved [28476]
It seems like the Apache returned another URL (returned 302, probably
due to proxy) and the new URL is broken.
Hi Derrick,
What are the contents of the file that wget saved? This should give you a clue as to what's going on.
--nate
Thanks in advance.
Cheers, Derrick ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Derrick Lin -
Nate Coraor