How to download data from History
Hello all, I have two related questions about getting data from the Galaxy History. I installed `galaxy in my server, I did some data analyses and now i want the resulting data back. But I am facing two problems: 1/ I just try to use the download button. If i do it with the button (or with curl or wget), i get the following error: <h1>Proxy Error</h1> <p>The proxy server received an invalid response from an upstream server.<br /> The proxy server could not handle the request <em><a href="/galaxy/datasets/b701da857886499b/display">GET /galaxy/datasets/b701da857886499b/display</a></em>.<p> Reason: <strong>Error reading from remote server</strong></p></p> </body></html> any ideas? Does I am suppose to modify the universe_wsgi.ini? 2/ In fact, i would prefer to just copy the result on the same server (no need to use the network). Something similar that what is used to add data to the datalibrary. Is it a way to just copy data from within galaxy to some given place in the same server? thank you colin -- Colin Molter University of Brussels - InSilico Team - http://insilico.ulb.ac.be/
Hello Colin, For #1, we have some wiki help pages that go through the set-up steps for this type of proxy configuration. Please see: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServer -- scroll to "Make the proxy handle uploads and downloads" which links to these two docs: Apache: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy -- the section "Sending files using Apache" specifically addresses downloads. nginx: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/nginxProxy -- also has a section for this function, titled "Sending files using nginx" For #2, a unix copy of the files could be done directly or by an external script, but it sounds as if you want to do this configured from within Galaxy itself. This could likely be done by setting the download destination for the UI "save" function to be a static location in the configuration. Let's see if any of our dev team community has done this already and would like to share the method. Hopefully this helps, Best, Jen Galaxy team On 6/30/11 7:48 AM, colin molter wrote:
Hello all, I have two related questions about getting data from the Galaxy History. I installed `galaxy in my server, I did some data analyses and now i want the resulting data back.
But I am facing two problems:
1/ I just try to use the download button. If i do it with the button (or with curl or wget), i get the following error: <h1>Proxy Error</h1> <p>The proxy server received an invalid response from an upstream server.<br /> The proxy server could not handle the request <em><a href="/galaxy/datasets/b701da857886499b/display">GET /galaxy/datasets/b701da857886499b/display</a></em>.<p> Reason: <strong>Error reading from remote server</strong></p></p> </body></html>
any ideas? Does I am suppose to modify the universe_wsgi.ini?
2/ In fact, i would prefer to just copy the result on the same server (no need to use the network). Something similar that what is used to add data to the datalibrary. Is it a way to just copy data from within galaxy to some given place in the same server?
thank you colin -- Colin Molter University of Brussels - InSilico Team - http://insilico.ulb.ac.be/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org/ http://galaxyproject.org/
Hi, thx for the reply. I will try to make a proper install of galaxy as soon as i have some time. But as for now, more importantly, i just need to make a unix copy of the files that interest me. so yes, having something from within galaxy could be cool. But a unix copy could make the trick for my first tests. Still I have a problem. I noticed that all my history is located in database/files/ However, how can i know which file corresponds to a given file in my history? Of course, i could work with the file size and the date of creation, but a more reliable and intuitive link would be nice. Could I find it somewhere? thx colin 2011/7/1 Jennifer Jackson <jen@bx.psu.edu>
Hello Colin,
For #1, we have some wiki help pages that go through the set-up steps for this type of proxy configuration. Please see:
https://bitbucket.org/galaxy/**galaxy-central/wiki/Config/** ProductionServer<https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServer> -- scroll to "Make the proxy handle uploads and downloads" which links to these two docs:
Apache: https://bitbucket.org/galaxy/**galaxy-central/wiki/Config/**ApacheProxy<https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy> -- the section "Sending files using Apache" specifically addresses downloads.
nginx: https://bitbucket.org/galaxy/**galaxy-central/wiki/Config/**nginxProxy<https://bitbucket.org/galaxy/galaxy-central/wiki/Config/nginxProxy> -- also has a section for this function, titled "Sending files using nginx"
For #2, a unix copy of the files could be done directly or by an external script, but it sounds as if you want to do this configured from within Galaxy itself. This could likely be done by setting the download destination for the UI "save" function to be a static location in the configuration. Let's see if any of our dev team community has done this already and would like to share the method.
Hopefully this helps,
Best,
Jen Galaxy team
On 6/30/11 7:48 AM, colin molter wrote:
Hello all, I have two related questions about getting data from the Galaxy History. I installed `galaxy in my server, I did some data analyses and now i want the resulting data back.
But I am facing two problems:
1/ I just try to use the download button. If i do it with the button (or with curl or wget), i get the following error: <h1>Proxy Error</h1> <p>The proxy server received an invalid response from an upstream server.<br /> The proxy server could not handle the request <em><a href="/galaxy/datasets/**b701da857886499b/display">GET&** nbsp;/galaxy/datasets/**b701da857886499b/display</a></**em>.<p> Reason: <strong>Error reading from remote server</strong></p></p> </body></html>
any ideas? Does I am suppose to modify the universe_wsgi.ini?
2/ In fact, i would prefer to just copy the result on the same server (no need to use the network). Something similar that what is used to add data to the datalibrary. Is it a way to just copy data from within galaxy to some given place in the same server?
thank you colin -- Colin Molter University of Brussels - InSilico Team - http://insilico.ulb.ac.be/
______________________________**_____________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org/ http://galaxyproject.org/
-- Colin Molter University of Brussels - InSilico Team - http://insilico.ulb.ac.be/
As to how to figure out the id for copying, there isn't an exposed method for doing this. If you're willing to do a little work to decode it, you can do the following in python. For the encoded id (the "b701da857886499b" portion that you see in a dataset url "/galaxy/datasets/b701da857886499b/display"), we use the blowfish implementation in pycrypto, with the 'id_secret' in your universe_wsgi.ini as the key. Given an object_id and the id_secret from Galaxy, you should be able to do something like (code from lib/galaxy/web/security/__init__.py): If you haven't changed it, the id_secret is literally what is used below, "USING THE DEFAULT IS NOT SECURE!", exactly.
id_secret_from_galaxy = "USING THE DEFAULT IS NOT SECURE!" obj_id = "b701da857886499b" from Crypto.Cipher import Blowfish cipher = Blowfish.new(id_secret_from_galaxy) cipher.decrypt( obj_id.decode('hex')).lstrip("!") '70'
And from this ID, we know to grab dataset_70.dat from the galaxy_root/database/files/000/ folder (assuming the default file storage path). Let me know if this isn't clear and I'll try and help out. -Dannon On Jul 3, 2011, at 3:28 PM, colin molter wrote:
Hi, thx for the reply. I will try to make a proper install of galaxy as soon as i have some time. But as for now, more importantly, i just need to make a unix copy of the files that interest me.
so yes, having something from within galaxy could be cool. But a unix copy could make the trick for my first tests. Still I have a problem. I noticed that all my history is located in database/files/ However, how can i know which file corresponds to a given file in my history? Of course, i could work with the file size and the date of creation, but a more reliable and intuitive link would be nice. Could I find it somewhere?
thx colin
2011/7/1 Jennifer Jackson <jen@bx.psu.edu> Hello Colin,
For #1, we have some wiki help pages that go through the set-up steps for this type of proxy configuration. Please see:
https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServer -- scroll to "Make the proxy handle uploads and downloads" which links to these two docs:
Apache: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy -- the section "Sending files using Apache" specifically addresses downloads.
nginx: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/nginxProxy -- also has a section for this function, titled "Sending files using nginx"
For #2, a unix copy of the files could be done directly or by an external script, but it sounds as if you want to do this configured from within Galaxy itself. This could likely be done by setting the download destination for the UI "save" function to be a static location in the configuration. Let's see if any of our dev team community has done this already and would like to share the method.
Hopefully this helps,
Best,
Jen Galaxy team
On 6/30/11 7:48 AM, colin molter wrote: Hello all, I have two related questions about getting data from the Galaxy History. I installed `galaxy in my server, I did some data analyses and now i want the resulting data back.
But I am facing two problems:
1/ I just try to use the download button. If i do it with the button (or with curl or wget), i get the following error: <h1>Proxy Error</h1> <p>The proxy server received an invalid response from an upstream server.<br /> The proxy server could not handle the request <em><a href="/galaxy/datasets/b701da857886499b/display">GET /galaxy/datasets/b701da857886499b/display</a></em>.<p> Reason: <strong>Error reading from remote server</strong></p></p> </body></html>
any ideas? Does I am suppose to modify the universe_wsgi.ini?
2/ In fact, i would prefer to just copy the result on the same server (no need to use the network). Something similar that what is used to add data to the datalibrary. Is it a way to just copy data from within galaxy to some given place in the same server?
thank you colin -- Colin Molter University of Brussels - InSilico Team - http://insilico.ulb.ac.be/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Jackson http://usegalaxy.org/ http://galaxyproject.org/
-- Colin Molter University of Brussels - InSilico Team - http://insilico.ulb.ac.be/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (3)
-
colin molter
-
Dannon Baker
-
Jennifer Jackson