Uploading large archive histories
Galaxy Team, I exported a large history that reduced to an 8GB compressed file. I would like to re-import this history but the "Import from file" options gives me an error (see attached image). I looked around the wiki and there appears to be an ftp upload for large files (http://wiki.galaxyproject.org/FTPUpload). Is there an equivalent FTP upload for exported large histories? I would like to export some of my large analysis histories and save them in an external hard drives to free some space in my account. However, I am not able to re-import large histories. I want to make sure I can re-import these large histories before I delete them from my Galaxy account. Getiria -- Getiria Onsongo, Ph.D. Informatics Analyst, Research Informatics Support System Minnesota Supercomputing Institute for Advanced Computational Research University of Minnesota Minneapolis, MN 55455 Phone: 612-624-0532
After discussing this with Getiria, I think what is needed here is a way to use FTP to upload compressed histories from the import screen as is available from the upload data tool. The problem we're currently experiencing is that when we try to upload a compressed history where the compressed file is greater than 2GB, the process fails. This is preventing us from being able to effectively have users archive data from Galaxy to reduce our disk usage. This is a very pressing issue for us with our production Galaxy server at the Minnesota Supercomputing Institute. Additionally, if someone could provide a better understanding of how the exported histories are treated on the Galaxy server that would be helpful. e.g. how long do the compressed histories stick around on disk, etc? We would like to have users archive their data on their own systems and permanently delete their Galaxy history when an analysis is complete, but if the compressed history persists on the Galaxy server the utility of this is somewhat reduced. Thanks! On Sat, Feb 16, 2013 at 9:16 AM, Innocent Onsongo <onson001@umn.edu> wrote:
Galaxy Team,
I exported a large history that reduced to an 8GB compressed file. I would like to re-import this history but the "Import from file" options gives me an error (see attached image). I looked around the wiki and there appears to be an ftp upload for large files ( http://wiki.galaxyproject.org/FTPUpload). Is there an equivalent FTP upload for exported large histories?
I would like to export some of my large analysis histories and save them in an external hard drives to free some space in my account. However, I am not able to re-import large histories. I want to make sure I can re-import these large histories before I delete them from my Galaxy account.
Getiria
-- Getiria Onsongo, Ph.D. Informatics Analyst, Research Informatics Support System Minnesota Supercomputing Institute for Advanced Computational Research University of Minnesota Minneapolis, MN 55455 Phone: 612-624-0532 ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jesse Erdmann jesse@jesseerdmann.com Twitter: http://twitter.com/jesseerdmann
Hi Getiria, Jeremy gave advice for the local, but I wanted to follow up about the trouble you were having uploading to the public Main Galaxy instance at https://main.g2.bx.psu.edu/ (usegalaxy.org). Would you please try a larger history upload again today, if you have not already done so, and let us know if it is still problematic? Our apologies for the problems later last week. We believe these are cleared now, but want to make sure this function is working from your end as expected. Best, Jen Galaxy team On 2/16/13 7:16 AM, Innocent Onsongo wrote:
Galaxy Team,
I exported a large history that reduced to an 8GB compressed file. I would like to re-import this history but the "Import from file" options gives me an error (see attached image). I looked around the wiki and there appears to be an ftp upload for large files (http://wiki.galaxyproject.org/FTPUpload). Is there an equivalent FTP upload for exported large histories?
I would like to export some of my large analysis histories and save them in an external hard drives to free some space in my account. However, I am not able to re-import large histories. I want to make sure I can re-import these large histories before I delete them from my Galaxy account.
Getiria
-- Getiria Onsongo, Ph.D. Informatics Analyst, Research Informatics Support System Minnesota Supercomputing Institute for Advanced Computational Research University of Minnesota Minneapolis, MN 55455 Phone: 612-624-0532
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org
participants (3)
-
Innocent Onsongo -
Jennifer Jackson -
Jesse Erdmann