Uploading large file in browser
Hello, I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files ( > 2GB). The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls. I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance. My questions are: -Is it too cumbersome to achieve this goal with external tools? -how deep should I hit the galaxy (at the code level) to integrate this jquery tool? -Are there any alternatives to upload large files in browser without FTP? Thanks, Daniel
On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify “upload file” so that I will be able to upload large files ( > 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi again, I'm kind of lost here. Does "Data Library" allow regular users to upload files directly to Galaxy from their remote work station, or does it allow users to use files that are already exist in Galaxy? My intention was to using HTML5 file uploader which chunks large file(> 2GB) into smaller pieces so that regular users can upload large files through Galaxy's GUI without external FTP tool. Thanks, Hyunsoo -----Original Message----- From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] Sent: Wednesday, January 25, 2012 4:02 AM To: Kim, Hyunsoo Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Uploading large file in browser On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files ( > 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On 01/25/2012 08:20 PM, Kim, Hyunsoo wrote:
Hi again,
I'm kind of lost here. Does "Data Library" allow regular users to upload files directly to Galaxy from their remote work station, or does it allow users to use files that are already exist in Galaxy?
just to clarify: regular user can copy (ie: using scp) their data to the galaxy server, and then use the advantages of Data Libraries, see the "user_library_import_dir" option in the 'universe_wsgi.ini' file. Regards, Hans
My intention was to using HTML5 file uploader which chunks large file(> 2GB) into smaller pieces so that regular users can upload large files through Galaxy's GUI without external FTP tool.
Thanks, Hyunsoo
-----Original Message----- From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] Sent: Wednesday, January 25, 2012 4:02 AM To: Kim, Hyunsoo Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Uploading large file in browser
On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files (> 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see:
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files
Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Jan 26, 2012, at 4:29 AM, Hans-Rudolf Hotz wrote:
On 01/25/2012 08:20 PM, Kim, Hyunsoo wrote:
Hi again,
I'm kind of lost here. Does "Data Library" allow regular users to upload files directly to Galaxy from their remote work station, or does it allow users to use files that are already exist in Galaxy?
just to clarify: regular user can copy (ie: using scp) their data to the galaxy server, and then use the advantages of Data Libraries, see the "user_library_import_dir" option in the 'universe_wsgi.ini' file.
Regards, Hans
It's also possible to use the "FTP Upload" functionality with a protocol other than standard FTP (i.e. with scp) so that users can upload scp'd files directly to a history. --nate
My intention was to using HTML5 file uploader which chunks large file(> 2GB) into smaller pieces so that regular users can upload large files through Galaxy's GUI without external FTP tool.
Thanks, Hyunsoo
-----Original Message----- From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] Sent: Wednesday, January 25, 2012 4:02 AM To: Kim, Hyunsoo Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Uploading large file in browser
On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files (> 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see:
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files
Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi All, Apologies for resurrecting an old thread, but an HTML5 chunk uploader as Hyunsoo previously suggested would actually be very useful for my organisation, in terms of streamlining our workflow for less technically able users. I was wondering if there was any intention to include functionality like this at some point, although I'm perfectly happy to use other methods if there isn't. Cheers, -Nic On 31/01/2012, at 7:12 AM, Nate Coraor wrote:
On Jan 26, 2012, at 4:29 AM, Hans-Rudolf Hotz wrote:
On 01/25/2012 08:20 PM, Kim, Hyunsoo wrote:
Hi again,
I'm kind of lost here. Does "Data Library" allow regular users to upload files directly to Galaxy from their remote work station, or does it allow users to use files that are already exist in Galaxy?
just to clarify: regular user can copy (ie: using scp) their data to the galaxy server, and then use the advantages of Data Libraries, see the "user_library_import_dir" option in the 'universe_wsgi.ini' file.
Regards, Hans
It's also possible to use the "FTP Upload" functionality with a protocol other than standard FTP (i.e. with scp) so that users can upload scp'd files directly to a history.
--nate
My intention was to using HTML5 file uploader which chunks large file(> 2GB) into smaller pieces so that regular users can upload large files through Galaxy's GUI without external FTP tool.
Thanks, Hyunsoo
-----Original Message----- From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] Sent: Wednesday, January 25, 2012 4:02 AM To: Kim, Hyunsoo Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Uploading large file in browser
On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files (> 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see:
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files
Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Feb 9, 2012, at 7:05 PM, Nic Barker wrote:
Hi All, Apologies for resurrecting an old thread, but an HTML5 chunk uploader as Hyunsoo previously suggested would actually be very useful for my organisation, in terms of streamlining our workflow for less technically able users. I was wondering if there was any intention to include functionality like this at some point, although I'm perfectly happy to use other methods if there isn't. Cheers, -Nic
Hi Nic, I started working on HTML5 uploading a while back but didn't get too far with it. We'll definitely get back to rewriting uploading at some point. --nate
On 31/01/2012, at 7:12 AM, Nate Coraor wrote:
On Jan 26, 2012, at 4:29 AM, Hans-Rudolf Hotz wrote:
On 01/25/2012 08:20 PM, Kim, Hyunsoo wrote:
Hi again,
I'm kind of lost here. Does "Data Library" allow regular users to upload files directly to Galaxy from their remote work station, or does it allow users to use files that are already exist in Galaxy?
just to clarify: regular user can copy (ie: using scp) their data to the galaxy server, and then use the advantages of Data Libraries, see the "user_library_import_dir" option in the 'universe_wsgi.ini' file.
Regards, Hans
It's also possible to use the "FTP Upload" functionality with a protocol other than standard FTP (i.e. with scp) so that users can upload scp'd files directly to a history.
--nate
My intention was to using HTML5 file uploader which chunks large file(> 2GB) into smaller pieces so that regular users can upload large files through Galaxy's GUI without external FTP tool.
Thanks, Hyunsoo
-----Original Message----- From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] Sent: Wednesday, January 25, 2012 4:02 AM To: Kim, Hyunsoo Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Uploading large file in browser
On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files (> 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see:
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files
Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Nate, That's good to hear. Just letting you know that testing on my local install of galaxy, if the following stars align: -Google Chrome 17 / Firefox 10 (Mac os x) -Nginx as galaxy's front end web server -Nginx's optional upload module installed HTML uploading of large files to galaxy (tested up to 20gb) just "works". I believe this is due to chrome and FF's support of the encoding type "multipart/form-data". Cheers, -Nic On 14/02/2012, at 4:00 AM, Nate Coraor wrote:
On Feb 9, 2012, at 7:05 PM, Nic Barker wrote:
Hi All, Apologies for resurrecting an old thread, but an HTML5 chunk uploader as Hyunsoo previously suggested would actually be very useful for my organisation, in terms of streamlining our workflow for less technically able users. I was wondering if there was any intention to include functionality like this at some point, although I'm perfectly happy to use other methods if there isn't. Cheers, -Nic
Hi Nic,
I started working on HTML5 uploading a while back but didn't get too far with it. We'll definitely get back to rewriting uploading at some point.
--nate
On 31/01/2012, at 7:12 AM, Nate Coraor wrote:
On Jan 26, 2012, at 4:29 AM, Hans-Rudolf Hotz wrote:
On 01/25/2012 08:20 PM, Kim, Hyunsoo wrote:
Hi again,
I'm kind of lost here. Does "Data Library" allow regular users to upload files directly to Galaxy from their remote work station, or does it allow users to use files that are already exist in Galaxy?
just to clarify: regular user can copy (ie: using scp) their data to the galaxy server, and then use the advantages of Data Libraries, see the "user_library_import_dir" option in the 'universe_wsgi.ini' file.
Regards, Hans
It's also possible to use the "FTP Upload" functionality with a protocol other than standard FTP (i.e. with scp) so that users can upload scp'd files directly to a history.
--nate
My intention was to using HTML5 file uploader which chunks large file(> 2GB) into smaller pieces so that regular users can upload large files through Galaxy's GUI without external FTP tool.
Thanks, Hyunsoo
-----Original Message----- From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] Sent: Wednesday, January 25, 2012 4:02 AM To: Kim, Hyunsoo Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Uploading large file in browser
On 01/24/2012 10:27 PM, Kim, Hyunsoo wrote:
Hello,
I have local instance of galaxy and wanted to modify "upload file" so that I will be able to upload large files (> 2GB).
The reason I am trying to do this in browser is that extra tools for FTP do not really work in my environment because of all the constraints and firewalls.
I came up with jQuery file upload tool (http://blueimp.github.com/jQuery-File-Upload/), and the tool seems fine if it is possible me to integrate into my galaxy instance.
My questions are:
-Is it too cumbersome to achieve this goal with external tools?
-how deep should I hit the galaxy (at the code level) to integrate this jquery tool?
-Are there any alternatives to upload large files in browser without FTP?
as an alternative: have you looked into using "Data Libraries", see:
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries
http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files
Regards, Hans
Thanks,
Daniel
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (4)
-
Hans-Rudolf Hotz -
Kim, Hyunsoo -
Nate Coraor -
Nic Barker