Quick supplement to what Nate said, if you did want to ship data from dropbox -> galaxy, you could (temporarily) throw the file to be shipped in your dropbox public folder and then put that public download link into galaxy's upload box and it would work fine. -Dannon On Aug 19, 2010, at 12:37 PM, Nate Coraor wrote:
Kevin Lam wrote:
Hi Nate, While I do not have ideas to improve browser uploads but i do have some untested ideas for uploads for ppl who like me are unlikely to have access to the institute's ftp or http webserver to serve files. It might be a tad convoluted but have you considered dropbox integration for galaxy? I have terrible upload speeds for amazon S3 but that might be a consideration for some.
Hi Kevin,
Thanks for the suggestion. The donwside I see here is that it'd be a two step process to get it to S3 and then from there to Galaxy (and in addition, it's not free). Ideally we'd need a method to upload to Galaxy directly, but without the browser flakiness.
Of course we can do this now with something like FTP, but a browser based solution would be more ideal.
with regards to local install, is there a way I can point galaxy directly to the file? or make it access it directly instead of my workaround by using python /perl scripts to emulate a webserver?
Not for history uploads, but you can with Library uploads (and then import the library item into your history):
http://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/UploadingFiles
--nate
Cheers Kevin On Thu, Aug 19, 2010 at 3:42 AM, Nate Coraor <nate@bx.psu.edu <mailto:nate@bx.psu.edu>> wrote: Hi Farhat and others, Thanks for reporting your experiences. Does it seem to be everyone's experience that direct file uploads of files > 2GB fails? We are looking in to more reliable methods for handling direct uploads, so any suggestions are welcome. --nate Farhat Habib wrote: I ran into this issue too repeatedly. Both with local install of Galaxy and the main instance. Files over roughly 2GB or so refused to upload, the page would still show them as uploading but nothing was happening. Compressing the same file so it was under 2GB and uploading worked fine both on the local instance and main. -Farhat -- Farhat Habib, PhD IISER Fellow, Biology Indian Institute of Science Education & Research, Sai Trinity Building, Garware Circle, Sutarwadi, Pashan, Pune-411 021. http://www.iiserpune.ac.in/~farhat <http://www.iiserpune.ac.in/%7Efarhat> Office: +91 20 2590 8015 Cell: +91 973 0996 233 On Wed, Aug 18, 2010 at 12:25 PM, Abhay Krishna <abhay.krishna@gmail.com <mailto:abhay.krishna@gmail.com> <mailto:abhay.krishna@gmail.com <mailto:abhay.krishna@gmail.com>>> wrote: Dear Asifullah As Kevin says, you are much better off by using http or ftp url, Also i know there are issues if you refresh the browser while uploading, So I guess if uploading does not work normally, try putting on http or ftp server and give a url, and do not touch the browser untill uploading is over also you can alternatively use a perl or python script locally http://seqanswers.com/forums/showthread.php?t=269 best Abhay Krishna On 18 August 2010 08:47, Kevin Lam <aboulia@gmail.com <mailto:aboulia@gmail.com> <mailto:aboulia@gmail.com <mailto:aboulia@gmail.com>>> wrote: I have created a local install of galaxy and i can tell you it is not about network connection. As I am using a local install but the 'upload' is horribly slow. a 3Gb file can't finish in 24 hours. in the end, I used a python script to 'serve' the file via http protocol and entered a local LAN url for galaxy and it loaded up in minutes. For your case, I think I have no issues up to 250 Mb on their galaxy main server, but as the wiki and manual (plus my experience with a local instance) suggests you would do well to put the file on a webserver and give the url instead of upload via browser. Cheers Kevin On Wed, Aug 18, 2010 at 2:17 PM, asifullah khan <asifullah111@gmail.com <mailto:asifullah111@gmail.com> <mailto:asifullah111@gmail.com <mailto:asifullah111@gmail.com>>> wrote: Dear Officer, I am a new user to assemble 75bp illumina solexa data. I have done single read illumina sequencing of my DNA of interest. it is a single file of about 370MB. The data, in file is initially in FASTQ format but letter the data arrangement become in some different kind of FASTq format inside the file. I want to convert the whole data in to simple FASTA format using GALAXY tools. I see the interesting videos on your site about using galaxy and it seems very interesting. But here i fail to upload my 370MB data file as an initial step. I am using 100Mbp LANE networking. Are you people preferring any other special network connection for uploading such huge files on GALAXY.? Kindly guide in this aspect. Moreover Kindly if possible then send me simple perl script and there command lines usage description for converting any format of FASTq in to FASTA. Regards, Asifullah Khan Research officer, DNA sequencing Labs, ICCBS, University of Karachi, Pakistan. _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu> <mailto:galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu>> http://lists.bx.psu.edu/listinfo/galaxy-user _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu> <mailto:galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu>> http://lists.bx.psu.edu/listinfo/galaxy-user _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu> <mailto:galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu>> http://lists.bx.psu.edu/listinfo/galaxy-user ------------------------------------------------------------------------ _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu> http://lists.bx.psu.edu/listinfo/galaxy-user _______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu <mailto:galaxy-user@lists.bx.psu.edu> http://lists.bx.psu.edu/listinfo/galaxy-user
_______________________________________________ galaxy-user mailing list galaxy-user@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-user