Hi Jim, You message was misthreaded (perhaps a reply to another thread, with just the subject line changed?), but I was able to dig it out. A this time, there are no known issues with FTP Upload to the public Main server. Any issues you have have found prior were either related to a problem with the original file content (compression problem) or a transitory issue with the FTP server that has since been resolved (there has been a handful in the last few years). The instructions to follow are here: http://wiki.galaxyproject.org/FTPUpload I am not exactly sure what your issue is, but any chance that you have more than one file per archive? That will certainly cause an issue, but usually with just the first file loading the remainder not. Please send more details if this continues. Does the failure occur at the FTP stage or at the point where you move from the FTP holding area into a history? Thanks! Jen Galaxy team On 2/5/13 5:48 AM, Jim Robinson wrote:
Hi,
I am having a lot of difficulty uploading some large gzipped fastqs (~ 10GB) to the public server. I have tried both ftp and "pulling" by http URL. The upload succeeds, however I get an error as it tries to gunzip it. I have tried more than 10 times now and succeeded once. These files are correct and complete, and gunzip properly locally. The error shown is usually this
empty format: txt, database: ? Problem decompressing gzipped data
However on 2 occasions (both ftp uploads) I got the traceback below. Am I missing some obvious trick? I searched the archives and see references to problems with large gzipped files but no solutions.
Thanks
Jim
Traceback (most recent call last): File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 384, in <module> __main__() File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 373, in __main__ add_file( dataset, registry, json_file, output_path ) File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 270, in add_file line_count, converted_path = sniff.convert_newlines( dataset.path, in_place=in_place ) File "/galaxy/home/g2main/galaxy_main/lib/galaxy/datatypes/sniff.py", line 106, in convert_newlines shutil.move( temp_name, fname ) File "/usr/lib/python2.7/shutil.py", line 299, in move copy2(src, real_dst) File "/usr/lib/python2.7/shutil.py", line 128, in copy2 copyfile(src, dst) File "/usr/lib/python2.7/shutil.py", line 84, in copyfile copyfileobj(fsrc, fdst) File "/usr/lib/python2.7/shutil.py", line 49, in copyfileobj buf = fsrc.read(length) IOError: [Errno 5] Input/output error
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org