Hi,
I am having a lot of difficulty uploading some large gzipped fastqs (~ 10GB) to the public server. I have tried both ftp and "pulling" by http URL. The upload succeeds, however I get an error as it tries to gunzip it. I have tried more than 10 times now and succeeded once. These files are correct and complete, and gunzip properly locally. The error shown is usually this
empty format: txt, database: ? Problem decompressing gzipped data
However on 2 occasions (both ftp uploads) I got the traceback below. Am I missing some obvious trick? I searched the archives and see references to problems with large gzipped files but no solutions.
Thanks
Jim
Traceback (most recent call last): File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 384, in <module> __main__() File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 373, in __main__ add_file( dataset, registry, json_file, output_path ) File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 270, in add_file line_count, converted_path = sniff.convert_newlines( dataset.path, in_place=in_place ) File "/galaxy/home/g2main/galaxy_main/lib/galaxy/datatypes/sniff.py", line 106, in convert_newlines shutil.move( temp_name, fname ) File "/usr/lib/python2.7/shutil.py", line 299, in move copy2(src, real_dst) File "/usr/lib/python2.7/shutil.py", line 128, in copy2 copyfile(src, dst) File "/usr/lib/python2.7/shutil.py", line 84, in copyfile copyfileobj(fsrc, fdst) File "/usr/lib/python2.7/shutil.py", line 49, in copyfileobj buf = fsrc.read(length) IOError: [Errno 5] Input/output error