I had considered cat, but from what I have read, not all readers understand it, or understand it as a single compressed stream. Since the resulting file would differ from a block gzipped file (embedded gzip headers with filenames, embedded per-file trailers, and the final CRC/size not matching the entire contents) I worry cat would be a brittle solution. Also, single AFAIK gzjoin is not part of any installable package (more of an example program) I don't know if it's something that can be addressed by a dependency installation system, unless you host the installations. There is stuff that happens on first start, such as copying *example files. Would it be far-fetched to compile the program at that stage? ________________________________________ From: James Taylor [james@jamestaylor.org] Sent: Tuesday, May 24, 2011 11:51 PM To: Duddy, John Cc: galaxy-dev@bx.psu.edu Dev Subject: Re: [galaxy-dev] Getting binary programs into Galaxy distribution? John, I'll take a look at the program. There isn't a great way to do this until the dependency installation system is working. A thin python wrapper (using Cython) would be the usual trick we would use. However: have you considered just using cat? This should be completely valid for gzip (at the cost of an extra 15 bytes per source file or so for duplicate headers). It looks like gzjoin does require decompression of all input data so this trade off may be worthwhile. On May 24, 2011, at 3:13 PM, Duddy, John wrote:
There is a C program for merging Gzip files (gzjoin) that I’d love to rely on for a core Galaxy capability. Is there a standard way to get things like this included in Galaxy? Recoding it in Python would be a bit of a pain, and might be a lot slower due to the IO layer not allowing the reuse of buffers.
Thanks -
John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jduddy@illumina.com
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: