Not what I was hoping for.

I usually link without copying into galaxy, so it won't help me. I also got excited because I thought there was the possibility of working with fastq files without having to keep unzipped datasets, that would safe a lot of space, even if processing gets slower.

Hans, I might take a look at modifying Galaxy underlying code, but in general I'm not fun of keeping up with my own custom code.

Thanks, Carlos

On Tue, Aug 21, 2012 at 11:04 AM, Langhorst, Brad Langhorst@neb.com wrote:

This is not tool specific. Instead, galaxy automatically unzips as files are uploaded.

beware: it only supports one data set per zipped file - so gzipped fastqs are find.. but a tarball of a bunch of fastqs is not.

Brad On Aug 21, 2012, at 10:23 AM, Carlos Borroto carlos.borroto@gmail.com wrote:

...

Interesting...

Hans, could you specify what tools currently available in Galaxy support gzip'd fastq files?

Any in this list?:

• Bowtie/Tophat
• FastQC
• FASTQ Groomer

Having the possibility to use these tools without needing to unzip fastq files would be great and I didn't know you could do that.

Thanks, Carlos

On Tue, Aug 21, 2012 at 3:16 AM, Hans-Rudolf Hotz hrh@fmi.ch wrote:

...

moving to dev......

Hi Jen and Brandon

well, I don't agree (or maybe I misunderstood the question/answer):

we work a lot with compressed data (ie g'zipped fastq files from the sequencer), and link them as a Galaxy data library. We preselect 'fastq' as file format.

And then, the first tool in our NGS analysis expects a gzipped fastq file.

Regards, Hans

-------- Original Message -------- Subject: Re: [galaxy-user] Linking to Compressed Data Date: Mon, 20 Aug 2012 10:32:26 -0700 From: Jennifer Jackson jen@bx.psu.edu To: Branden Timm btimm@wisc.edu CC: galaxy-user@lists.bx.psu.edu

Hi Branden,

Data in a library needs to be uncompressed in order for Galaxy to index and have access to it correctly.

I updated our wiki for the "upload by filesystem link" option: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files

Good question!

Jen Galaxy team

On 8/15/12 9:09 AM, Branden Timm wrote:

...

Hi All, Is it possible to link to compressed files in a Galaxy data library? We receive all of our NGS data in bz2 or gzip format for obvious reasons, just wondering if I have to decompress it on the filesystem before I link to it or not. Thanks!

-- Branden Timm btimm@glbrc.wisc.edu ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- Jennifer Jackson http://galaxyproject.org ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- Brad Langhorst langhorst@neb.com 978-380-7564

On 08/21/2012 04:23 PM, Carlos Borroto wrote:

Interesting...

Hans, could you specify what tools currently available in Galaxy support gzip'd fastq files?

Any in this list?:

• Bowtie/Tophat
• FastQC
• FASTQ Groomer

I can't tell you, as we don't use any of these tools in Galaxy as provided in galaxy-dist.

For NGS, we mostly use our self written tools which are tightly coupled to our local set up. And one of them explicitly expect a gzip'd fastq file.

The Galaxy implementation of the FASTX-toolkit written by Assaf Gordon (http://hannonlab.cshl.edu/fastx_toolkit/index.html) works with gzip'd fastq files.

And keep in mind, you can modify the galaxy wrappers to include a conditional 'zcat' (or 'gunzip -c')

Regards Hans

Having the possibility to use these tools without needing to unzip fastq files would be great and I didn't know you could do that.

Thanks, Carlos

On Tue, Aug 21, 2012 at 3:16 AM, Hans-Rudolf Hotz hrh@fmi.ch wrote:

...

moving to dev......

Hi Jen and Brandon

well, I don't agree (or maybe I misunderstood the question/answer):

we work a lot with compressed data (ie g'zipped fastq files from the sequencer), and link them as a Galaxy data library. We preselect 'fastq' as file format.

And then, the first tool in our NGS analysis expects a gzipped fastq file.

Regards, Hans

-------- Original Message -------- Subject: Re: [galaxy-user] Linking to Compressed Data Date: Mon, 20 Aug 2012 10:32:26 -0700 From: Jennifer Jackson jen@bx.psu.edu To: Branden Timm btimm@wisc.edu CC: galaxy-user@lists.bx.psu.edu

Hi Branden,

Data in a library needs to be uncompressed in order for Galaxy to index and have access to it correctly.

I updated our wiki for the "upload by filesystem link" option: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files

Good question!

Jen Galaxy team

On 8/15/12 9:09 AM, Branden Timm wrote:

...

Hi All, Is it possible to link to compressed files in a Galaxy data library? We receive all of our NGS data in bz2 or gzip format for obvious reasons, just wondering if I have to decompress it on the filesystem before I link to it or not. Thanks!

-- Branden Timm btimm@glbrc.wisc.edu ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

-- Jennifer Jackson http://galaxyproject.org ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

http://lists.bx.psu.edu/