Could someone take a look at this history in Galaxy?  

http://main.g2.bx.psu.edu/u/kenaka/h/count-issue

collapser is not reporting the correct counts in the output for one of my files. 
seqdata1 and seqdata2 should both have 50 reads. 
Yet it is reporting more reads for seqdata1 in both the verbose reporting and the output. 

I have the same issue with the fastx_toolkit, as per below. 
What could be the cause? 

Thanks.
Ken


From: keneclip@hotmail.com
To: galaxy-user@bx.psu.edu
Date: Thu, 14 Apr 2011 03:12:14 +0000
Subject: Re: [galaxy-user] regarding read counts from fastx_clipper

Apologies for the long content..


I seem to be having a discrepancy with fastx_collapser as well. 

I have converted from s_8_sequence_clipped.fa to s_8_sequence_collapsed.fa

fastx_collapser -v -i s8_sequence_clipped.fa -o s8_sequence_collapsed.fa
Input: 26580941 sequences (representing 212647528 reads)    --->  what the..?
Output: 3177400 sequences (representing 212647528 reads)

In my collapsed file, my top read is: 
>1-35820208
TACCTGGTTGATCCTGCCAGTAG

(this is already over my original read count)

When I 

grep -e TACCTGGTTGATCCTGCCAGTAG -c s_8_sequence_clipped.fa 
4,503,566

....

Ok, I've reanalyzed a previous dataset and the output is consistent with my previous numbers:

fastx_collapser -v -i s3-sequence.fa -o s4-sequence-RETEST.fa
Input: 36008043 sequences (representing 36008043 reads)
Output: 3886503 sequences (representing 36008043 reads)

so it doesn't appear to be the toolkit.

How are the tools counting "reads" and "sequences" ?
Could the format of the headers affect it? hyphens vs underscores? 

This data set:
>ILLUMINA-08A740_0000:8:1:1736:1055#0/1
GCGAGCGTAGTTCAATGGTAAAACATCTCCTTGCCAAGGA

Previous data set:
>GAPC_0034_FC:1:1:1548:1028#0/1
AAACTTCATCGTTATCGAGCGA


Thanks

From: keneclip@hotmail.com
To: galaxy-user@bx.psu.edu
Date: Thu, 14 Apr 2011 00:54:05 +0000
Subject: [galaxy-user] regarding read counts from fastx_clipper

Hi, 

I'm using the fastx_toolkit (v0.0.13) command line scripts. 

When using fastx_clipper, I get: 

fastx_clipper -a TCGTATGCCGTCTTCTGCTTG -v -c -l 15 -M 5 -i s_8_sequence.fa -o s_8_sequence_clipped.fa
Clipping Adapter: TCGTATGCCGTCTTCTGCTTG
Min. Length: 15
Non-Clipped reads - discarded.
Input: 227673720 reads.
Output: 212647528 reads.
discarded 3527200 too-short reads.
discarded 725608 adapter-only reads.
discarded 10773384 non-clipped reads.
discarded 0 N reads.

The s_8_sequence.fa file is 2.2Gb, s_8_sequence_clipped.fa file is 1.7Gb.... seems like fastx_clipper is reporting way too many reads in this instance.  
I also tried without the -M option but same thing. 

I checked with:

wc -l s_8_sequence.fa
56918430
(divide this by 2 gives 28,459,215 reads)

wc -l s_8_sequence_clipped.fa
53161882
(divided by 2 gives 26,580,941 reads)


There has never been such a discrepancy with this tool. 

I'm not sure if I'm doing something silly this time round, or somethings changed in my system that's affecting fastx_clipper counting. 

Heres a couple of lines from input and output:

head -n 6 s_8_sequence.fa
>ILLUMINA-08A740_0000:8:1:1736:1055#0/1
GCGAGCGTAGTTCAATGGTAAAACATCTCCTTGCCAAGGA
>ILLUMINA-08A740_0000:8:1:2219:1057#0/1
CAAGCGTCGGAGGTTTAGTCTTTCGTATGCCGTCTTCTGC
>ILLUMINA-08A740_0000:8:1:2316:1056#0/1
TACCTGGTTGATCCTGCCAGTAGTCGTATGCCGTCTTCTG

head -n 6 s_8_sequence_clipped.fa
>ILLUMINA-08A740_0000:8:1:2219:1057#0/1
CAAGCGTCGGAGGTTTAGTCTT
>ILLUMINA-08A740_0000:8:1:2316:1056#0/1
TACCTGGTTGATCCTGCCAGTAG
>ILLUMINA-08A740_0000:8:1:3041:1059#0/1
GAAGCTGCGGGTTCGAGCCCCGTCAGTCCCGCCA


Any ideas? 

Thanks, 

Ken

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/