Cuffdiff statistical calculations are inconsistent?
From run 1: YFR026C FPKM 1 = 17.2434 FPKM 2 = 196.735 log2(fold change) = 3.51214
From run 2: YFR026C FPKM 1 = 14.4489 FPKM 2 = 144.939 log2(fold change) = 3.32641
Hi, I'll preface my concern by saying that I'm a novice to Cufflinks. Back in September, I performed a Cuffdiff analysis comparing a wild-type and mutant condition. The analysis returned ~800 transcripts differentially regulated between the two with statistical significance. Recently, I've rerun the Cuffdiff analysis - using exactly the same files stored in Galaxy for all inputs, and with all the same parameters - and only get a few dozen statistically significant hits. However, all of the data besides the p and q values are essentially identical between these two runs, so I am really unclear as to what is causing the difference. Here is just one clear example: p = 1.64E-8 q = 7.33E-6 significant = yes p = 0.000170034 q = 0.0719964 significant = no The second Cuffdiff analysis shows there is still a ~10-fold difference between conditions, but this is not statistically significant. Has the version of Cuffdiff on Galaxy been updated such that some parameters have changed, that could explain this difference? Or, is there some setting I am missing that would cause very large changes to fail statistical significance testing? Any help or input would be appreciated, I am really at a loss for why executing what should be exactly the same task is giving vastly different results. I could just be overlooking something very fundamental that is obvious to someone with more experience with this program. Thanks. -Jenna Smith
We are having the exact same issue, on the main server and our (recent) cloud instances. Were some of the hidden Cuffdiff parameters modified since fall 2012? Cheers, Mo Heydarian On Mar 13, 2013 11:02 AM, "Jenna Smith" <jes227@case.edu> wrote:
Hi,
I'll preface my concern by saying that I'm a novice to Cufflinks. Back in September, I performed a Cuffdiff analysis comparing a wild-type and mutant condition. The analysis returned ~800 transcripts differentially regulated between the two with statistical significance. Recently, I've rerun the Cuffdiff analysis - using exactly the same files stored in Galaxy for all inputs, and with all the same parameters - and only get a few dozen statistically significant hits. However, all of the data besides the p and q values are essentially identical between these two runs, so I am really unclear as to what is causing the difference. Here is just one clear example:
From run 1: YFR026C FPKM 1 = 17.2434 FPKM 2 = 196.735 log2(fold change) = 3.51214 p = 1.64E-8 q = 7.33E-6 significant = yes
From run 2: YFR026C FPKM 1 = 14.4489 FPKM 2 = 144.939 log2(fold change) = 3.32641 p = 0.000170034 q = 0.0719964 significant = no
The second Cuffdiff analysis shows there is still a ~10-fold difference between conditions, but this is not statistically significant. Has the version of Cuffdiff on Galaxy been updated such that some parameters have changed, that could explain this difference? Or, is there some setting I am missing that would cause very large changes to fail statistical significance testing? Any help or input would be appreciated, I am really at a loss for why executing what should be exactly the same task is giving vastly different results. I could just be overlooking something very fundamental that is obvious to someone with more experience with this program. Thanks.
-Jenna Smith
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
This is likely due to the upgrade from Cufflinks 1.3.x to Cufflinks 2.0.x; Cufflinks 2.0 introduced a new algorithm for Cuffdiff in particular. You can read about these changes on the website: http://cufflinks.cbcb.umd.edu/ (and there's a manuscript describing the changes as well). You might consider writer to to the tool authors directly for more details: tophat.cufflinks@gmail.com Of course, please consider sharing anything you learn with members of this list as well. Best, J. On Mar 13, 2013, at 12:06 PM, Mohammad Heydarian wrote:
We are having the exact same issue, on the main server and our (recent) cloud instances.
Were some of the hidden Cuffdiff parameters modified since fall 2012?
Cheers, Mo Heydarian
On Mar 13, 2013 11:02 AM, "Jenna Smith" <jes227@case.edu> wrote: Hi,
I'll preface my concern by saying that I'm a novice to Cufflinks. Back in September, I performed a Cuffdiff analysis comparing a wild-type and mutant condition. The analysis returned ~800 transcripts differentially regulated between the two with statistical significance. Recently, I've rerun the Cuffdiff analysis - using exactly the same files stored in Galaxy for all inputs, and with all the same parameters - and only get a few dozen statistically significant hits. However, all of the data besides the p and q values are essentially identical between these two runs, so I am really unclear as to what is causing the difference. Here is just one clear example:
From run 1: YFR026C FPKM 1 = 17.2434 FPKM 2 = 196.735 log2(fold change) = 3.51214 p = 1.64E-8 q = 7.33E-6 significant = yes
From run 2: YFR026C FPKM 1 = 14.4489 FPKM 2 = 144.939 log2(fold change) = 3.32641 p = 0.000170034 q = 0.0719964 significant = no
The second Cuffdiff analysis shows there is still a ~10-fold difference between conditions, but this is not statistically significant. Has the version of Cuffdiff on Galaxy been updated such that some parameters have changed, that could explain this difference? Or, is there some setting I am missing that would cause very large changes to fail statistical significance testing? Any help or input would be appreciated, I am really at a loss for why executing what should be exactly the same task is giving vastly different results. I could just be overlooking something very fundamental that is obvious to someone with more experience with this program. Thanks.
-Jenna Smith
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/ ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Jeremy, The header of the Cuffdiff tool page says it is version 0.0.5 Is there a way, or setting, on Cuffdiff 2.0 to revert the parameters to be more similar to Cuffdiff 1.3? Cheers, Mo Heydarian PhD candidate The Johns Hopkins School of Medicine Department of Biological Chemistry 725 Wolfe Street 402 Biophysics Baltimore, MD 21205 On Wed, Mar 13, 2013 at 12:41 PM, Jeremy Goecks <jeremy.goecks@emory.edu>wrote:
This is likely due to the upgrade from Cufflinks 1.3.x to Cufflinks 2.0.x; Cufflinks 2.0 introduced a new algorithm for Cuffdiff in particular. You can read about these changes on the website: http://cufflinks.cbcb.umd.edu/ (and there's a manuscript describing the changes as well).
You might consider writer to to the tool authors directly for more details: tophat.cufflinks@gmail.com Of course, please consider sharing anything you learn with members of this list as well.
Best, J.
On Mar 13, 2013, at 12:06 PM, Mohammad Heydarian wrote:
We are having the exact same issue, on the main server and our (recent) cloud instances.
Were some of the hidden Cuffdiff parameters modified since fall 2012?
Cheers, Mo Heydarian On Mar 13, 2013 11:02 AM, "Jenna Smith" <jes227@case.edu> wrote:
Hi,
I'll preface my concern by saying that I'm a novice to Cufflinks. Back in September, I performed a Cuffdiff analysis comparing a wild-type and mutant condition. The analysis returned ~800 transcripts differentially regulated between the two with statistical significance. Recently, I've rerun the Cuffdiff analysis - using exactly the same files stored in Galaxy for all inputs, and with all the same parameters - and only get a few dozen statistically significant hits. However, all of the data besides the p and q values are essentially identical between these two runs, so I am really unclear as to what is causing the difference. Here is just one clear example:
From run 1: YFR026C FPKM 1 = 17.2434 FPKM 2 = 196.735 log2(fold change) = 3.51214 p = 1.64E-8 q = 7.33E-6 significant = yes
From run 2: YFR026C FPKM 1 = 14.4489 FPKM 2 = 144.939 log2(fold change) = 3.32641 p = 0.000170034 q = 0.0719964 significant = no
The second Cuffdiff analysis shows there is still a ~10-fold difference between conditions, but this is not statistically significant. Has the version of Cuffdiff on Galaxy been updated such that some parameters have changed, that could explain this difference? Or, is there some setting I am missing that would cause very large changes to fail statistical significance testing? Any help or input would be appreciated, I am really at a loss for why executing what should be exactly the same task is giving vastly different results. I could just be overlooking something very fundamental that is obvious to someone with more experience with this program. Thanks.
-Jenna Smith
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
The header of the Cuffdiff tool page says it is version 0.0.5
This version is the Galaxy tool wrapper version, not the tool version. (Yes, this is a usability issue.) You can find the tool version in the dataset's information panel by clicking on the 'i' icon.
Is there a way, or setting, on Cuffdiff 2.0 to revert the parameters to be more similar to Cuffdiff 1.3?
This isn't a parameter issue. The Cuffdiff algorithm has changed substantially, and it's not clear to me if/how (or whether it's a good idea at all) to modify parameters to obtain 1.3-esque results. Best, J.
participants (3)
-
Jenna Smith -
Jeremy Goecks -
Mohammad Heydarian