Additional p-value adjustment for multiple comparisons in DESeq2

Question

[goodeats12]

Hi,

I'm using DESeq2 to analyze RNAseq data. The experiment has 18 conditions (3 different neg. controls and 15 different treatments). In the analysis, I plan to compare each treatment with just one of the neg. controls. I execute this through 15 separate calls to results:

ddssva_de <- DESeq(ddssva)

res1 <- results(ddssva_de, c("condition", "treat1", "ref1"))
res2 <- results(ddssva_de, c("condition", "treat2", "ref1"))
res3 <- results(ddssva_de, c("condition", "treat3", "ref1"))
res4 <- results(ddssva_de, c("condition", "treat4", "ref1"))
res5 <- results(ddssva_de, c("condition", "treat5", "ref1"))
res6 <- results(ddssva_de, c("condition", "treat6", "ref2"))
res7 <- results(ddssva_de, c("condition", "treat7", "ref2"))
res8 <- results(ddssva_de, c("condition", "treat8", "ref2"))
res9 <- results(ddssva_de, c("condition", "treat9", "ref2"))
res10 <- results(ddssva_de, c("condition", "treat10", "ref2"))
res11 <- results(ddssva_de, c("condition", "treat11", "ref3"))
res12 <- results(ddssva_de, c("condition", "treat12", "ref3"))
res13 <- results(ddssva_de, c("condition", "treat13", "ref3"))
res14 <- results(ddssva_de, c("condition", "treat14", "ref3"))
res15 <- results(ddssva_de, c("condition", "treat15", "ref3"))

In doing this, I realize that each results set has adjusted p-values (padj) calculated only using the number of comparisons for that specific contrast while in reality, I am making many, many more comparisons with each additional call to "results."

I would like to further adjust the p-values in each of these results (res1 through res15) taking into account the total number of comparisons made across all 15 calls. What would be a good way to accomplish this?

Thank you.

Answer 1

[Devon Ryan]

You can make a vector of all of the p-values from all of the comparisons and run that through p.adjust(). I think that's overkill, though, since I think most people would argue that you're not testing each contrast "simultaneously", thus removing the need to adjust for multiple comparisons across contrasts. Also note that performing multiple comparisons correction across contrasts will remove the ability to perform independent filtering within each contrast.

[goodeats12]

Thanks for your comment Devon. Perhaps this question is really about where to draw the line for "overkill" with respect to adjusting p-values. If, instead of calling results() 15 times on the same DESeqDataSet, I instead split up the data and create 15 separate DESeqDataSets and call DESeq() on each one separately, I can obtain even more permissive adjusted p-values (and hence more significant genes). Where to draw the line?

[Devon Ryan]

That's fitting a different model, so the differences are actually do to that.

[goodeats12]

Ah, yes. You are correct. That helps me think about this problem in a better way. Thank you!