This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How reproducible is the DEG analysis?

Hi,

Currently I am replicating a few DEG analysis for publicly available datasets. However, my results are not that close to original publications. For example, I got genes related to immune response just like in the original paper. However, genes are not the same. Since I found similar pathways, I thought maybe DEG analysis is not that reproducible? Thoughts, experiences?

deg analysis
• 1,737 views • link
updated by Biostar Biostar • written by asumani asumani

When you say you are "replicating" the DEG analysis do you mean:

  1. using their data and their methods in an attempt to get the same results as them.

  2. Using their data but your methods

  3. Generating a new datasets (making new RNA libraries etc) that should be a replicate of theirs?

• 0 views • link
written by i.sudbery i.sudbery

Ah, I forgot it. Thanks! I mean 2.

• 0 views • link
written by asumani asumani

1 answer

The extent to which two different pipelines produce the same results will depend on how different the two piplines are, but in general I would expect two pipelines run on the same data to more or less the same results.

For example, in a recent study on alignment methods, it was found that the majority of DE genes were the same whether Bowtie/STAR/quasi alignment or selective alignment were used. However each difference will cause more differences in the results. This will be particularly the case if the study is "on the edge" - if it it only finds a small number of genes that are hovering close to the boundary of significance, then this is likely to vary more between pipelines.

• 0 views • link
written by i.sudbery i.sudbery

However each difference will cause more differences in the results.

I use different tools for each alignment/assembly/DE steps. Maybe that's why I find only common pathways and not the genes. I would like to hear more about this point if possible.

if the study is "on the edge"

Also, on your last point, I checked out one reference paper, it was stated that there is "substantial differential expression" at more than a thousand genes with FDR <= 0.05. However in the supplementary file, fold change of the genes is not given. I am left to believe that there is at least 1.5 fold change, which I use as threshold in my analysis. So, do you think this can also make the study "on the edge"? This is a highly reputable paper, so I am not sure about how to make an inference on this point.

• 0 views • link
written by asumani asumani

No, 1000 genes at FDR<0.05 does not sound like its on the edge. Also I wouldn't assume that they've used a 1.5 fold change threshold unless they say so. Do you get a similar number of genes?

You might be intrested in this paper: https://www.nature.com/articles/s41467-017-00050-4

which looks at lots of different combinations of pipeline steps .

• 0 views • link
written by i.sudbery i.sudbery

I believe the number of genes is not similar. They got "substantially expressed" and significant 1412 genes. I got 1170 (FDR < 0.05). However, with abs(logFC) > 0.58 filtering, I got 515 genes at the end.

• 0 views • link
written by asumani asumani

Log in to answer this question.

More posts like this