I will analyze the dataset with 5 samples (2 samples in group1, 2 samples in group2 and 1 sample in group3). Due to having no replicates in group3, I'm confused that if I should use a certain dispersion value in edgeR which is described in its manual, or I should use DESeq2 normally without any change about this issue. I have run DESeq2 and I had so many differentially expressed genes, and such high number of DEGs didn't make sense to me. Results of edgeR made more sense, but I couldn't be sure that I already have 2 replicates in 2 groups and defining a dispersion value is a right for this case.
2 answers
It is just not recommended to try an analysis with just 1 replicate. If you added a second replicate to group3, then the results could entirely change; if you added a third replicate, they may then change back. I am sure that DESeq2 will now issue a warning if you try an analysis with just 1 sample in a particular group. Even 2 replicates in your group1 and group2 is not ideal. The bare minimum should be 3, but better to have 5.
The EdgeR user guide has a section that shows one how to perform a differential expression analysis using just 1 versus 1, but I am not sure what that actually means, biologically. The best that one can do in such a situation is to think of ratios of expression in one sample over the other. The results could entirely filp with the addition of more replicates, though.
You may consider excluding group3 or trying to add another replicate to that group. You could also try merging group3 with 1 or 2, but this may make no sense for the disease / condition that you're studying.
Look at it this way: if you attempted to publish your work, you would definitely be asked to increase your sample size (depending on the journal, of course).
Kevin
Log in to answer this question.