How cuffcompare calculates sensitivity and specificity?

Question

[scchess]

According to wikipedia, sensitivity and specificity are defined as:

However, in cuffcompare, the values are calculated as:

sp=(100.0*(double)ps->exonTP)/(ps->exonTP+ps->exonFP);
sn=(100.0*(double)ps->exonTP)/(ps->exonTP+ps->exonFN);

sn is sensitivity and sp is specificity.

The formulas don't match, in particular true-positive (exonTP) is now used in both formulas. Does anybody know why?

[Istvan Albert]

you should probably open a ticket with the developer

[scchess]

I found the answer. Please look at my answer.

Answer 1

[scchess]

That was explained in their paper.

In this context, we are measuring the accuracy at the base level. In the base level, we will have significantly more noncoding bases than coding bases. Therefore, TN will be much larger than FP. Therefore, SPC (specificity) will tend to be close to one and not very suitable as a metric. To fix the issue, cufflinks prefers another formula:

sp=(100.0*(double)ps->exonTP)/(ps->exonTP+ps->exonFP);

It is important to note that cufflinks is not not calculating specificity anymore. Instead, we are getting the prediction accuracy of the experiment. It is a technical mistake that cufflinks should fix. They should never claim it's specificity when it's not.

[Istvan Albert]

thanks for following up - it is quite interesting to know