This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Which correlation test better for small ordinal data

Hi all,

I would appreciate some advice on which correlation test would be most appropriate for my data. I have a continuous measure of cell abundance and an ordinal variable with three ordered categories (coded as 0, 1, and 2).

From what I understand, non-parametric methods such as Spearman’s rank correlation or Kendall’s tau could be suitable options. However, given that my statistics background is limited, I am unsure which approach would be most appropriate and easiest to justify in this case. Or an other method is more suitable?

My dataset is relatively small, with approximately >10 cases and >15 controls. Really appreciate any guidance.

Thanks

ordinal correlation data test
• 1,530 views • link
updated by kalavattam • kalavattam • written by MS • MS

1 answer

If I understand correctly, the question is, “Is there a monotonic trend in the continuous measure as the group categories increase?”

In that case, Spearman can be a reasonable first choice, but Kendall—specifically, tau-b—may be better in this case, because of how it handles ties on the ordinal variables (since there are only three ordered categories, there are likely to be many ties; Kendall tau-a assumes no ties). Without going into details, how Spearman handles ties means it may be less stable than Kendall tau-b, particularly because of the small n size. The Kendall tau-b test results can be interpreted as how often a higher group category (ordinal level) corresponds to higher continuous values.

If you just want to look at any differences across the three groups (and not necessarily an ordered trend), then Kruskal-Wallis can be good here. A quick search and double-check with an LLM tells me Jonckheere-Terpstra is another rank-based test specifically for ordered groups (but I have no experience with it, although it seems to be recommended for this setup).

• 0 views • link
written by kalavattam • kalavattam

Thanks, thats helpful. I have used Kruskal-Wallis to check differences and results are consistent with Spearman and Kendall. Question you stated is right, only calrification is categories were not in ordered, instead more like "present", "middle" and "absent" so I gave them code 0, 1 2 (is Kendall still okay in that case?). Also, I def can't use linear regression but what about oridnal logistic to check relation between categories and abundance? Corelation is more resoanable or regression to answer the question you stated above?

• 0 views • link
written by MS • MS

If there is a defined and defensible order (e.g., presumably “absent < middle < present”), then Kendall tau-b is fine, and coding 0, 1, and 2 makes sense.

If your categories are not ordered, then I’d probably stick with Kruskal–Wallis and run post-hoc pairwise tests such as Dunn (with multiple-testing correction, e.g., Benjamini-Hochberg). In that case, using Kendall or Spearman for testing a “trend” is not appropriate since you’d be imposing an order. The pairwise tests are, IMO, what people would want to see after KW anyway.

Regarding regression, it’s not that you “definitely” can’t use it here, it’s more that with (i) a small n, (ii) potential skew/heavy tails or heteroscedasticity, the usual assumptions may not hold well enough that I’d trust the stats without some checks. That said, a simple linear model with the 3-level variable treated as a factor (dummy-coded groups) is basically a parametric analog of “group differences.” My understanding is that it can be OK if residuals aren’t crazy and/or if you use robust standard errors. Also, if the continuous variable is very skewed, a log transform could be appropriate.

Regarding ordinal logistic, that’s appropriate if the (i) categories truly are ordered, (ii) you want to predict category from abundance, and (iii) the proportional-odds assumption is met. But that answers a slightly different question than “do abundances differ by group?”

• 0 views • link
written by kalavattam • kalavattam

Thanks very much for detailed response. Thtat's really helpful.

• 0 views • link
written by MS • MS

Hi @kalavattam, thanks for your help earlier. I have one more confusion to discuss.

I have some data like this

Condition, dependent variable, independent variable

A 1,2,3,4,5,5,6... x and y

B 1,2,3,4,5,5,6... x and y

C 1,2,3,4,5,5,6... x and y

D 1,2,3,4,5,5,6... x and y

..... each condition for three times

For this data, I am interested in the difference between to categoreis x and y for each condition testing for each time separately (not repeated measure) . According to my understanding, I can use an independent t-test, one way anova with two factors or linear regression. I used the lm model in R, which is a linear regression method, but tutorials say it can also be called one-way anova. So am I right if I report results as one way anova?

Next, since I have the same condition for three times so for multiple testing correction options I have is either estimated marginal mean (emm) followed by Pairwise comparisons with FDR adjustment. Or direct per condition FDR correction across all time. I get more or less similar results with both, but not sure which one is most reasonable to justify or if I am missing anything major here. My stats knowledge is mostly from tutorials and vidoes so maybe I am completely wrong. Would really appreciate any guidance.

• 0 views • link
written by MS • MS

When they’re set up the same way, t-test, ANOVA, and lm are all the same model here. (There was a popular writeup discussing this from several years back, but I cannot seem to find it to link to it here. You can use a search engine for, e.g., “github t-test linear regression”. Actually, here it is.)

Also, a quick thing: you allude to a one-way ANOVA with two factors, but “one-way” means one factor. Since you have more than one factor (e.g., condition, time, group), then it’s a multi-factor ANOVA (or a linear model).

So, do you want the x vs y difference within each condition at each timepoint? If so, you could try this:

## R scripting ##

library(emmeans)

#  Fit the model
fit <- lm(outcome ~ condition * time * group, data=df)

#  Get x vs y within each condition by time
emm <- emmeans(fit, ~ group | condition * time)
contr <- contrast(emm, method="pairwise")

#  Adjust across all those cell-wise contrasts
summary(contr, adjust="BH")

emmeans lets you ask the fitted model for the estimated mean outcome in x and y within each condition by time, and then it tests the x–y difference in each cell; the p-value adjustment (BH = Benjamini-Hochberg FDR procedure) accounts for the fact you’re doing many of those cell-wise comparisons.

Disclosure: I used an LLM, ChatGPT 5.2-Thinking, to quickly generate the code (untested) and for the suggestion for a package to use for this interaction analysis based on your post, emmeans.

For emmeans, more details on interaction analysis here and comparisons and contrasts here. (Not sure if these are up-to-date.)

I get more or less similar results with both, but not sure which one is most reasonable to justify or if I am missing anything major here.

Getting similar results from both approaches is promising. If I had to pick, I’d go with the above: fitting a single lm(), getting emmeans x–y contrasts (for each condition by time), then doing FDR adjustment across the contrasts (b/c that’s what you’re reporting).

• 0 views • link
written by kalavattam • kalavattam

Thank you, link is really helpful.I structured my data more properly now, looks like this

Class   Time    value   group

A   1st 11  X

A   1st 22  Y

A   1st 12  X

A   2nd 11  X

A   2nd 22  X

A   2nd 12  Y

A   3rd 22  Y

A   3rd 13  X

A   3rd 14  X

B... and so on.

I hope it makes more sense. I get your point, but since data is not repeated measure so I was doing this

#  Fit the model
fit <- lm(Value ~ group, data=df)

looping over each Time and dealing with each class at a time.So it make it one way anova as there is one factor (group) with two categories (x, y) do you think its right approach? Then within the same loop I am doing emmeans, which means I am adjusting for one Class and one Time only at a time.

#  Adjust across all those cell-wise contrasts
summary(contr, adjust="BH")

Is it okay?

and last thing, so even with two categories, can I report my results as one-way ANOVA (using lm) for reviewers? (although it requires >2 categories) Or should I report it as independent t-test? Thanks

• 0 views • link
updated by GenoMax GenoMax • written by MS • MS

Fit the model

fit <- lm(Value ~ group, data=df)

looping over each Time and dealing with each class at a time.So it make it one way anova as there is one factor (group) with two categories (x, y) do you think its right approach? Then within the same loop I am doing emmeans, which means I am adjusting for one Class and one Time only at a time.

If I understand correctly, you’re doing the BH adjustment inside each loop. That means it isn’t actually adjusting across all the tests (the whole family). If you want FDR across all Class by Time x–y comparisons, I’d fit one model like lm(value ~ class * time * group), run emmeans once, and then run BH procedure once across all contrasts.

and last thing, so even with two categories, can I report my results as one-way ANOVA (using lm) for reviewers? (although it requires >2 categories) Or should I report it as independent t-test? Thanks

With two groups, an independent two-sample t-test and a one-way ANOVA are the same test (b/c F = t^2 when they’re set up the same way). So, I think either label is fine. Just be clear when describing this in your methods section and other appropriate parts of the text. I’d probably write something like, “...two-sample comparisons by linear model (which is equivalent to an independent two-sample t-test) with Benjamini-Hochberg FDR adjustments across Class by Time contrasts.”

• 0 views • link
written by kalavattam • kalavattam

Thank you, Kalavattam. This answered my many questions.

• 0 views • link
written by MS • MS

Hi Kalavattam,

Thanks for the help earlier. I need some guidance on a similar issue. In the above data, I now have only one Time. So, I am running tests between each group (X vs Y) for each class (different cell types) in a loop. Then merge all results and check FDR (for all cell types together). Which appearanlty looks a right approach, but I am a bit skeptical in terms of 18 cell types (a lot of variability). Specially it shows an obvious difference for one cell type between X and Y with pval 0.007 but adj p val become drastically insignificant when adjusted across all cell types. So am I missing anything here or it is what it is? Thanks for your guidance

• 0 views • link
written by MS • MS

Hi MS — what you describe can be completely normal. This is often just how Benjamini-Hochberg false discovery rate (BH FDR) adjustment works. If interested, see this StatQuest video for a nice, straightforward breakdown of the method.

If you are testing X vs Y separately in each of 18 cell types and then adjusting those p-values together, the adjusted p-values reflect the fact that you asked 18 related questions, not just 1. That means a raw p-value of 0.007 can become non-significant after BH FDR correction, especially if the other tests are “weaker” (i.e., are higher p-values). That is not necessarily a sign that anything is wrong; it is just the expected consequence of multiple-testing correction.

It might be helpful to step back and reexamine your scientific question: is it really “within each cell type, is X different from Y?” That is, are all 18 cell types part of the same analysis question? If so, then adjusting across those 18 cell-type-wise tests is reasonable.

But if not (e.g., one cell type was the primary hypothesis from the beginning and the others were more exploratory [or something along those lines]), then that can change how the testing strategy should be framed.

Ideally, that question should be answered before reaching this step. One really needs to be careful not to define the testing family after seeing which raw p-values look “promising.”

• 0 views • link
written by kalavattam • kalavattam

One really needs to be careful not to define the testing family after seeing which raw p-values look “promising.”

This is a form post hoc inference and is something that should be avoided, since it can drift into “p-hacking” (in this case, changing the testing family in a way that favors a smaller adjusted p-value).

• 0 views • link
written by kalavattam • kalavattam

Log in to answer this question.

More posts like this