I'm trying to run Differential Transcript analysis on some single cell Kallisto data using Sleuth. Unfortunately, the cell types are really unbalanced. I have 32 samples from 1 cell type, and 6 from the other. I have found that Sleuth uses a filter function that removes transcripts in <47% of samples. Since the dataset I have is very unbalanced, I tried to replace it with the following filtering function:
custom_filter <- function(row, min_reads = 5, min_prop = 0.2) { mean(row >= min_reads) >= min_prop }
Note that this still requires a transcript to be expressed in at least 8 samples, meaning that any transcript expressed in only the smaller cell type (even if expressed in 100% of samples therein), would be filtered out. Even so, I get the following error:
Error in sleuth_results(so, "reduced:full", "lrt", show_all = TRUE): The provided weighting function for the mean observations results in negative values, which are not allowed for the lancaster method.
Is it possible to use Sleuth with such an unbalanced sample distribution? Or should I try using a different program. I'm currently looking at Kallisto -> Tximport -> DESeq2 as a possible alternative pipeline, but I would appreciate any better recommendations as well
