That's what I thought as well, but it's happening none the less which is why I'm confused. I don't think this is a consequence of small read depths because I have more 140-4k reads per sample for this gene and library sizes are relatively consistent.
I know that v$E here doesn't correspond directly to the lmFit results and normally I wouldn't plot it. That's how I discovered this was happening when my results from lmFit disagreed in sign with the log2(cpm) results that I was plotting for them.
Thanks for your offer to take a look! I'm e-mailing you a concise + clean copy of the code and the data needed to reproduce the effect.
Cross-posted to Bioconductor https://support.bioconductor.org/p/9139613/