Karl is right. When you have very few samples (few residual degrees of freedom) the information to estimate the dispersion is small, so the final dispersion estimates are often close to the parametric line, which is a function of the mean of all normalized samples. The standard errors are calculated using the dispersions, so this dependence on mean will also be seen in the standard errors. Because the groups are unbalanced, one of the groups is contributing more to the mean of normalized samples than the others. Also, at very small mean, the LFC and standard errors start looking like a function of the mean, as the LFC shrinkage increases at low counts. If you color the SE plot with something like col=ifelse(res$baseMean > 5,"black","red"), you will see that the curve of points is just genes which have very small counts, and the curve is just an artifact of these properties mentioned above. These genes have very small power for detection of DE so it's not a problem for inference.
Although with really unbalanced design, it might make sense to not use a parametric function of the mean of normalized counts from all samples (fitType="parametric"), but instead just use a common value across all genes to set the dispersion prior:
DESeq(dds, fitType="mean")
Then the dispersion estimation is not affected by the mean of normalized counts.



How many samples are in each timepoint? When you say "few samples" in each group, is that 1 or five?