I work with the MACSima cyclic immunofluorescence platform (Miltenyi Biotec), and kept running into the same issue: automated segmentation (MACSiQView, or Cellpose/StarDist masks) is fast, but there was no quantitative way to check how good it actually was versus a manual ground truth beyond visually comparing overlays.

I tried putting together a small Python package to address this:

• Compares automated segmentation(s) against a manual Fiji/ImageJ ROI reference using the Kolmogorov–Smirnov distance on morphological features (area, perimeter, Feret diameter, mean intensity), to rank which segmentation method/parameters are closest to ground truth.
• For QC at scale (when a manual reference isn't available per image), trains an Isolation Forest on a reference cell population to flag morphological outliers in new segmentation runs, with a sensitivity analysis on the contamination parameter.
• Runs Mann-Whitney U tests between flagged "OK" vs "KO" cells to suggest which acquisition parameters might be worth adjusting.

GitHub: https://github.com/mathisbouvet/macsima-qc PyPI: pip install macsima-qc

It's still early (v0.1.x) and built around my own MACSima workflow, so I'm not sure yet how well it generalizes to other segmentation/imaging setups. Posting here mainly to get feedback has anyone tackled segmentation validation this way before, or is there a more established approach I should be aware of?