Bridging the Real-Synthetic Gap in Microscopy with Inversion-Based Diffusion 

Accurate cell counting in fluorescence microscopy images is a fundamental task in biomedical research and clinical diagnostics, supporting applications such as cancer monitoring and stem cell therapy. However, the effectiveness of deep learning approaches for automated cell counting is often limited by the scarcity of large, well-annotated microscopy datasets. One promising strategy to address this limitation is the generation of synthetic training data. Yet, a significant performance gap persists between models trained on synthetic data and those evaluated on real images, primarily due to differences in visual appearance and structural complexity—a challenge known as the domain gap.

This thesis proposes a novel Sim2Real framework based on Inversion-Based Style Transfer (InST) with diffusion models to bridge the real-synthetic gap in microscopy. Specifically, the method leverages latent-space Adaptive Instance Normalization (AdaIN) and stochastic inversion within a diffusion-based generative model to transfer the visual style of real microscopy images onto synthetically generated images, while weakly preserving the underlying cell structure. This approach enables the synthesis of realistic, structure-aware microscopy images that can supplement limited annotated data for model training.

Comprehensive experiments were conducted to evaluate the effectiveness of the proposed InST-based framework for downstream cell counting tasks. By pre-training and fine-tuning EcientNet-B0 models on various data sources—including real images, hard-coded synthetic data, and the publicly available Cell200-s dataset—our results demonstrate that models trained with InST-synthesized images achieve up to a 37% reduction in Mean Absolute Error (MAE) compared to models trained on hard-coded synthetic data, and a 52% reduction compared to Cell200-s. Remarkably, this approach also outperforms models trained solely on real data. Further improvements were achieved when combining InST-synthesized data with lightweight domain adaptation techniques such as DACS with CutMix.

The findings of this thesis provide strong evidence that InST-based style transfer can substantially reduce the domain gap between synthetic and real microscopy data, offering a scalable and effective pathway for improving automated cell counting performance while minimizing the need for manual annotation.

Committee: Wallapak Tavanapong (major professor), Ali Jannesari and Qi Li