A foundation model for multi-task cross-distribution restoration of fluorescence microscopy images

Dataemia
11 Min Read


  • Schermelleh, L. et al. Super-resolution microscopy demystified. Nat. Cell Biol. 21, 72–84 (2019).


    Google Scholar
     

  • Scherf, N. & Huisken, J. The smart and gentle microscope. Nat. Biotechnol. 33, 815–818 (2015).


    Google Scholar
     

  • Goodwin, P. C. Quantitative deconvolution microscopy. in Methods in Cell Biology vol. 123 177–192 (Elsevier, 2014).

  • Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).


    Google Scholar
     

  • Chen, J. et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat. Methods 18, 678–687 (2021).


    Google Scholar
     

  • Li, Y. et al. Incorporating the image formation process into deep learning improves network performance. Nat. Methods 19, 1427–1437 (2022).


    Google Scholar
     

  • Qiao, C. et al. Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy. Nat. Commun. 15, 4180 (2024).


    Google Scholar
     

  • Qiao, C. et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat. Methods 18, 194–202 (2021).


    Google Scholar
     

  • Chen, R. et al. Single-frame deep-learning super-resolution microscopy for intracellular dynamics imaging. Nat. Commun. 14, 2854 (2023).


    Google Scholar
     

  • Pawley, J. B. Handbook of Biological Confocal Microscopy. Springer: New York, NY, 2006.


    Google Scholar
     

  • Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).


    Google Scholar
     

  • Kondepudi, A. et al. Foundation models for fast, label-free detection of glioma infiltration. Nature 637, 439–445 (2025).


    Google Scholar
     

  • Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259–265 (2023).


    Google Scholar
     

  • Wang, X. et al. A pathology foundation model for cancer diagnosis and prognosis prediction. Nature 634, 970–978 (2024).


    Google Scholar
     

  • Xiang, J. et al. A vision–language foundation model for precision oncology. Nature 638, 769–778 (2025).


    Google Scholar
     

  • Xu, H. et al. A whole-slide foundation model for digital pathology from real-world data. Nature 630, 181–188 (2024).


    Google Scholar
     

  • Zhou, Y. et al. A foundation model for generalizable disease detection from retinal images. Nature 622, 156–163 (2023).


    Google Scholar
     

  • Sun, Y., Wang, L., Li, G., Lin, W. & Wang, L. A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks. Nat. Biomed. Eng. 9, 521–538 (2025).


    Google Scholar
     

  • Ma, C., Tan, W., He, R. & Yan, B. Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration. Nat. Methods 21, 1558–1567 (2024).


    Google Scholar
     

  • Jiao, J. et al. USFM: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis. Med. Image Anal. 96, 103202 (2024).


    Google Scholar
     

  • Yan, Q. et al. Textual prompt guided image restoration. Eng. Appl. Artif. Intell. 155, 110981 (2025).

  • Bai, Y. et al. TextIR: A Simple Framework for Text-based Editable Image Restoration. IEEE Trans. Vis. Comput. Graph. 1–16 (2025) https://doi.org/10.1109/TVCG.2025.3550844.

  • Ronneberger, O., Fischer, P. & Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 (eds Navab, N., Hornegger, J., Wells, W. M. & Frangi, A. F.) vol. 9351 234–241 (Springer International Publishing, 2015).

  • Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-Resolution Image Synthesis with Latent Diffusion Models. in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10674–10685 (IEEE, 2022). https://doi.org/10.1109/CVPR52688.2022.01042.

  • Vaswani, A. et al. Attention is All you Need. In Advances in Neural Information Processing Systems (eds Guyon, I. et al.). Vol. 30 (Curran Associates, Inc., 2017).

  • Zhang, S. et al. A Multimodal Biomedical Foundation Model Trained from Fifteen Million Image–Text Pairs. NEJM AI 2, (2025).

  • Reed, S. et al. Generative Adversarial Text to Image Synthesis. In Proceedings of The 33rd International Conference on Machine Learning 1060–1069 (PMLR, 2016).

  • Bai, Y. et al. TextIR: A Simple Framework for Text-based Editable Image Restoration. IEEE Trans. Visual. Comput. Graphics 1–16 (2025) https://doi.org/10.1109/TVCG.2025.3550844.

  • Lu, Y. et al. Priors in Deep Image Restoration and Enhancement: A Survey. Preprint at https://doi.org/10.48550/arXiv.2206.02070 (2023).

  • Theodoris, C. V. et al. Transfer learning enables predictions in network biology. Nature 618, 616–624 (2023).


    Google Scholar
     

  • Liao, T. et al. A super-resolution strategy for mass spectrometry imaging via transfer learning. Nat. Mach. Intell. 5, 656–668 (2023).


    Google Scholar
     

  • Boutros, M., Heigwer, F. & Laufer, C. Microscopy-based high-content screening. Cell 163, 1314–1325 (2015).


    Google Scholar
     

  • Pepperkok, R. & Ellenberg, J. High-throughput fluorescence microscopy for systems biology. Nat. Rev. Mol. Cell Biol. 7, 690–696 (2006).


    Google Scholar
     

  • Pachitariu, M., Rariden, M. & Stringer, C. Cellpose-SAM: superhuman generalization for cellular segmentation. Preprint at https://doi.org/10.1101/2025.04.28.651001 (2025).

  • Lefebvre, A. E. Y. T. et al. Nellie: automated organelle segmentation, tracking and hierarchical feature extraction in 2D/3D live-cell microscopy. Nat. Methods 22, 751–763 (2025).


    Google Scholar
     

  • Sofroniew, N. et al. Napari: a multi-dimensional image viewer for Python. Zenodo https://doi.org/10.5281/zenodo.12722145 (2024).

  • Guo, M. et al. Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy. Nat. Commun. 16, 313 (2025).


    Google Scholar
     

  • Verveer, P. J. Advanced Fluorescence Microscopy: Methods and Protocols. vol. 1251 (Springer New York, New York, NY, 2015).

  • Esser, P. et al. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. In Proceedings of the 41st International Conference on Machine Learning 12606–12633 (PMLR, 2024).

  • Video generation models as world simulators. https://openai.com/index/video-generation-models-as-world-simulators/ (2024).

  • Liang, J. et al. SwinIR: Image Restoration Using Swin Transformer. In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) 1833–1844 (IEEE, Montreal, BC, Canada, 2021). doi:10.1109/ICCVW54120.2021.00210.

  • Zamir, S. W. et al. Restormer: Efficient Transformer for High-Resolution Image Restoration. in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 5718–5729 (IEEE, 2022). https://doi.org/10.1109/CVPR52688.2022.00564.

  • Tay, Y., Dehghani, M., Bahri, D. & Metzler, D. Efficient Transformers: A Survey. ACM Comput. Surv. 55, 1–28 (2023).

  • Zhou, R. et al. W2S: Microscopy Data with Joint Denoising and Super-Resolution for Widefield to SIM Mapping. in Computer Vision–ECCV 2020 Workshops (eds Bartoli, A. & Fusiello, A.) vol. 12535 474–491 (Springer International Publishing, 2020).

  • Zhang, Y. et al. A Poisson-Gaussian Denoising Dataset With Real Fluorescence Microscopy Images. in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 11702–11710 (IEEE, 2019). https://doi.org/10.1109/CVPR.2019.01198.

  • Yao, K. et al. Scaffold-A549: a benchmark 3D fluorescence image dataset for unsupervised nuclei segmentation. Cogn. Comput. 13, 1603–1608 (2021).


    Google Scholar
     

  • Spahn, C. et al. DeepBacs for multi-task bacterial image analysis using open-source deep learning approaches. Commun. Biol. 5, 688 (2022).


    Google Scholar
     

  • Belharbi, S. et al. SR-CACO-2: a dataset for confocal fluorescence microscopy image super-resolution. In Proceedings of the 38th International Conference on Neural Information Processing Systems vol. 37, 59948–59983 (Curran Associates Inc., Red Hook, NY, USA, 2024).

  • Markwirth, A. et al. Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction. Nat. Commun. 10, 4315 (2019).


    Google Scholar
     

  • Svoboda, D., Homola, O. & Stejskal, S. Generation of 3D Digital Phantoms of Colon Tissue. In Image Analysis and Recognition (eds Kamel, M. & Campilho, A.) vol. 6754, 31–39 (Springer Berlin Heidelberg, 2011).

  • Svoboda, D., Kozubek, M. & Stejskal, S. Generation of digital phantoms of cell nuclei and simulation of image formation in 3D image cytometry. Cytometry A. 75A, 494–509 (2009).


    Google Scholar
     

  • Loshchilov, I. & Hutter, F. Decoupled Weight Decay Regularization. in International Conference on Learning Representations (Open Review.net, 2019).

  • Van Der Walt, S. et al. scikit-image: image processing in Python. PeerJ. 2, e453 (2014).


    Google Scholar
     

  • Wang, Z., Simoncelli, E. P. & Bovik, A. C. Multiscale structural similarity for image quality assessment. in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 1398–1402 (IEEE, 2003). https://doi.org/10.1109/ACSSC.2003.1292216.

  • Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004).


    Google Scholar
     

  • Guo, M. et al. Rapid image deconvolution and multiview fusion for optical microscopy. Nat. Biotechnol. 38, 1337–1346 (2020).


    Google Scholar
     

  • Culley, S. et al. Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nat. Methods 15, 263–266 (2018).


    Google Scholar
     

  • Fang, L. et al. Deep learning-based point-scanning super-resolution imaging. Nat. Methods 18, 406–416 (2021).


    Google Scholar
     

  • Saraiva, B. M. et al. Efficiently accelerated bioimage analysis with NanoPyx, a Liquid Engine-powered Python framework. Nat. Methods 22, 283–286 (2025).


    Google Scholar
     

  • Descloux, A., Grußmayer, K. S. & Radenovic, A. Parameter-free image resolution estimation based on decorrelation analysis. Nat. Methods 16, 918–924 (2019).


    Google Scholar
     

  • Fang, S. et al. Resolution assessment of super-resolution microscopy imaging: structural and technical dependencies for cell biology. Cytotechnology 77, 170 (2025).


    Google Scholar
     

  • Stringer, C. & Pachitariu, M. Cellpose3: one-click image restoration for improved cellular segmentation. Nat. Methods 22, 592–599 (2025).


    Google Scholar
     

  • Zhang, Z., Nishimura, Y. & Kanchanawong, P. Extracting microtubule networks from superresolution single-molecule localization microscopy data. Mol. Biol. Cell 28, 333–345 (2017).


    Google Scholar
     

  • Lu, Q., Liu, X., Feng, Q., Zeng, S. & Cheng, S. A foundation model for multi-task cross-distribution restoration of fluorescence microscopy image. Zenodo https://doi.org/10.5281/zenodo.18383925 (2026).



  • Source link

    Share This Article
    Leave a Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    error: Content is protected !!