Using Diffusion Models to do Data Assimilation

Abstract The recent surge in machine learning (ML) methods for geophysical modeling has raised the question of how these methods might be applied to data assimilation (DA). We focus on diffusion modeling (a form of generative artificial intelligence) for systems that can perform the entire DA process, rather than on ML-based tools used within a conventional DA system. We identify at least three distinct types of diffusion-based DA systems and show that they differ in the posterior distribution they target for sampling. These posterior distributions correspond to different priors and/or likelihoods, which in turn result in unique training datasets, computational requirements, and state estimate qualities. Our analysis further shows that a diffusion DA system designed to target the same posterior distribution as current ensemble DA algorithms requires re-training at each DA cycle, which is computationally costly. We discuss the implications of these findings for the use of diffusion modeling in DA.