Projects
CAMIRA: A Flexible, Modular Platform to Accelerate Scientific Imaging
PI: Katherine L. Bouman and Yisong Yue (Division of Engineering and Applied Science)
SASE: Elena Williams, Scholar
Computational cameras offer a breakthrough by replacing traditional optics with computation, unlocking the ability to capture targets once thought impossible. Model-based imaging, which combines physics, machine learning, and optimization, leads to reliable and interpretable output images for computational cameras; however, these approaches have not been widely adopted in the sciences because they require significant time and expertise to adapt to new problems. Inspired by the success of deep learning software packages, the Bouman and Yue groups are building CAMIRA (Computational Architecture for Model-based Image Reconstruction Algorithms), a flexible, plug-and-play software framework that enables scientists to seamlessly apply advanced imaging methods on any physics-based computational imaging problem without deep computational expertise. This framework will democratize model-based imaging, enabling ongoing community contributions that will accelerate cross-disciplinary imaging and drive breakthroughs across diverse scientific and medical fields. Imaging is essential for scientific advancement, but traditional imaging systems are reaching their limits. For example, traditional telescopes cannot be constructed large enough to resolve a black hole, traditional microscopes are not able to see transparent cells, and traditional cameras cannot be used to study the inner core of a cloud due to scattering. “Computational cameras” have emerged as a solution, replacing optics with computation and enabling imaging processes previously impossible with conventional techniques. However, this paradigm shift in imaging presents new challenges, particularly in recovering images from severely limited, noisy data. This requires the incorporation of additional information into the imaging process in the form of statistical assumptions about the image structure or physical constraints.
In collaboration with the Software Academy, we are developing CAMIRA, a flexible, modular platform designed for model-based imaging. CAMIRA will allow scientists to easily integrate different modeling assumptions—such as modern diffusion-based priors or complex physical constraints—without the need for deep computational expertise. Unlike current deep learning frameworks, which lack interpretability and struggle with incorporating physical models, CAMIRA would enable researchers to seamlessly combine physical laws with data-driven methods, creating more robust and scientifically grounded solutions. The Bouman and Yue groups have already jointly taken initial steps by benchmarking recent plug-and-play diffusion methods for scientific inverse problems. However, transforming this concept into a practical platform for scientists requires substantial software engineering from the Academy. We are currently focusing on designing the core architecture and implementing well-established approaches within the framework, ensuring a robust foundation and immediate utility for researchers while enabling testing and evaluation. These approaches will include traditional inverse imaging techniques that use hand-crafted image regularizers, such as total variation or maximum entropy, as well as modern methods that leverage rich image priors defined by diffusion models, the backbone of recent advances in AI image generation.
Figure 1: A sample of applications where our Plug-and-Play Diffusion Model method has been successfully applied. While these results demonstrate its potential for generalization, each application required significant time and expertise to adapt to new problem setup.