Summary:
UCLA researchers in the department of Electrical Engineering have developed a deep learning approach that is capable of performing image reconstruction from raw holograms, which proves to be faster and superior at reconstruction as well as generalizable to new data.
Background:
Holographic imaging provides significant advantages over standard phase contrast microscopy. In conventional microscopy, the sample is projected onto a sensor. The distance of the sample and the multiple optical parts determine which part of the sample is in focus at any time. This has significant drawbacks. The optics required are inherently complex and expensive, and only the information from a single two-dimensional region is captured at any time. In addition, standard microscopy often utilizes inaccurate autofocus to ensure that the area imaged is exactly what is desired.
In contrast, holographic imaging acquires the hologram of the entire thickness of the sample at once. This means that all the information for all focal distances are present in the data, removing the need for complex optics or expensive autofocus systems. Users also acquire all the three-dimensional information at once which is limited in speed only by the camera’s frame rate rather than the thickness of the sample and speed of conventional autofocus systems. While the image generated from standard microscopy is human-interpretable right away, the data collected from a hologram is not as clear. Fortunately, all of the information exists within the data and algorithms can be created to generate interpretable images from holograms. These algorithms should be capable of performing digital autofocus across the sample as well as correcting optical aberration and reconstruct the image, preserving the three-dimensional information in the case of tomography. While there are some statistical approaches for holographic image reconstruction, the majority of the success has been in deep learning approaches. Consistent of most machine learning approaches, these algorithms are typically only capable of reconstructing samples that they were trained on, leading to the inability to generalize to new data. There is a clear and pressing need to improve these algorithms in speed, accuracy, and generalizability to new samples before holographic imaging can truly begin to be useful in replacing conventional phase contrast microscopy.
Innovation:
Researchers at UCLA have developed a deep learning-based Fourier transform-assisted holographic image reconstruction algorithm that is far superior than other existing options. In terms of speed the algorithm is capable of reconstructing images of 1 mm2 in ~0.04 s which is near the frame rate of the camera. Reconstructed images using this algorithm prove to be accurate when compared to phase contrast microscopy images collected on the same samples. The algorithm also has shown to be generalizable to vastly different sample types which were not used for initial training.
Potential Applications:
• Computational Microscopy
• Inspection of materials
• Detection of Counterfeiting
• Image Reconstruction
• Holographic Imaging
• Holographic Tomography
• Fluorescence and brightfield compatible
Advantages:
• >10X Faster (approximately same speed as sample collection)
• Ability to parallelize to increase speed
• Accurate compared to phase contrast images
• Generalizable to other sample types
Development-To-Date:
Fourier Imager Network (FIN) has been successfully demonstrated on a variety of sample types that were not part of the initial training.