Spectral machine vision traditionally acquires dense 3D hypercubes (x, y, λ) of spatial and spectral information, which must be processed downstream. This imposes major computational burdens, tradeoffs in spatial/spectral resolution, frame rate, and power. The innovation embodied in spectral kernel machines (SKM) is a photodetector architecture that learns tasks in situ: instead of outputting raw spectra, the device’s photocurrent encodes the classification or identification result directly. By combining electrically tunable detection layers (e.g. bipolar black phosphorus–MoSâ‚‚ photodiodes for mid-IR, silicon photoconductors for visible) with internal kernel-machine–style processing, the SKM compresses spectral information and performs intelligent inference at the sensor level.
Ultra-low power: Projected up to 1,000× lower power consumption than conventional hyperspectral imaging + digital processing pipelines
High speed: Estimated ~100× faster processing in hyperspectral task throughput versus existing solutions
Compact, integrated inference: Shifts computational load into the photodetector itself, reducing data transfer, latency, and backend compute requirements.
Flexible spectral bands: Demonstrated in both visible and mid-IR (MIR) bands, enabling a wide range of sensing / spectral tasks.
Real-time “sniff-and-seek” operation: Learns from examples and classifies unknown samples with minimal data overhead.
Precision agriculture (e.g. plant health, nutrient / moisture sensing)
Waste sorting and recycling (material discrimination)
Food quality & safety inspection
Pharmaceuticals and chemometrics
Semiconductor wafer metrology and inspection
Satellite, drone, or robotics spectral sensing in constrained power/latency environments
Onboard hyperspectral machine vision in mobile or portable systems
Proof-of-concept devices have been experimentally built (visible and MIR bands) and tested on real spectral classification and metrology tasks. Simulations and benchmark calculations suggest the performance advantages in power and speed are significant. Further development is needed for robustness, scaling, packaging, calibration, and system-level integration.
Inventors: Dehui Zhang, Yuhang Li, Jamie Geng, Hyong Min Kim, et al. (with Aydogan Ozcan, Ali Javey)