Ultra-high density and nonvolatile nanophotonic convolution processing unit

Photonic convolution processing units are the core components of photonic neural networks (PNNs). Traditionally, for an N × N photonic convolution processing unit, it typically requires O(N2) or O(Nlog2N) cascaded photonic devices, which leads to larger sizes and lower density. Therefore, nanophotonic neural networks (N-PNN) with high density are highly attractive. In this work, we propose an ultra-high density and nonvolatile N-PNN scheme, requiring only a compact photonic device to represent all weight values in the N × N convolution kernel, which results in a computational density of more than 5 petaflop operations per second per square millimeter (POPS/mm2) under ideal conditions. The footprints of the proposed 2 × 2 and 3 × 3 nanophotonic convolution processing units are only 4 × 4 μm2 and 6 × 6 μm2, respectively. Moreover, based on these two types of units, the N-PNN achieves image classification and recognition capabilities comparable to traditional computers. In addition, Owing to its lower insertion loss, our approach holds great significance for the scalability and on-chip integration of large-scale N-PNNs. This facilitated the integration of N-PNNs with existing electronic systems.