Haipeng Lin, Jiali Ou, Zhen Fan(樊贞), Xiaobing Yan(闫小兵), Wenjie Hu, Boyuan Cui, Jikang Xu, Wenjie Li, Zhiwei Chen, Biao Yang, Kun Liu, Linyuan Mo, Meixia Li, Xubing Lu, Guofu Zhou, Xingsen Gao(高兴森), and Jun-Ming Liu(刘俊明)
In situ training of an in-sensor artificial neural network based on ferroelectric photosensors
Nature Communications 16, 421(2025)
In-sensor computing has emerged as an ultrafast and low-power technique for next-generation machine vision. However, in situ training of in-sensor computing systems remains challenging due to the demands for both high-performance devices and efficient programming schemes. Here, we experimentally demonstrate the in situ training of an in-sensor artificial neural network (ANN) based on ferroelectric photosensors (FE-PSs). Our FE-PS exhibits self-powered, fast (<30 μs), and multilevel (>4 bits) photoresponses, as well as long retention (50 days), high endurance (109), high write speed (100 ns), and small cycle-to-cycle and device-to-device variations (~0.66% and ~2.72%, respectively), all of which are desirable for the in situ training. Additionally, a bi-directional closed-loop programming scheme is developed, achieving a precise and efficient weight update for the FE-PS. Using this programming scheme, an in-sensor ANN based on the FE-PSs is trained in situ to recognize traffic signs for commanding a prototype autonomous vehicle. Moreover, this in-sensor ANN operates 50 times faster than a von Neumann machine vision system. This study paves the way for the development of in-sensor computing systems with in situ training capability, which may find applications in new data-streaming machine vision tasks.
URL: https://doi.org/10.1038/s41467-024-55508-z