“A 0.8V multimode vision sensor for motion and saliency detection with ping-pong PWM pixel” by the National Tsing Hua Univ., Taiwan.
Energy-efficient always-on motion detection (MD) sensors are in high demand and are widely used in machine vision applications. To achieve real-time and continuous motion monitoring, high speed low-power temporal difference imagers with corresponding processing architectures are widely investigated. Event-based dynamic vision sensors (DVS) have been reported to reduce the redundant data an power through the asynchronous timestamped event-address readout. But these sensors need special data processing to collect enough events for information extraction. Noise and dynamic effects can be issues as well. Frame-based MD rolling shutter sensors were reported to reduce the data bandwidth and power by sub-sampling operation, global shutter MD sensors were reported using in-pixel analog memory for reference image storage. In a frame-based MD sensor, the required analog processing circuit and two successive frames for temporal difference operation comes at a cost in power, area and speed. In this paper a frame-based MD vision sensor is presented, featuring three operation modes :
- Image capture,
- Frame-difference with on/off event detection,
- Saliency detection.
Using a low-voltage ping-pong PWM pixel and multi-mode operation, it achieves high-speed low-power full resolution motion detection, consecutive event frame reporting, and image capture features. Moreover, saliency detection by counting the block-level event number is also implemented for efficient optic flow extraction of the companion processing chip using simple neuromorphic circuits.
“A 1280×720 back-illuminated stacked temporal contrast event-based bision sensor with 4.86 um pixels, 1.066 GEPS readout, programmable event-rate controller and compressive data-formatting pipeline” by Prophesee and Sony.
Event-based (EB) vision sensors pixel-individually detect temporal contrast exceeding a preset relative threshold to follow the temporal evolution of relative light changes and to define sampling points for frame-free pixel-level measurement of absolute intensity. EB sensors gain popularity in high-speed low-power machine vision applications thanks to temporal precision of recorded data, inherent suppression of temporal redundancy resulting in reduced post-processing cost, and wide intra-scene dynamic range operation.
The heart of the pixel is a logarithmic responding photodiode, and every time the pixel exceeded a certain threshold in amplitude, the pixel detects “an event”. Events can be positive and negative. This concept is illustrated below.
The photodiode is partially pinned and the logarithmic response is realized by means of a subthreshold MOS based logarithmic photocurrent-to-voltage converter. The chip is making use of stacking technology with a per-pixel interconnect. The toplayer (90 nm BI CIS) consists of the photodiode plus 2 nMOS transistors, all other pixel circuitry (50 transistors) are located on the bottom layer (40 nm CMOS). Pixel pitch is 4.86 um with a fill factor of over 77 %. The overall power consumption of the chip depends on the number of events that are detected, e.g. at 100 kEPS the chip consumes 32 mW, at 300 MEPS, power consumption is equal to 73 mW.
The latter paper got a lot of attention, not just because of the device performance, but also because of the remarkable cooperation between the two companies.
Albert, 23 February 2020.