Event cameras represent a shift toward next generation performance and efficiency for computer vision

Neuromorphic engineering is a type of electronic design and implementation that mimics biology. The fundamental premise is that biology has evolved over time to make us an extremely efficient and adaptable species, so taking cues from those traits and applying them to machines holds vast potential to improve their performance, power consumption and flexibility.

Prophesee focuses on how neuromorphic technology can be used for vision – basing its research and development on how the brain and eyes work together to allow us to see and process visual information in an optimal way. We call the approach Event-Based Vision and it is based on the fact that, like the human eye, machines can operate more effectively if they only process changes in a scene instead of processing everything in a scene all the time. The result is far better performance, less bandwidth required, and much improved energy efficiency. It also lends itself to enhanced operational characteristics in challenging lighting conditions.

This development has given rise to the event camera, a paradigm shift from the traditional (frame-based) camera. Event cameras can deliver new levels of performance and efficiency in a wide range of applications and use cases, from autonomous mobility, to ADAS applications, to industrial automation, AR/VR systems and mobile devices.

How event cameras work?

In a conventional camera an entire image (i.e. the light intensity at each pixel) is recorded at a pre-assigned interval, called the frame rate. While this works well in representing the ‘real world’ when displayed on a screen, recording the entire image every time increment oversamples all the parts of the image that have not changed. With frame-based cameras, this means that lots of unnecessary data is acquired, especially at high frame rates, whereas the most dynamic part of the image is under-sampled. The human eye (on which Prophesee’s technology is based) samples changes at up to 1000 times per second but does not record the primarily static background at such high frequencies.

Event cameras are a biologically inspired solution to this under and over-sampling. In an event camera, each pixel of the Event-Based Vision sensor works independently (i.e., asynchronously). Each pixel reports when it detects a relative change in the illumination intensity that is above or below a defined percentage of the previous intensity. An added benefit of this asynchronous approach is a much-enhanced dynamic range, since the pixel sensitivity does not have to be set globally in the event camera. As such, all the dynamic information is recorded but not the unnecessary static information such as background landscapes or sky.

This makes them extremely useful for scenes where this is motion – think: high-speed counting, or driver safety systems.