WHAT IS EVENT BASED VISION?
EVENT-BASED VISION FUNCTIONS LIKE THE EYE AND THE BRAIN TO OVERCOME
INHERENT LIMITATIONS OF CONVENTIONAL MACHINE VISION
The human eye shares little with a conventional video camera.
Since their inception 150 years ago, all conventional video tools have represented motion by capturing a number of still frames each second. Displayed rapidly, such images create an illusion of continuous movement. From the flip book to the movie camera, the illusion became more convincing but its basic structure never changed.
For a computer, this representation of motion is of little use. The camera is blind between each frame, losing information on moving objects. Even when the camera is recording, each of its “snapshot” images contains no information about the motion of elements in the scene. Worse still, within each image, the same irrelevant background objects are repeatedly recorded, generating excessive unhelpful data.
Consider a video of a golfer taking a swing
Oversampled: Sky, grass and trees
Undersampled: Motion of golfer, club and ball
Consider a video of a golfer taking a swing. A conventional sensor applies an arbitrary frame rate to the whole scene, let’s say 30 frames per second. The important information is the swing of the club and the movement of the ball but ironically, the sensor will miss segments of this information while repeatedly taking an extensive inventory of the sky, trees and grass behind him.
NATURE’S ELEGANT SOLUTION
Evolution developed an elegant solution so that natural vision never encounters these problems. It doesn’t take frames. Cells in your eye report back to the brain when they detect a change in the scene – an event. If nothing changes, the cell doesn’t report anything. The more an object moves, the more your eye and brain sample it.
This process allows human vision to collect all the information it needs, without wasting time and energy reprocessing images of the unchanging parts of the scene.
By only recording what changes, the eye and brain can gather useful information from things changing at up to 1000 times a second, without needing to engage enormous amounts of brain power. Neither the predator nor the prey has time to waste processing irrelevant information.
This is event-based vision – independent receptors collecting all the essential information, and nothing else.
EVENT-BASED VISION SYSTEMS PERCEIVE THE VITALITY OF THE SCENE
AND OVERLOOK THE IRRELEVANT
PROPHESEE creates both neuromorphic sensors and bio-inspired algorithms that function like the eye and brain. This holistic approach is a fundamental shift in computer vision – the departure from frame-based sensors, to event-based vision systems, also known as event cameras.
Each pixel only reports when it senses movement. Whereas in a frame-based sensor all pixels record at the same time, in an event-based sensor each pixel is perfectly independent.
NOTHING IS LOST BETWEEN THE FRAMES
When each pixel is free to record only when it is triggered, the information created does not arrive frame by frame. Rather, movement is captured as a continuous stream of information. Nothing is lost between frames.
The Prophesee sensor records a rotating robotic arm as a continuous stream of movement.
A DISRUPTIVE TECHNOLOGY
Event-based vision systems perceive the vitality of the scene and overlook the irrelevant. They produce up to 1000 times fewer data than a conventional sensor whilst achieving a higher equivalent temporal resolution of >10 000 fps.
By bypassing inherent limitations in conventional computer vision, event-based vision is disrupting the current technology in fields such as automotive vehicles, artificial intelligence & deep learning, industrial automation, IoT, security, surveillance, and health-care among others.
