[info] physorg, strebgreebling: visual processing on the chip can lead to some pretty impressive demos

[Alejandro Dubrovsky]

(
http://cordis.europa.eu/ictresults/index.cfm/section/news/tpl/article/BrowsingType/Features/ID/89794
videos:
http://www.robokeeper.com/index.php?id=16
http://thomaspfeifer.net/robokeeper.htm
)


Vision sensors keep their eye on the ball at Euro 2008
A few of the Euro 2008 teams may wish they had a Robogoalie. Image: ©
Caviar project
The silky skills of Europe’s top footballers will not be the only eye
catcher this summer as the movement of the fans themselves fall under
the watchful gaze of state-of-the-art video surveillance.

While entertainment is high on the agenda at the event, just as
important is the safety and security of the hundreds of thousands of
visitors who will be visiting the host nations of Switzerland and
Austria.

Applying the DVS technology developed by European researchers will play
a big part in ensuring the safety of the fans as they travel to and from
matches in Vienna. Traditional video surveillance is limited by
relatively slow response times and heavy demands on computer time and
memory.

For over 40 years the technology has focussed on the concept of the
frame. Processing every pixel in every frame, even pixels that register
no change, takes its toll on computing and human time.

But by focussing on the pixel and not the frame, researchers in the
EU-funded Caviar project developed a frame-free vision system that uses
so-called ‘spike events’, produced whenever there is movement in the
scene. Spike events are also produced by human eyes and used in human
brains.

“Instead of using frames, each pixel decides when it wants to send
information, and that is dependent on changes in brightness," says Tobi
Delbruck, the project’s spokesman from the Institute of Neuroinformatics
at the Swiss Federal Institute of Technology in Zurich. “This is a nice
property because if nothing changes there is no output.”

By outputting spike events, the silicon retina, or DVS as it is known,
allows for processing of novel visual information at the moment it
occurs, resulting in greatly reduced power consumption and response
time.

While traditional cameras operate at a rate of about 50 frames per
second, the equivalent speed of the Caviar system is about 10,000 frames
per second, says Delbruck, who was responsible in Caviar for the
development of the silicon retina.

"The speed is in the data processing,” he says. “It comes down to the
fact of immediately processing only the pixels that need to be
processed.

“The beautiful thing about the Caviar system is that it is capable of
recognising an object and tracking it with 40,000 spiking elements or
‘neurons’ in the system without a single line of computer code.

"This avoids the need for computing power as it is a pure hardware
solution. We showed that it is possible to build hardware machines
capable of recognition and high speed object tracking without writing
code, but rather emulating a simplified brain-like neural architecture.”

That ability to selectively process visual information is what makes the
sensor and its chip technology so attractive for use in the Vienna
subway at Euro 2008. On the spot processing allows the instant counting
of people.

Eight cameras developed by Austria Research Corporation using the DVS
technology will be employed in a subway station near the main soccer
stadium in Vienna, and will be linked to entrance gates, which would
automatically close when the system determines the platform is too
crowded.
A dynamic vision sensor developed by the Caviar project. © Caviar
project
A look behind the lens

To achieve such results, the DVS developers had to develop four key
advances compared with previous work for the system. One was the pixel
design. Another was a design to allow the chip to work independently of
temperature and transistor parameter variation.

They also used USB2 technology so the system could easily interface with
computers, and finally developed software that could track people based
on the visual information, allow the events to be viewed in a variety of
formats, captured and replayed, as well as processed using precise
timing.

“The beautiful thing about the Caviar system, is that the front end
which includes the DVS is capable of recognising an object and tracking
it with 40,000 spiking elements or ‘neurons’ in the system without a
single line of computer code," says Delbruck.

"This avoids the need for computing power as it is a pure hardware
solution. We showed that it is possible to build hardware machines
capable of recognition and high speed object tracking without writing
code, but rather emulating a simplified brain-like neural architecture.”

The chips individually are still fairly expensive, says Delbruck, but if
these were to be produced in high volumes he calculates the cost would
work out to be about less than a euro for the silicon per chip.
An eye on the future

The Austria Research Corporation (ARC), which licences the technology,
has integrated the DVS into cameras for traffic monitoring as well as
for Euro 2008.

While the Austrian Research Centres (ARC) is focusing on surveillance,
what sets the Caviar sensor apart is the speed. And as such Delbruck
sees the greatest potential for its application in robotics.

Examples demonstrating this application include a car that drives itself
along a track. The car uses the sensor to follow a line on the track.
The application could be used for driver assistance.

The researchers have developed a robotic goalkeeper, Robogoalie that
some teams in the Euro 2008 might come to wish they had during the
tournament.

“It’s fun to play," says Delbruck. "The goalie is very responsive and
can ignore distractions. You could use multiple balls but the goalie
will only focus on one. It also puts hardly any load on the computer.
It’s a fun demonstration of the potential."

The project, which received funding from the EU's ‘Future and Emerging
Technologies’ during the Fifth Framework Programme, has officially
finished but the researchers involved continue to develop the silicon
retina and other components.

Indeed, a new chip with higher resolution is being developed. Part of
the project team based in Seville is working on the next-generation
convolution chip, which is the next stage of the Caviar hardware system.

Robogoalie is a glimpse of things to come, but the DVS’ immediate
contribution to the ‘beautiful game’ will be off the pitch at Euro 2008
helping with crowd control and contributing to increased safety for the
fans.

The Euro 2008 event takes place from 7–29 June.

Media note: This feature can be republished without charge provided ICT
Results is acknowledged as the source at the top or the bottom of the
story. You must request permission before you use any of the photographs
on the site. If you do republish, we would be grateful if you could link
back to the ICT Results site (http://cordis.europa.eu/ictresults). Let
us know if you republish so as to help us provide you with a better
service. If you want further contact information on any of the projects
cited in this story please contact us at ictresults at esn.eu.