The Independent's journalism is supported by our readers. When you purchase through links on our site, we may earn commission.

This amazing 'neural network' program describes what it sees on a trip through Amsterdam

The program can identify what it's looking at, live, through an ordinary laptop webcam

Undefined
Wednesday 25 November 2015 20:31 GMT
Comments
The program managed to correctly say that it was looking at an image of a man eating a hot dog
The program managed to correctly say that it was looking at an image of a man eating a hot dog (Kyle McDonald/Vimeo)

A digital 'neural network' has been put through its paces on the streets of Amsterdam - with a video showing how the program can identify people, vehicles and even different types of food by analysing live webcam images.

The NeuralTalk code was originally designed as a way to automatically caption images - but after some tweaking by American coder and artist Kyle McDonald, it was made to caption live video instead.

NeuralTalk and Walk from Kyle McDonald on Vimeo.

The software isn't 100 per cent accurate, but it's pretty impressive - seeing a man walking with a bag, the software produces the caption: 'Man walking down the street with a suitcase'.

It also manages to correctly identify boats, cars and bicycles (useful for when you're using it in Amsterdam).

The software really comes into its own when McDonald approaches a group of guys at a hot dog stand.

Looking at the laptop's screen, they're surprised to see the program identifying them correctly - as 'a man holding a hot dog in a bun'.

Pointing the webcam at the hot dog stand's counter, the program correctly says it is a 'refrigerator filled with lots of food and drinks'.

It's a cool thing in itself, but the technology could have important applications - such as helping the blind by automatically describing images and videos. But for now, the hot dog stand is a major achievement.

Join our commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in