Automatic Alternative Text: How Facebook's new tool is helping blind people explore the internet

Andrea Peterson looks at how tech companies are changing the way blind people experience the world, on and offline

Social media is a very different experience for the visually impaired (Rex)

Matt King sat down one Saturday morning six years ago with what seemed like a simple mission: he wanted to add a few friends on Facebook.

"I eventually accomplished my goal," he says. "But I have to say, it took me an entire morning – at least four hours – to do what somebody else probably could have done in a few minutes."

The reason it took so long is that King, in addition to being an engineer with a past as a world-class cyclist, is blind. He was born with a degenerative disease that slowly cost him almost all of his vision by the time he was at university.

Now, he's one of the people trying to make Facebook usable for everyone – including the new feature that Facebook rolled out this week that lets blind users "see" pictures that their friends post like never before. This feature, called Automatic Alternative Text, is just one way tech companies are changing the way blind people experience the world, on and offline.

There are an estimated 39 million blind people around the globe, according to the World Health Organisation. And just because they can't see, it doesn't mean they can't use the internet. But the way they experience it is very different from how sighted people do.

"You can see a whole screen full of information and because of that you're able to make decisions about how to get to the information you need," explains Chris Danielsen, director of public relations for the National Federation of the Blind (NFB) in the US. "For a blind person, it's a bit more cumbersome."

The blind rely on tools called screen readers to narrate websites and apps or translate them into braille – one line of text or link at a time. Instead of using a mouse, they also generally jump around a page using keyboard shortcuts.

And because screen readers work by looking at a website's code, not just what is visible on a display, sites can be almost incomprehensible if they weren't designed to be read by the tech.

But there are simple things, often invisible to a sighted viewer of a site, that a developer can do to help. One is making sure images on a site have "alt texts", descriptions of what's in the pictures so they can be described by a screen reader.

"There are all kinds of little tricks like that," says Danielsen. "If the website is properly coded, you can get around it pretty easily – otherwise, it can be very frustrating to navigate."

The early days of the internet were actually easier for blind users because almost everything was text, according to King. "But over time not just the content, but also the design has become much more visual in nature," he says.

Social media has been a big part of that shift. Facebook says that people share more than two billion photos a day across its various products. But those pictures can cause an accessibility headache because they often don't come with text that describes what they show.

Facebook is trying to fix that with Automatic Alternative Text. It works when a user has Apple's built-in screen reader turned on and selects a picture in the app. Then Facebook uses artificial intelligence algorithms to detect basic features in the image and create a new alt text that the screen reader will share with the user – basically describing what's in the photo.

If a picture shows a couple overlooking the ocean while wearing sunglasses, for instance, the new alt text will be something like this: "This image may contain: two people, smiling, sunglasses, sky, outdoor, water."

It's not exactly poetry. Eventually, King says, the hope is to make the alt texts more like a narrative. But it still gives blind users a new way to imagine what's in images.

Facebook is far from alone in trying to make the internet more usable for people who are blind or suffer from impaired vision. Twitter recently announced a feature that will finally allow users to attach an alt text to the images they upload.

And most major tech companies have dedicated accessibility teams. In fact last summer, Facebook poached King from IBM – where he had worked on accessibility for a quarter of a century.

Many tech companies, including Facebook, along with several universities, are also part of a working group focused on better preparing students to create inclusive technology.

The World Wide Web Consortium, an online standards body, has also long had international best practices for how to design sites that everyone can use. But not all sites adhere to them.

And while there have been many strides in making the internet more accessible for the blind, getting to this point has been tough. The NFB has brought legal action against tech companies in the hopes of making their services more accessible: it sued AOL in 1999, and in 2008 Apple agreed to a settlement with the NFB and the Massachusetts attorney general aimed at making iTunes accessible to the blind.

Matt King was an engineer for IBM prior to joining Facebook

As more and more of our lives go online, it's even more important that the blind are able to stay connected, Danielsen says: "This is not just a matter of making life easier for blind people – it's a matter of how they're actually going to get along in the 21st century."

Microsoft showed off an app dubbed "SeeingAI" at its Build 2016 developers' conference last week. In a video presentation, Saqib Shaikh – a blind developer who has been with Microsoft for a decade – used the app with a pair of smart glasses that let him take pictures in real time and have the world narrated.

The company said it doesn't have a timeline for when the app will actually be released. But its development hints at how wearable computing could have a game-changing effect.

King says there's still more to be done on the online front, however. He can easily imagine Facebook using its artificial intelligence tech and facial recognition data to tell him not just how many people are in a photo, but also who they are.

There may be privacy concerns, but it could at least be used to help identify public figures, he says.

"If everyone else can see that's a picture of Donald Trump, I should be able to know that too," King argued.

The Washington Post