When the app stays quiet

17. June 2021

When it comes to surfing, texting, shopping, finding your way through a city and, yes, even making phone calls, smartphones have become an integral part of the everyday lives of a lot of blind or visually impaired people. But their path to digital freedom is often still strewn with stumbling blocks - and, in the worst case, blocked altogether. Domingos de Oliveira told us all about where things stand with digital accessibility and where the pinch points are. The political scientist has been blind from birth and advises authorities and companies on accessibility issues.

For visually-impaired many people, the dozens of app symbols which jostle for space on the smartphone display are nothing more than blobs of colour. To make it easier for them to use their mobile phones, the relevant operating systems offer various operating aids: Magnifying glass functions, options to change font and symbol size, stronger contrasts and inverted colours, which are colour inversions that cause the background to turn black and the font to become white. “This reduces dazzle and makes texts easier to read,” explains Domingos de Oliveira. Once enabled, the settings are active in all apps – assuming they support these functions. But that is by no means the case for everyone, says the accessibility expert.

For blind people, their smartphone use depends entirely on having a reliable screen reader: The voice assistant reads out all the icons, labels and texts on the display, from the battery status and the name of the caller through to the app you happen to be selecting. Allowing the user to navigate via the touch screen without seeing the screen. On iPhones, the screen reader is called VoiceOver, on Android it’s TalkBack, on Samsung devices you have the Samsung screen reader, and Amazon devices use a function called VoiceView. In the best-case scenario, the screen reader also offers voice-operated input aid and acoustic feedback during operation and can be paired with Braille keyboards to read or write in Braille.

Whether you use iOS or Android, when it comes to operating aids for the visually impaired, there’s little to choose between the two operating systems, says Domingos de Oliveira. “For blind people, however, Apple is still much better.” VoiceOver offers more settings options which also work more cleanly. With his new Android phone, the accessibility expert got no further than installation: For blind people to be able to set up their smartphones themselves, voice output needs to be activated as soon as the phone is switched on. Apple found a simple solution to this: if you press the side or the home button three times, the screen reader pipes up. “I didn't manage to get that far on my Android smartphone – despite having the current operating system,” says de Oliveira.

To recognise street signs and house numbers or to read menus and timetables, the visually impaired often use small hand telescopes, known as monoculars. The smartphone camera’s digital zoom combined with the large displays on today’s mobile phones has become a convenient alternative, explains de Oliveira. When combined with artificial intelligence, however, the camera can do much more, as Seeing AI demonstrates. This Microsoft-developed app describes itself as a Swiss army knife for the blind and visually impaired: it reads short texts and longer documents and recognises people, products and banknotes. If you’re in the supermarket and want to know whether a tin contains tomatoes or beans, you can scan the barcode or have the label read to you. This was originally available only in English; since the end of 2019, there have also been German, French, Spanish, Japanese and Dutch versions.

The app also features an integrated scene description. When you take a photo, the AI describes what it thinks it can recognise on it. “This works quite well in many everyday situations,” says de Oliveira. However, the app needs to offer blind people more help when it comes to using it. This might for instance include acoustic warnings that you aren’t holding the camera straight, that a document is blank side up or that the lighting conditions will rule out good results: “After all, as a person who can’t see anything, you’ve absolutely no way of knowing.”

Blind or visually impaired people can also “borrow” the eyes of volunteers – via the Be My Eyes app. If they want to know whether a carton of milk has gone off in the refrigerator, whether they’re trying to put on odd socks or what colour their shirt is, they can launch a video call and share their camera with sighted people. “This is a great help, especially since apps like Seeing AI almost always get it wrong when it came to colour recognition,” says Domingos de Oliveira. According to the app’s own statistics, more than 4.7 million volunteers and around 310,000 blind and visually impaired people are connected by it. In the USA, there is Aira, a chargeable service with similar functionality, which the visually impaired can use any time they are looking for a house number, sorting the laundry or find that the stops are only displayed visually on the bus and not announced. “We could really do with something like this here in Germany,” says the expert.

In everyday life, Domingos de Oliveira mainly uses ordinary apps: Google Maps to find his way through the city, the weather app for a “view” out of the window, and Twitter as his link to the internet debate. As his experience shows, relevant app giants such as Facebook, WhatsApp and Twitter are doing quite well in terms of accessibility. Not least since their millions and millions of users keep a strict eye on them. “Every time Twitter installs features that aren’t one hundred percent accessible, you get something of a shitstorm.” And the Internet giants want to avoid this kind of trouble from the outset if they possibly can. “There are problems especially with the smaller apps that are less in the public eye,” says the expert. Weak contrasts make them difficult for visually impaired people to read, icons aren’t labelled, and text is not stored as text, but as an image instead. For the screen reader there is then nothing to read, meaning that the app stays quiet for blind people.

Although websites and apps run by public authorities have had to be accessible for several years now, this is by no means the case for everyone, Domingos de Oliveira notes. “If I want to book an appointment to see someone at the local authority, for example, it sometimes works with a lot of jumping through hoops, but sometimes it doesn’t work at all.” The problem from the expert’s point of view is this: Developers often still lack know-how in terms of accessibility, and their clients – be they authorities or companies – don’t have the expertise to verify the resulting products.

Accessible services would not only benefit blind or motor-impaired people. Older people, for example, are less able to recognise contrasts, and demographic change means that they are increasingly in the majority. Even so, developers still all too often regard accessibility as a kind of bonus. “But accessibility isn’t a bonus; it’s a matter of software quality. This is something developers finally need to see,” pleads Domingos de Oliveira.

With official forms, too, the expert still sees a lot of room for improvement. The tax return, for example, with its various tables and columns, is extremely difficult for blind and visually impaired people to make sense of. “It might one day fit on a beer mat, but I’m not holding my breath. And yet, simplifying the tax return would make life much easier for me and others.” This is likely to resonate with many sighted people.