Amazon’s Alexa is developing dynamically. She was once supposed to start the shopping revolution, but today she is sometimes only allowed to set a timer. But for people with disabilities, Alexa is an almost perfect everyday aid. What does the assistance system really do? We asked around.
Even if Alexa has not yet developed into the desired shopping companion with billions in sales, Amazon’s work on the project continues steadily. Especially for those who can hardly do without Alexa – people with disabilities who only have language to cope with everyday life. The star spoke to Dr. Philipp Berger, Alexa boss in Germany, Sven Paukstadt, Alexa expert and responsible for the “Alexa for All” project, and Hamoun Kamai, an Alexa user from the very beginning who has been paralyzed from the shoulders down since an accident inclusive project – and what it really means to some.
Mr. Kamai, where exactly does Alexa help you in everyday life?
Kamai: The question is more: What doesn’t she do? I am paralyzed from the shoulders down and can only use technology through speech. This now affects my entire home. Opening doors, operating the television, controlling lighting and room temperature – I do everything with Alexa.
Does Alexa support all of this out of the box?
Kamai: A lot, yes. But when it came to a solution for controlling my nursing bed, I had to come up with my own. This isn’t always easy, but it’s at least possible thanks to the many smart home products that can be integrated. And if I have a problem, I talk to Amazon directly.
How does this collaboration work and who do you contact?
Kamai: When Alexa was launched, I simply contacted the communications department. I still do that today because people know me there.
Is this actually the desired path?
Paukstadt: The path that Hamoun chose was certainly the right one at the time. Many people also contact us via social media, where we also have employees who forward relevant inquiries to us. There is also an email inbox that you can contact, called [email protected]. Another way to reach us is through the partner associations we work with, for example the Josefs Society, the German Alzheimer Society or the German Red Cross. There are always test series there that those interested can take part in. The feedback from this or further ideas are usually shared with us.
Dr. Philipp Berger is Country Manager Alexa for Germany and Austria and is therefore basically Alexa’s boss in this country.
© Amazon / PR
Was it planned from the start to develop Alexa with a focus on people with disabilities or did it happen by chance?
Dr. Berger: Yes and no. It stands to reason that you have people with you Disabilities are taken into account when developing a product based on voice control. For example, you immediately think of blind people. On the other hand, we were in a completely different place all those years ago. It was something special to have voice control in the room at all – the subtleties came little by little.
Because customers first had to discover the technology.
Dr. Berger: Of course. All those years ago, we didn’t know how the customer even interacted with Alexa. There were a handful of features and then we realized very quickly: We weren’t expecting how it would be used.
What does that mean?
Dr. Berger: Interactions like saying good morning to Alexa every day – at the beginning we didn’t have as strong a focus as we do today because we were initially surprised at how much people wanted to have a dialogue with Alexa. We assumed that Alexa had to be functional and not such a powerful everyday companion. Luckily, Alexa’s development has always been fairly open-ended, allowing us to accommodate adjustments.
That would be?
Dr. Berger: Older people or children who talk to Alexa, for example, often have pauses for thought – you have to experience this in practice before you can optimize for it. This so-called adaptive listening came then. This means that Alexa stays on reception longer after the activation word and thus compensates for the problems of people who have difficulty with articulation. It shows that we take feedback from different customer groups in order to further develop the product – Alexa for everyone.
For me, for everyone also means: for the deaf. They can’t yet integrate Alexa into their everyday lives.
Paukstadt: We know that. But we have already thought about it, even if it is of course a real problem with voice control with sound playback. Theoretically, Alexa devices, which already work with a camera and a screen, could be made suitable for entering sign language. But this is more complicated than you think, because really understanding sign language and processing it correctly is very difficult because it is not the same internationally, but rather differs greatly. What we know as a dialect in the language also exists there. A mammoth task. We can’t do this overnight, but will make it available when the technology and effort allow.
In addition, these people can operate touchscreens.
Dr. Berger: When it comes to buttons and extensions on a graphical level, we are well positioned. There are even physical buttons to which Alexa routines can be bound without voice commands. We don’t just have deaf people in mind, but also people with Alzheimer’s, for whom something like this can help to structure their everyday lives. This can be done using very simple routines, such as a switch to raise all the blinds or start an otherwise perhaps too complicated video call with the family.
Paukstadt: In such cases, the subtitles also help on devices with screens, i.e. Echo Show devices.
Hamoun Kamai has been in a wheelchair for years and has been paralyzed from the shoulders down since an accident. For him, Alexa is an invaluable help in everyday life because he can only control many things with his voice.
© Amazon / PR
That all sounds very rounded. Is there anything missing for a perfect everyday life?
Kamai: Yes, absolutely. Since I started using Alexa, communication has become too simple for me. I would like to see real dialogues that are ideally based on prior knowledge. Such a real co-pilot for everyday life.
Like Jarvis in Iron Man.
Kamai: That would be the dream, yes. If Alexa were then able to give me tips on new projects, for example my bed control, on how best to set it up and what I need for it, that would be super useful.
Could a large language model like ChatGPT help with this?
Dr. Berger: It is compatible, but there are many steps that we have to take. Generative AI still needs to be properly integrated into the physical world so that you can really control things with it. Something like ChatGPT is not enough.
Because ChatGPT also makes a lot of mistakes.
Dr. Berger: The hallucinations are a problem, yes. Sometimes the system simply says nonsense and the data is based on very old knowledge, which is not good for a voice assistant with everyday functions. That doesn’t mean we don’t work with it. LLMs help us enormously to process large amounts of text and thus train Alexa. And we continually optimize the Alexa experience. To this end, we will roll out additional Alexa functions based on generative AI over the course of the year.
What do people who rely on Alexa actually do when the internet goes down?
Kamai: I’ll step in between them. This is actually irrelevant, since almost all of my devices would fail if there was an internet failure. Whether I can reach them with voice commands or not doesn’t matter. A functioning internet simply has to be there, which is why offline functions don’t play a big role in my opinion – unless, of course, the entire ecosystem communicates on a local level. But unfortunately it doesn’t.