Cruise knew that robo-taxis hardly recognize children and let them onto the street

Danger from autonomous cars
Cruise knew that his robo-taxis hardly recognize children – and sent them further onto the road

Due to numerous accidents, the robotaxi company Cruise was banned from driving in San Francisco. An investigation has now revealed that there are completely different dangers looming.

Cruise’s driverless taxis have been making headlines for months, from stalled ambulances to accidents. After one of the vehicles hit a passerby in October, California pulled the ripcord – depriving the company of its most important market San Francisco the license. But the problems are apparently even bigger than previously known.

This emerges from internal documents that were leaked to The Intercept magazine, which specializes in leaks. Accordingly, self-driving taxis have major problems recognizing children as such. Construction pits in the ground are also simply ignored – even if there are workers in them. And all this despite the fact that people have to intervene in automatic driving more often than was previously known.

Children not recognized

The company is well aware that the carts should actually recognize children as such. According to the internal documents, the technicians were concerned about the special challenges faced by the smallest road users, such as unpredictable behavior, a sudden run onto the road or unexpected falls.

However, the cars can obviously only respond adequately to this – because the artificial intelligence behind the wheel simply cannot reliably distinguish children from adults. “Cruise cars may not show any particular consideration for children,” the magazine says in internal documents.

The consequences were also noticeable in tests and simulations. “Based on the simulation results, we cannot rule out that the car would have hit the child,” says a results report. In another, a dummy the size of a small child was recognized, but still grazed by the side mirror – at 45 km/h. At least in test scenarios, the cars are said to have occasionally simply lost sight of children and therefore no longer showed any consideration.

Cruise denies allegations

When confronted with this, the company rejected the allegations. “It is inaccurate to claim that our autonomous vehicles do not recognize children or treat them with special consideration,” said a spokesman. The robots would certainly recognize and treat children as their own danger. According to communications boss Erik Moser, we expect around one accident involving children for every 300 million kilometers driven. “So far there have been no accidents involving children on the road.”

Still, the revelations don’t exactly paint CEO Kyle Vogt in the best light. Just in August, he complained to the New York Times about a case in which a 71-year-old fatally injured a four-year-old girl with her car – and claimed that self-driving cars would reduce such risks.

Missing experience

According to the documents, the problems also arise because the cars have hardly had the opportunity to deal with the particular danger. “We have very little contact with very vulnerable road users,” says one document, referring to the term used internally to describe children. The self-learning algorithms have a correspondingly difficult time learning how to use them.

One reason for this is a safety measure: In order to reduce the dangers for other road users, Cruise in San Francisco was initially only allowed to offer its services without a driver at night. And then of course there are hardly any children on the streets. A measure announced in August is therefore unlikely to contribute to the solution: just weeks after opening for daytime traffic, the company announced that it would continue to offer its services primarily at night.

No eye for construction pits

This does not affect a lack of awareness of danger elsewhere: Holes in the ground are apparently hardly noticeable for cruise taxis – even if they are large excavation pits. This has also been known for at least a year, according to The Intercept.

This is not just a problem in test scenarios. In at least one case, a cruise vehicle is said to have even driven through the barrier of a construction site – while the workers in the excavation pit tried to stop it from entering the pit by waving. A video stored by the vehicle documented the incident, but it is not clear whether the car actually came to a stop in front of the pit.

It is known that the cruise vehicles have a problem with barriers. The city’s fire department chief complained that robotaxis had already been directed several times in San Francisco to secured emergency locations and simply driven through barrier tape.

More human intervention than known

The decision to stop all driverless journeys seems entirely appropriate given these previously unknown problems. It was only at the weekend that CEO Vogt admitted in a forum that the cars needed more help than previously known, even in non-dangerous situations. Every four to six kilometers, the cruise taxis would call the headquarters to have a situation assessed. “Two to four percent of trips are controlled remotely,” explained the post on “Hacker News.” The company confirmed the authenticity of the post to CNBC.

Sources:The Intercept,CNBC, Hacker News