I loathe statistical averages because they are misleading. For example: “The average person has one breast and one testicle.” Across the population as a whole that statement is statistically valid, while also being anatomically stupid. But what really makes me furious is hearing AV executives spouting the “94 percent human error” garbage.
I am clearly not alone in my concerns about the AV industry demonstrating a loose relationship with hard facts. Writing on Twitter on October 4, Jennifer Homendy, Chair of the National Transportation Safety Board (NTSB) wrote: “Stop with the 94%!” followed by a link to a blog piece entitled “The 94% Error: We Need to Understand the True Cause of Crashes.”
Dissemination of the dodgy statistic is common and was even spoken by Waymo co-CEO Tekedra Mawakana in an interview published right after Homendy’s intervention. In it, Mawakana said:
“Really deciding that if 94-ish percent, give or take, of those are caused by human error, then we have the opportunity to innovate and remove the human from the equation.”
I am not being persnickety. Why is this casual campaign of misinformation by AV companies so unacceptable? Here’s three reasons for starters:
- Because Waymo, along with every other AV company, is testing experimental technology on public roads.
- Because Waymo, along with every other AV company, is socializing development risk using non-consenting subjects in the pursuit of private profit.
- Because, as shown below, the original source of the data doesn’t even say “Humans,” it says “Drivers.”
It would be unthinkable for the CEO of a food or drug company to make unsubstantiated claims without incurring intervention by the Food and Drug Administration (FDA), as Theranos founder Elizabeth Holmes and COO Ramesh Balwani discovered.
When even former U.S. Secretary of Transportation Elaine Chao is on record saying “94% of auto accidents involve human error,” something, somewhere, has gone seriously wrong. So, let’s turn to Phil Koopman, associate professor at Carnegie Mellon University, to help us get into the 94 percent statistic and understand the details behind the soundbite.
Lies, damned lies and statistics
In a detailed blog post from June 2018 entitled “A Reality Check on the 94 Percent Human Error Statistic for Automated Cars,” Koopman observes:
“You’ve heard that the reason we desperately need automated cars is that 94% of crashes are due to human error, right? And that humans make poor choices such as driving impaired, right? Surely, then, autonomous vehicles will give us a factor of 10 or more improvement simply by not driving stupid, right?
“Not so fast. That’s not actually what the research data says. It’s important for us to set realistic expectations for this promising new technology. Probably it’s more like 50%. Let’s dig deeper.”
If you haven’t read the blog in its entirety, I strongly invite you to do so right now, especially if you are an AV executive that ever gets quoted in the press. Koopman goes further in an in-depth video called “Implications of Removing the Human Driver,” and in a three-minute video (shown below) called “The 94 Percent Human Error False Narrative.”
Koopman’s analysis soberly reveals that behind the human error myth lies a much-overlooked body of evidence of humans as accident preventors. The conclusion that “Humans are amazing fault mitigators,” should be noted by policymakers who believe that reducing fatalities on our roads just means waving through AV technology as quickly as possible.
AV companies are not “saving lives,” but “saying lies.” We need a much more nuanced debate about the role of technology to reduce road deaths.
Safe system approach
We can roughly break down the role of safety-related technology in the transportation sector into two areas: To make human drivers safer; and to replace humans as drivers. Proven safety technology to make human drivers safer already exists, but is mostly overlooked amid the “self-driving” hype.
Four key factors which contribute to deaths on U.S. roads are distraction, drowsiness, impairment, and speeding. As part of the research for this article I attempted to quantify these factors and the analysis is presented below.
It shows impaired (drunk) driving and speeding were the leading causes of fatalities on U.S. roads in 2019. Practical experience of driving shows distraction from smartphones and other electronic devices to now be an everyday occurrence on public roads, while for drowsy driving NHTSA (National Highway Traffic Safety Administration) notes:
“Unfortunately, determining a precise number of drowsy-driving crashes, injuries, and fatalities is not yet possible. Crash investigators can look for clues that drowsiness contributed to a crash, but these clues are not always identifiable or conclusive.”
Data provided by Guardian (an aftermarket driver monitoring system installed in more than 30,000 heavy trucks) from Seeing Machines suggests truck drivers fall asleep about once every 8,500 miles and get distracted about once every 475 miles. I would argue therefore that the prevalence of distraction and drowsiness is much greater than NHTSA’s estimates suggest, and that further investigation, research and action is needed urgently.
Thus, rather than blaming human drivers, it is obvious that substantial safety benefits can be obtained by NHTSA mandating the installation of automatic emergency braking (in particular the most advanced variant to protect vulnerable road users), robust DMS (to maintain driver attention and engagement and to monitor for impairment) and intelligent speed adaptation to help drivers keep to speed limits.
The video below is the second roundtable panel chaired by Jennifer Homendy and is part of the NTSB’s Safe System Approach series, on the subject of safe roads.
At 41 minutes in, David Zipper, Visiting Fellow from Harvard Kennedy School says:
“I want to give a shout-out to one of the stupidest and most destructive statistics to come out of our federal government around transportation for a long time, the idea, which is so deeply flawed, that 94 percent of crashes are due to human error.”
Two months after being sworn into the role of NTSB Chair, Homendy’s influence on the road safety debate is startling and with NTSB advocating for a safe system approach, changes are likely to be both immediate and drastic.
One of the greatest differences I hope to see is procedural. In future, any AV executive that wantonly lies to the media or policymakers by repeating the 94 percent human error statistic should be fired.