Google self-driving cars would have caused 10 accidents in 2015 without test drivers taking control

Google published today its annual report on disengagements of autonomous mode for its self-driving car program (SDC), which is required by California’s DMV. The report details events where Google’s test drivers have taken control over the car’s autonomous system, whether it’d be because of a shutdown due to technology failure or for safety reasons.

After having driven over 1.3 million miles in autonomous mode, Google’s self-driving cars were involved in 17 reported accidents, but the company prides itself of not having been at fault in any of them. But with today’s report, we learn the self-driving cars would have caused 10 accidents during the past year if test drivers wouldn’t have disengaged the system and taken controls.

Test drivers disengaged the self-driving system for safety related reasons on 69 occasions between September 2014 and November 2015. It doesn’t mean that the SDC would have caused an accident on each of the 69 occasions, but Google developed a simulator to predict what would have happened if test drivers would have left the system alone. Google describes the simulator:

“To help evaluate the significance of driver disengagements, we employ a powerful simulator program — developed in-house by our engineers — that allows the team to “replay” each incident and predict the behavior of the self-driving car (had the driver not taken control of it) as well as the behavior and positions of other road users in the vicinity (such as pedestrians, cyclists, and other vehicles).”

As per its own simulations, Google determined that its SDC would have made contact with an object on 13 occasions out of the 69 events. In 10 of the 13 simulated contacts, the system would have been at fault. Though 2 of the 10 simulated contacts would have been with traffic cones…

The chart below shows that up until Q3 2015, Google was making significant progress and driving more and more miles without having to disengage the system for safety reasons, but things changed during the second half of the year:

Though it’s not all bad. The report also shows great improvements in autonomous miles driven per disengagement for technology failures over the same period:

As the company clearly states in the report, Google’s objective is not to minimize disengagements. They are a critical part of the testing process and help identify areas of improvement to ultimately make the system safer.

After reading the report, we have to give credit to Google’s test drivers. At the end of the day, Google’s SDCs have yet to cause an accident, but the test drivers are just as responsible for that as are the engineers behind the project.

Here’s the report in full:

[slideshare id=56981461&doc=report-annual-15-160112233509&type=d]