Tesla claims you are ‘7x less likely to be in a crash’ with Autopilot, but the data is not that clear

Tesla claims that a driver is “7x less likely to be involved in an accident” with Autopilot enabled.

The data is not that clear, but it shows improvements in a new Autopilot safety report from Tesla.

Tesla Autopilot safety report

Since 2018, Tesla has been trying to create a benchmark for its improvement in Autopilot safety by releasing a quarterly report that compares the number of miles per accident on Autopilot versus off Autopilot.

Today, Tesla released its report for Q4 2020:

In the 4th quarter, we registered one accident for every 3.45 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.05 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.27 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles.

The best comparison point is against the same metric over the same period last year to account for the weather and seasonal conditions:

• Autopilot Engaged: It is at one accident for every 3.45 million miles driven in Q4 2020, which is an improvement from every 3.07 million miles driven in Q4 2019.
• Autopilot disengaged but with active safety features: It is at one accident for every 2.05 million miles driven in Q4 2020, which is actually down from every 2.10 million miles driven in Q4 2019.
• Autopilot disengaged and without active safety features: It is at one accident for every 1.27 million miles driven in Q4 2020, which is down from every 1.64 million miles driven in Q4 2019.

Therefore, it shows an overall improvement with Autopilot active, but the data also shows worse performance with the safety features only.

Tesla claims Autopilot 7x less likely to be involved in a crash

When Tesla shared the new report today, the automaker interpreted the data as being “7x less likely to be involved in an accident” with Autopilot enabled:

Tesla appears to come to that number by comparing miles driven in which drivers had Autopilot engaged to overall miles driven between crashes in the US.

That’s a flawed way to look at the data since Autopilot is currently primarily used on highways where it’s easier to accumulate a lot of mileage without accidents, and non-Autopilot mileage is coming from city driving, where accidents are more likely, so the two datasets can not really be compared.

Overall, NHTSA average combines both city and highway driving, and with all types of passenger vehicles.

Electrek’s Take

Having driven with Autopilot for thousands of miles, there’s no doubt in my mind that it is safer than without.

But I don’t think we are at a point where we can be throwing around these numbers just yet.

Tesla is going to need a lot more data and present it in a more accurate way if they ever want to convince authorities to allow them to deploy their full self-driving system.

Nonetheless, I think comparing the safety report is a good start, and it is useful when compared to the same per period in the previous year.