The Tesla Full Self-Driving crowdsourced dataset that Elon Musk has approved has doubled since the CEO shared it last month.
What does it say about the state of Tesla’s Full Self-Driving program now?
We have been complaining for years about Tesla’s lack of data on its Full Self-Driving program. In September 2024, Tesla started releasing monthly AI/self-driving roadmap updates and referenced improvements in some metrics that it wouldn’t even share.
Unsurprisingly, Tesla quickly stopped releasing the monthly updates after missing several milestones.
Because Tesla has not provided data, we have to rely on a less-than-ideal but better-than-nothing crowdsourced dataset.
Tesla fans often criticized the dataset because it doesn’t make the Full Self-Driving system look great, but it’s harder to do now that Tesla CEO Elon Musk has positively referenced the dataset on two occasions.
The last time Musk referenced the data, he misrepresented it as confirming “exponential improvement” by singling out highway mileage.
This was data back when Musk referenced it last month:
Musk particularly praised the highway mileage of 723 miles between disengagement, which was a big jump over Tesla FSD v12, but we noted that this was misleading because v13 finally merged city and highway software stacks to include end-to-end neural nets.
This had been delayed for highway driving for years. Therefore, Musk was trying to make this update look like an “exponential improvement” when, in fact, it was simply Tesla finally using years of development in city driving and transferring it to highway driving, which was supposed to have happened a long time ago.
However, Musk’s approval of crowdsource data confirms that it is in the ballpark of what Tesla sees with FSD. Otherwise, it would be incredibly misleading for him to share it positively, even if he has misrepresented it to make it look positive.
When Musk shared the data, there were only 8,000 miles on the new FSD v13 update. Now, the crowdsource dataset has almost twice as many miles on Tesla FSD v13.
With more miles, the performance actually went down. Miles between disengagement on highway went from 723 to 624 miles.
Tesla’s primary metric has been “miles between necessary disengagement,” which is currently 489 miles.
The automaker previously claimed that v13 would bring “5 to 6x improved miles between necessary interventions” compared to v12.5.
The average of all v12.5 updates is at 183 miles. Therefore, it’s actually closer to a 2.7x improvement.
Where does Tesla need to go from there?
Ashok Elluswamy, the head of FSD at Tesla, has previously stated that for Tesla to enable unsupervised self-driving, Tesla needs to achieve the average in miles per critical intervention “equivalent of human miles between collision,” which stands at 700,000 miles, according to NHTSA.
Electrek’s Take
Yes, a 15,000-mile dataset is quite limited, but you FSD fans can’t complain since Elon bragged about it with just 8,000 miles.
500 miles sounds about right to me. It means that Tesla would need a 1,400x improvement to achieve the level of safety it said it needs to achieve unsupervised self-driving.
Anyone who thinks this is happening this year is kidding themselves.
However, I believe that Tesla is working around that. Elon said that Tesla is launching “unsupervised self-driving as a paid service in Austin in June,” and he confirmed that it will be done with “Tesla’s internal fleet.”
This means a Waymo-style geo-fenced autonomous ride-hailing system assisted by teleoperation – aka what Elon trashed for years and said that Tesla would easily surpassed with its approach.
Top comment by Preston
I use FSD (HW3), and I'm pretty impressed. Most of my disengagements are due to navigation issues. It's particularly bad about intersections with multiple turning lanes, picking the wrong one, or sometimes turning into the wrong one. There's also one intersection I just don't trust (a stop sign where the through traffic is coming up a ramp at a sharp acute angle). Avoiding those, I usually only disengage for routing or where it's too timid when I have cars behind me.
The one thing Tesla isn't doing that they should is using FSD for map updates. They could easily have FSD report back when the observed road conditions differ from the map data by some threshold, which would then automatically update the map. Accurate maps aren't critical for safety, but they are for navigation.
You can deploy such a system with much lower miles between disengagement. Something like 10,000-20,000 miles is achievable, but the lower the mileage between disengagement, the messier it will be.
In a limited geo-fenced environment, Tesla could possibly achieve that by June in Austin.
The thing that Tesla fans need to understand is that this will only mean that Tesla will be where Waymo was years ago. It won’t get them closer to what Elon has been promising for years: unsupervised self-driving for the entire customer fleet built since 2016.
Tesla would need to scale like Waymo is, which Elon has been claiming is not scalable.
FTC: We use income earning auto affiliate links. More.
Comments