Tesla Autopilot is going to detect potholes and make mini-maps to remember them

Tesla’s Autopilot, which the automaker is trying to turn into a self-driving system, is going to detect potholes and make mini-maps to remember them, according to a new comment from CEO Elon Musk.

In order to achieve full self-driving, a system would have to be able to handle a wide range of different scenarios, including different weather and road conditions.

These conditions, like potholes, can sometimes be difficult for human drivers to handle, and some people find it improbable that self-driving systems will be able to appropriately navigate them.

Tesla is leveraging its large customer fleet equipped with Autopilot hardware to capture data on those corner cases and teach its neural network to handle them.

As we recently reported, Tesla is going through “a significant foundational rewrite in the Tesla Autopilot.” As part of the rewrite, Musk says that the “neural net is absorbing more and more of the problem.”

It will also include a more in-depth labeling system.

He added in an interview on the Third Row Podcast:

3D labeling is the next big thing — where the car can go through a scene with 8 cameras and paint a path in real-time, then label the path in 3D. This is probably a two to three orders of magnitude improvement in labeling efficiency and accuracy.

When asked if Tesla could use this new 3D labeling to create maps of potholes, Musk answered in the affirmative:

Yes

— Elon Musk (@elonmusk) February 3, 2020

It means that Tesla vehicles could build maps that detect potholes and help other Tesla vehicles avoid them.

Musk didn’t say when the update is coming, but he did say that the Autopilot rewrite is almost completed.

Electrek’s Take

This is a big deal.

To be fair, plenty of human drivers are not attentive or quick enough to avoid some potholes, but a full self-driving system will be held to much higher standards and will have to avoid potholes

Last year, I logged in over 300 km (186 miles) of driving on the latest Autopilot update available in my Model 3 over a few hours.

The system performed extremely well, autonomously steering on the highway and adapting the speed to the traffic around the electric car.

With one hand on the steering wheel applying light pressure, Autopilot controlled the vehicle from point A to B almost without any input from me.

The only times I had to take over was to avoid large potholes, which are unfortunately a plague in Quebec, especially during the spring:

I just drove over 150 km on @Tesla Autopilot. The only interventions were related to avoiding potholes. Any progress on that front @elonmusk? pic.twitter.com/pYOnD57eIt

— Fred Lambert (@FredericLambert) April 6, 2019

If Tesla can figure this out, it will be a significant step forward. And of course, the same system can be replicated for other things than potholes.