Tesla Autopilot 2.0: next gen Autopilot powered by more radar, new triple camera, some equipment already in production

Tesla introduced the first Autopilot hardware suite, a front-facing camera, a front-facing radar, and 360° ultrasonic sensor in September 2014. A year later, in October 2015, Tesla released the first ‘Autopilot update’ (v7.0) to introduce features like Autosteer and Autopark. While the automaker was able to significantly improve the software and release new features on the same hardware, the sensor suite still offers its limitations

Enter Autopilot 2.0. The next generation Autopilot will feature new hardware on which Tesla will incrementally push new features through over-the-air software updates. The new sensor suite will enable level 3 autonomous driving and potentially level 4 fully autonomous driving in a not too distant future.

We now have some concrete information on Tesla’s second generation Autopilot hardware and it is coming sooner than you might think…

Sources with knowledge of the Autopilot program told us that the new suite will keep the current front-facing radar and add more around the car, likely one in each corner. Additionally, the system will feature a new front-facing triple camera system for which we are told Tesla started installing new housing in the Model S production this week.

Last month, dual front-facing camera housing with the second camera slot uncovered was spotted on Model X’s, but now it looks like the rearview cutouts is for a triple camera system, like the new Model S.

We reached out to Tesla for a comment on this report earlier this morning, but we didn’t get an answer. We will update if we do.

Update: We heard back from Tesla – a Tesla spokesperson refused to comment.

The reliance on radars is something Tesla CEO Elon Musk made clear would be in the future of Tesla’s Autopilot program, as opposed to lidar sensors. Even though a Model S with a lidar mounted on top was recently spotted at Tesla’s HQ, it doesn’t look like the technology will be part of Autopilot 2.0.

Musk referenced radar’s capability to see through rain, snow, fog and dust, as reasons to favor the tech over lidar. Tesla’s CEO hopes to be able to achieve similar coarse point cloud maps, something easier to achieve with lidar, with radars through “temporal smoothing”.

Adding several radar antennas to Tesla’s Autopilot system should help achieve this goal.

We are told by multiple sources that Tesla already wrote the software required to run the currently available features of the Autopilot on the new hardware suite. The company’s team would then build on the features and data accumulated through the Autopilot 1.0 program to release more advanced autonomous and safety features.

Tesla reveals new details of its Autopilot program: 780M miles of data, 100M miles driven and more

The front-facing triple camera system is likely based on, but not part of,  Mobileye’s ‘Front-facing Trifocal Constellation’. That system features a main 50° field of view (FOV) camera for general inputs, a narrow camera with a 25° FOV for redundancy in object detection, lanes and traffic lights, and a fisheye 150° FOV wide camera for parallel lanes, lane cutting, cyclist and pedestrian detection.

That’s despite the recent announcement that Tesla and Mobileye will soon end their partnership on the development of Autopilot systems. We are told that any disagreement had little to do with hardware and either way, Mobileye doesn’t make the cameras, but has “recommended sensors” to use with its ADAS products. Musk confirmed that Tesla is looking to bring more of the development in-house for a more integrated product.

Schematics of the triple cam system used in Tesla Autopilot’s engine control unit leaked earlier this year. The system Autopilot 2.0 camera will likely be very similar to it – though we are told it has changed since it leaked:

The Autopilot 2.0 hardware could also feature some additional sensors that we are not aware of or introduce the sensors incrementally, like with an Autopilot 1.5 for example.

When is it coming?

It’s already here. Well, sort of.

The new sensors have yet to be introduced in production vehicles, but everything points toward a release fairly soon. Several components have already been introduced in the production cars, like wiring harnesses and as previously mentioned, the housing for the triple camera system on the rearview mirror cutout.

With this said, there are also several other essential components to Autopilot 2.0 that need to make it to the vehicles before Tesla can release the system and we can’t confirm at which stage of development/production they are at the moment and therefore, we can’t offer an expected time of arrival for Autopilot 2.0.

For example, Tesla’s Autopilot 1.0 rely in part on the Media Control Unit (MCU) to relay information from the Autopilot to powertrain controls, but that wouldn’t work for 2.0. The system will need a dedicated computing channel and we can’t confirm when that will make it to Tesla’s vehicles.

Tesla has been rumoured to be working with Nvidia for its next computing platform for self-driving cars. Nvidia is already a Tesla supplier and its Tegra processors are powering the center console, infotainment, navigation and instrument-cluster systems of the Model S and X.  The company’s next-generation visual computing platform for cars, NVIDIA DRIVE PX 2, is expected to become available this fall. Again, it’s only a rumour at this point, but the timing is interesting.

The good news is that once the new hardware suite is available in the vehicles, the Autopilot will start improving a lot faster. Tesla’s system relies heavily on GPS data and its custom high precision maps. The system can, of course, still work from live data, but the ride is rougher.

Depending on the car’s GPS position, Tesla’s Autopilot downloads new high precision map tiles of the area based on data collected from the fleet.

Tesla’s custom maps will get a lot better with the data from the upcoming new array of sensors.  Tesla’s fleet equipped with Autopilot 1.0 is likely to also benefit from it.

In the meantime, Tesla owners with autopilot can still expect some significant improvements with the upcoming v8.0 of Tesla OS. There’s still no word on when the wide release of the update should happen, but it has been in closed beta for over a month now. Based on Tesla’s history of closed beta programs, it should be just a few weeks away from release.

Last week, Elon Musk commented on Tesla’s next generation autonomous technology: ‘What we’ve got will blow people’s minds, it blows my mind… it’ll come sooner than people think’