Watch how Tesla Autopilot sees a goose crossing the road

In an attempt to be the first to deploy a truly fully autonomous driving system later this year, Tesla is collecting corner cases that its vehicles encounter on the road with its Autopilot system.

We now get an interesting example with a visualization of how Autopilot interprets and reacts to a goose crossing the road.

At a presentation about Tesla’s latest self-driving and computer vision capabilities last month, Tesla Sr. Director of AI Andrej Karpathy said that they are currently working to identify and react to corner case scenarios that vehicles encounter on the roads.

Karpathy says that they are pooling a lot of different rare events from their entire fleet of vehicles equipped with cameras in order to teach their Autopilot system to react to them.

Earlier this month, a Tesla owner claimed that Autopilot stopped for a rabbit on the road caught on a dashcam video.

At the time, we noted that the accuracy of the owner’s claim is unclear and that a visualization of what the Autopilot can see, like hacker verygreen has been producing, would be useful in this situation.

While green couldn’t produce a rabbit out of his hat, a goose did appear on the road while the Autopilot data was being collected and he produced the following visualization using that data:

Well, this ain't no rabbit, but we can clearly see Autopilot does not respect Canadian Geese, it just does not respect them. pic.twitter.com/dlBOIFmnaw

— green (@greentheonly) May 21, 2019

As you can see, Tesla’s Autopilot doesn’t detect the goose for the most part of the clip.

It does eventually detect and classify it as a pedestrian for a fraction of a second.

During the second part of the video, the goose is not classified, but Autopilot seems to know that it can’t drive over it:

The green space is where Autopilot detects as safe to drive while the clear box, which appears over the goose, is interpreted as not appropriate to drive over.

Electrek’s Take

There’s a lot of talks right now about whether or not Tesla’s goal of achieving full self-driving by the end of the year is realistic.

I think the consensus is that it’s not and I tend to agree, but I also think that Tesla is making a lot of progress on the AI and machine learning front.

If that progress proves to be exponential, we could have a surprise on the full self-driving front.

However, those corner cases, like animals crossing the road, seems like an extremely hard problem to solve.

While it looks like the Tesla vehicle would have run over the goose in this video if it was on Autopilot, I see some potential based on the visualization.

In the meantime, drivers need to make sure that they are paying attention and ready to take control at all times.