Chris Urmson: How a driverless car sees the road

Here's Chris Urmson talking about this 7 years ago when he was still working on (what is now) Waymo: ▶ this video

Driverless cars have advantages over humans too. Faster reaction speed, constant 360 degree surveillance, lack of distraction, (theoretically) better control of brakes and steering to handle skids. Waymo has demonstrated solutions for many of the edge cases you mention. This video is from 3 years ago: ▶ this video Regarding snow, it maybe a difficult problem, but if cars can't see road markings, neither can humans, so how do they do it? Sometimes by following a car in front. Though far from real autonomy, my Tesla doesn't need to see road markings for autopilot to work, it will fall back to following the car in front. And.. Why does Waymo have to solve this inclement weather problem now? Surely it would be better to demonstrate safety in good weather, then light rain, then heavy rain, then light snow? They can just refuse to operate in bad weather unless they are confident about it.

> They don't look remotely similar to me All of these agents follow a similar pattern of clothing (as I said, combination of similar garnments and colours) and behaviour (placement on the road, gesturing with authority, directly facing the car, etc.). Machine learning algorithms are especially good at recognizing patterns and storing their abstracted form, so it should come as no surprise that understanding what a police officer looks like in abstract is not the main issue of self driving. > Humans are just a lot better at gauging intention and even simple stuff like parsing the word "police" at an oblique angle Google seems to understand these intentions well enough, and at a much higher level than mere word parsing. This video is from 2015: ▶ this video You got the car understanding all that happens at a complex intersection at 9'05, understanding what a police car looks like at 9'35, then detecting and reacting to a schoolbus and then parsing a police officer gestures right at the 10' mark. I'd say chances are these are pretty solved situations 3 years later. You can even see some creatures from their "zoo" of patterns for cars & people at 10'35. Additionally, here is an article from last year: link > "When a Waymo car hears sirens, it will automatically pull over, yield, and stop. For example, when a number of vehicles are moving towards the scene of an accident on a highway and ambulances and other emergency vehicles are headed toward it, driverless cars will move aside and

> Tesla already does fully autonomous driving, they released that video in November. That video, for reference, is: link I'm not super convinced by that video. That drive had almost no challenges/obstacles. No bicycles, no construction, basically no pedestrians, no other vehicles behaving erratically. Google has cars that can respond to everything from a cyclist making hand signals, to a school bus (which must not be passed), to a police car pulling it over, to a woman chasing a duck across the road: ▶ this video I'm also not sure this is a simple case of "throw data and hardware at the problem." If it were that easy, it wouldn't have taken Google so long to get to where they are today.

> I realise it's not apples to apples and that Google's cars may be more autonomous for now. I think that is an understatement. This video shows why staying in your lane on a highway (what Tesla autopilot does) and dealing with city streets are two just completely different things: ▶ this video I think Tesla has a ton of catching up to do to compete with where Google is right now.

This is why Google is training their cars now in the real world to handle all of these kinds of things. Here's a Ted Talk [0] from 2015 where the head of their self driving unit makes the case for full autonomy vs. semi-autonomous because the semi-autonomous is good enough where people trust it, but not good enough to be trusted. Kind of like what happened where the Tesla driver went underneath the truck that wasn't properly detected. [0] ▶ this video edit: Around the 8min mark you can see some interesting interactions of the lidar and the world.

1. If that's the case perhaps another kind of blurring? "Intriguing properties of neural networks" ( link page 6) has examples where you get radically different classifications that I don't think would occur naturally or survive a blur with some random element, let alone two moving cameras and a sequence of images. As the title says it's an intriguing property, not necessarily a huge problem. 2. I honestly can't think of a situation where this could occur. It's the equivalent of kids shining lasers into the eyes of airline pilots, but the kids need a PhD in deep learning and specialised equipment to be able to do it. A hacker doing some update to the software via a network sounds much more plausible than attacking the system through its vision while it's traveling. 3. This is the real point in the end I guess, this Google presentation ( ▶ this video ) shows that the first autonomous cars to be sold will be very sophisticated with multiple systems and a lot of traditional software engineering. Hopefully LIDAR costs will come down.

> A Slate article from 2014 stated that Google's cars wouldn't know a stoplight existed, and would simply run it, if it wasn't in Google Maps That muse be old. In their TED talk from last year, they had it spotting and reacting to cyclist hand signals, school bus mechanical stop signs, and even a police officer directing traffic at an intersection. ▶ this video edit: I see you mention this below. Weird article, though. The author says: > In fact, when I first wrote about the car for MIT Technology Review, Google admitted to me that the process it currently uses to make the maps are too inefficient to work in the country as a whole. but his previous article he links to says > Maps have so far been prepared for only a few thousand miles of roadway, but achieving Google’s vision will require maintaining a constantly updating map of the nation’s millions of miles of roads and driveways. Urmson says Google’s researchers “don’t see any particular roadblocks” to accomplishing that. (and, incidentally, you quote the word "impractical", which appears in neither quote :)

The TED talk that Chris Urmson gave shows an example of the Google cars detecting a police officer giving signals.[1] [1] ▶ this video