Lexus Rx300 autonomous test vehicle.

Lexus Rx300 autonomous test vehicle.

Between 2013 to 2015 I was a part of Google’s Self-Driving Car Project, today known as Waymo.

Vehicle on road testing

One of my first responsibilities is to perform on-road testing of our autonomous vehicles. This involved driving on public roads with the vehicle in autonomous mode, but constantly monitoring vehicle behavior, and documenting any behaviors that is out of the norm to be investigated by the relevant engineers.

Our vehicle driving on the freeway.

Our vehicle driving on the freeway.

Co-driver monitors what the vehicle is seeing and inputs information as needed, especially when it comes up unexpected behavior, so it can be reviewed later by engineering.

Co-driver monitors what the vehicle is seeing and inputs information as needed, especially when it comes up unexpected behavior, so it can be reviewed later by engineering.

Data labeling

The vehicle needs to know what it is “seeing” in the world using LIDAR, and to do that, we need to take existing LIDAR data teach the system what a vehicle looks like, what a pedestrian looks like, what a bicycle looks like, etc, and train it with the machine learning algorithm.

Many examples what vehicles looks like as point cloud.

Many examples what vehicles looks like as point cloud.

After training, the vehicle can now identify the raw LIDAR point cloud as vehicles, pedestrians, etc.

After training, the vehicle can now identify the raw LIDAR point cloud as vehicles, pedestrians, etc.

Simulation creation and testing

Before we deploy our software in our fleet for real world test, we need to test it in a virtual environment.Using the road data collected we can re-created the world based on our needs and test for specific scenarios. I was responsible for designing thousands of scenarios that can test various aspects of vehicle behavior such as intersections, merging, interaction with pedestrians, reacting to signals, etc.

A playback of a what is captured during live vehicle test, showing vehicle behavior and prediction.

A playback of a what is captured during live vehicle test, showing vehicle behavior and prediction.

In virtual environment, I can design tests by manually create testing scenarios by selecting location, add physical elements, and set their behaviors.

In virtual environment, I can design tests by manually create testing scenarios by selecting location, add physical elements, and set their behaviors.

Miscellaneous support

Since we were a relatively small team at the time, many of us were multiple hats based on business needs. Outside of my normal tasks I was involved in user testing, industrial design, prototype construction, filming, etc.

Conducting internal “dogfood“ testing with Google employees

Conducting internal “dogfood“ testing with Google employees

Taking part in filming for promotional videos.

Disclaimer: Most of the software screen shots are borrowed from This Atlantic article. which has Google’s blessing for public release.