System to automatically measure perception sensor latency in an autonomous vehicle

• US 8,838,322 B1
• Filed: 08/14/2012
• Issued: 09/16/2014
• Est. Priority Date: 08/14/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Ferguson, David I.
• Primary Examiner(s): Algahaim, Helal A
• Assistant Examiner(s): PATEL, SHARDUL D
• Application Number:
  US13/585,486
• Time in Patent Office: 763 Days
• Field of Search: 701/1, 701/26, 348/46
• US Class Current: 701/26
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/10 Path keeping cruise control for automatically following a preceding vehicle B60W30/165
• CPC Class Codes: B60W 30/143 Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: G01C 22/00 Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4004 of parts of a radar system
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931

Abstract

A vehicle may operate in an autonomous mode in an environment during a test period. The vehicle may include at least one sensor coupled to the vehicle, configured to acquire sensor data during the test period. The sensor data may include data representative of a target object in the environment. The vehicle may operate the sensor to obtain the sensor data. The vehicle may define a movement of the vehicle, determine a predicted movement of the target object in the sensor data based on the defined movement, initiate the defined movement of the vehicle at an initiation time during the test period, complete the defined movement of the vehicle at a completion time during the test period, analyze the sensor data obtained during the test period, and determine a latency of the at least one sensor based on the analyzed data.

Claims

1. A method comprising:

obtaining, during a test period, sensor data from at least one sensor of a vehicle, wherein the vehicle is operating in an autonomous mode controlled by a computer system during the test period, wherein the sensor data includes data representative of a target object in an environment of the vehicle, and wherein the vehicle has a first pose in the environment at a beginning of the test period;

defining a movement of the vehicle from the first pose to a second pose;

determining a predicted movement of the at least one target object in the sensor data based on the defined movement of the vehicle;

initiating the defined movement of the vehicle at an initiation time during the test period;

completing the defined movement of the vehicle at a completion time during the test period;

analyzing the sensor data obtained during the test period to identify at least one of (i) a start time when the at least one target object begins the predicted movement in the sensor data and (ii) a stop time when the at least one target object stops the predicted movement in the sensor data; and

determining a latency of the at least one sensor based on at least one of (i) a difference between the start time and the initiation time and (ii) a difference between the stop time and the completion time.

2. The method of claim 1, wherein the first pose and the second pose comprise at least one of a speed, a heading, an acceleration, a deceleration, and a yaw rate.

3. The method of claim 2, wherein defining the movement of the vehicle from the first pose to the second pose comprises defining one or more of the speed, the heading, the acceleration, the deceleration, and the yaw rate.

4. The method of claim 3, wherein initiating the defined movement of the vehicle at the initiation time during the test period comprises automatically changing one or more of the speed to be the defined speed, the heading to be the defined heading, the acceleration to be the defined acceleration, the deceleration to be the defined deceleration, and the yaw rate to be the defined yaw rate.

5. The method of claim 2, wherein

defining the movement of the vehicle from the first pose to a second pose comprises defining the heading;

determining the predicted movement of the at least one target object in the sensor data based on the defined movement of the vehicle comprises determining the predicted movement of the at least one target object in the sensor data to be a lateral movement in a direction opposite to the defined heading of the vehicle; and

initiating the defined movement of the vehicle at the initiation time during the test period comprises automatically changing the heading of the vehicle to the defined heading.

6. The method of claim 5, wherein the at least one sensor comprises a laser rangefinder.

7. The method of claim 5, wherein the at least one sensor comprises a camera.

8. The method of claim 5, wherein the at least one sensor comprises a radar.

9. A vehicle comprising:

at least one sensor, coupled to the vehicle, configured to acquire sensor data during a test period, wherein the vehicle is configured to operate in an autonomous mode during the test period, wherein the sensor data includes data representative of a target object in an environment of the vehicle, and wherein the vehicle has a first pose in the environment at a beginning of the test period; and

a computer system, wherein the computer system is configured to;

operate the at least one sensor and the vehicle in the autonomous mode to obtain the sensor data;

define a movement of the vehicle from the first pose to a second pose;

determine a predicted movement of the at least one target object in the sensor data based on the defined movement of the vehicle;

initiate the defined movement of the vehicle at an initiation time during the test period;

complete the defined movement of the vehicle at a completion time during the test period;

analyze the sensor data obtained during the test period to identify at least one of (i) a start time when the at least one target object begins the predicted movement in the sensor data and (ii) a stop time when the at least one target object stops the predicted movement in the sensor data; and

determine a latency of the at least one sensor based on at least one of (i) a difference between the start time and the initiation time and (ii) a difference between the stop time and the completion time.

10. The vehicle of claim 9, wherein the first pose and the second pose comprise at least one of a speed, a heading, an acceleration, a deceleration, and a yaw rate.

11. The vehicle of claim 10, wherein defining the movement of the vehicle from the first pose to the second pose comprises defining one or more of the speed, the heading, the acceleration, the deceleration, and the yaw rate.

12. The vehicle of claim 11, wherein initiating the defined movement of the vehicle at the initiation time during the test period comprises automatically changing one or more of the speed to be the defined speed, the heading to be the defined heading, the acceleration to be the defined acceleration, the deceleration to the defined deceleration, and the yaw rate to the defined yaw rate.

13. The vehicle of claim 10, wherein

defining the movement of the vehicle from the first pose to a second pose comprises defining the heading;

determining the predicted movement of the at least one target object in the sensor data based on the defined movement of the vehicle comprises determining the predicted movement of the at least one target object in the sensor data to be a lateral movement in a direction opposite to the defined heading of the vehicle; and

initiating the defined movement of the vehicle at the initiation time during the test period comprises automatically changing the heading of the vehicle to the defined heading.

14. The vehicle of claim 13, wherein the at least one sensor comprises a laser rangefinder, a camera, or a radar.

15. A non-transitory computer readable medium having stored therein instructions executable by a computer system to cause the computer system to perform functions comprising:

obtaining, during a test period, sensor data from at least one sensor of a vehicle, wherein the vehicle is operating in an autonomous mode during the test period, wherein the sensor data includes data representative of a target object in an environment of the vehicle, and wherein the vehicle has a first pose in the environment at a beginning of the test period;

defining a movement of the vehicle from the first pose to a second pose;

determining a predicted movement of the at least one target object in the sensor data based on the defined movement of the vehicle;

initiating the defined movement of the vehicle at an initiation time during the test period;

completing the defined movement of the vehicle at a completion time during the test period;

analyzing the sensor data obtained during the test period to identify at least one of (i) a start time when the at least one target object begins the predicted movement in the sensor data and (ii) a stop time when the at least one target object stops the predicted movement in the sensor data; and

determining a latency of the at least one sensor based on at least one of (i) a difference between the start time and the initiation time and (ii) a difference between the stop time and the completion time.

16. The non-transitory computer readable medium of claim 15, wherein the first pose and the second pose comprise at least one of a speed, a heading, an acceleration, a deceleration, and a yaw rate.

17. The non-transitory computer readable medium of claim 16, wherein the instructions are further executable by the computing device to perform functions comprising defining one or more of the speed, the heading, the acceleration, the deceleration, and the yaw rate.

18. The non-transitory computer readable medium of claim 17, wherein the instructions are further executable by the computing device to perform functions comprising automatically changing one or more of the speed to be the defined speed, the heading to be the defined heading, the acceleration to be the defined acceleration, the deceleration to be the defined deceleration, and the yaw rate to be the defined yaw rate.

19. The non-transitory computer readable medium of claim 16, wherein the instructions are further executable by the computing device to perform functions comprising:

defining the heading;

determining the predicted movement of the at least one target object in the sensor data to be a lateral movement in a direction opposite to the defined heading of the vehicle; and

automatically changing the heading of the vehicle to the defined heading.

20. The non-transitory computer readable medium of claim 19, wherein the at least one sensor comprises a laser rangefinder, a camera, or a radar.