Detecting sensor degradation by actively controlling an autonomous vehicle

• US 9,927,813 B1
• Filed: 01/25/2017
• Issued: 03/27/2018
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Ferguson, David I., Zhu, Jiajun
• Primary Examiner(s): Ingram, Thomas
• Application Number:
  US15/415,037
• Time in Patent Office: 426 Days
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60W 10/06 including control of combustion engines
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/14 Adaptive cruise control
• CPC Class Codes: B60W 50/0205 : Diagnosing or detecting failures; Failure detection models
• CPC Class Codes: B60W 50/029 : Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
• CPC Class Codes: B60W 60/00186 : related to the vehicle
• CPC Class Codes: B60W 60/0053 : from vehicle to occupant
• CPC Class Codes: B60W 60/0059 : Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
• CPC Class Codes: G01C 21/28 : with correlation of data from several navigational instruments
• CPC Class Codes: G01S 13/58 : Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/931 : of land vehicles
• CPC Class Codes: G01S 17/66 : Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 : Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 : Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 2007/4975 : of sensor obstruction by, e.g. dirt- or ice-coating, e.g. by reflection measurement on front-screen
• CPC Class Codes: G01S 7/4004 : of parts of a radar system
• CPC Class Codes: G01S 7/4039 : of sensor or antenna obstruction, e.g. dirt- or ice-coating
• CPC Class Codes: G01S 7/497 : Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4972 : Alignment of sensor
• CPC Class Codes: G01S 7/52004 : Means for monitoring or calibrating short-range imaging G01S7/5205
• CPC Class Codes: G05D 1/00 : Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 : extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0255 : using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/0257 : using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0274 : using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

Methods and systems are disclosed for determining sensor degradation by actively controlling an autonomous vehicle. Determining sensor degradation may include obtaining sensor readings from a sensor of an autonomous vehicle, and determining baseline state information from the obtained sensor readings. A movement characteristic of the autonomous vehicle, such as speed or position, may then be changed. The sensor may then obtain additional sensor readings, and second state information may be determined from these additional sensor readings. Expected state information may be determined from the baseline state information and the change in the movement characteristic of the autonomous vehicle. A comparison of the expected state information and the second state information may then be performed. Based on this comparison, a determination may be made as to whether the sensor has degraded.

Claims

1. An apparatus, comprising:

a computer-readable memory storing map information for a driving environment of an autonomous vehicle;

a laser sensor unit configured to detect a static object in the driving environment of the autonomous vehicle; and

one or more processors, in communication with the laser sensor unit and the computer-readable memory, the one or more processors configured to;

receive readings from the laser sensor unit relating to the static object, the readings including recorded laser points and corresponding received intensity values;

compare the received intensity values of the recorded laser points with stored intensity values corresponding to map laser points of the stored map information;

determine, based on the comparison, one or more deviation values; and

determine, based on the deviation values, whether the laser sensor unit has degraded.

2. The apparatus of claim 1, wherein the received readings include geometry readings.

3. The apparatus of claim 2, wherein comparing the recorded laser points with the map laser points comprises comparing position information.

4. The apparatus of claim 1, wherein in determining the one or more deviation values, the one or more processors are configured to determine a number of recorded laser points that are different from the map laser points associated with the detailed map.

5. The apparatus of claim 1, wherein the one or more processors are further configured to:

compare the deviation values with one or more corresponding deviation thresholds values; and

determine that the laser sensor has degraded when the deviation values meet or exceed the one or more deviation threshold values.

6. The apparatus of claim 1, wherein the one or more processors are configured to perform the comparing of the recorded laser points with map laser points of the stored map information, determining the one or more deviation values, and determining whether the laser sensor unit has degraded separately for individual laser beams.

7. The apparatus of claim 6, wherein the one or more processors are further configured to perform the comparing of the recorded laser points with map laser points of the stored map information, determining the one or more deviation values, and determining whether the laser sensor unit has degraded separately for the laser sensor unit as a whole.

8. A method, comprising:

receiving, with one or more processors, readings from a laser sensor unit, the readings relating to a static object in a driving environment of an autonomous vehicle, the readings including recorded laser points and corresponding received intensity values;

accessing a memory storing map information for the driving environment of the autonomous vehicle;

comparing, with the one or more processors, the received intensity values of the recorded laser points with stored intensity values corresponding to map laser points of the stored map information;

determining, with the one or more processors, based on the comparison, one or more deviation values; and

determining, with the one or more processors, based on the deviation values, whether the laser sensor unit has degraded.

9. The method of claim 8, wherein the received readings include geometry readings.

10. The method of claim 9, wherein comparing the recorded laser points with the map laser points comprises comparing position information.

11. The method of claim 8, wherein determining the one or more deviation values comprises determining a number of recorded laser points that are different from the map laser points associated with the detailed map.

12. The method of claim 8, further comprising:

comparing the deviation values with one or more corresponding deviation thresholds values; and

determining that the laser sensor unit has degraded when the deviation values meet or exceed the one or more deviation threshold values.

13. The method of claim 8, wherein comparing the recorded laser points with map laser points of the stored map information, determining the one or more deviation values, and determining whether the laser sensor unit has degraded is performed separately for individual laser beams.

14. The method of claim 13, further comprising comparing the recorded laser points with map laser points of the stored map information, determining the one or more deviation values, and determining whether the laser sensor unit has degraded for the laser sensor unit as a whole.

15. An autonomous vehicle, comprising:

a laser sensor unit configured to detect a static object in a driving environment of the autonomous vehicle; and

one or more processors, in communication with the laser sensor unit, the one or more processors configured to;

receive readings from the laser sensor unit relating to the static object, the readings including recorded laser points;

receive map information for the driving environment of the autonomous vehicle, the map information including map laser points and corresponding received intensity values;

compare the received intensity values of the recorded laser points with stored intensity values corresponding to map laser points;

determine, based on the comparison, one or more deviation values; and

determine, based on the deviation values, whether the laser sensor unit has degraded.

16. The autonomous vehicle of claim 15, wherein the received readings include geometry readings, and wherein comparing the recorded laser points with the map laser points comprises comparing position information.

17. The autonomous vehicle of claim 15, wherein in determining the one or more deviation values, the one or more processors are configured to determine a number of recorded laser points that are different from the map laser points associated with the detailed map.