Detecting sensor degradation by actively controlling an autonomous vehicle

• US 9,274,525 B1
• Filed: 09/28/2012
• Issued: 03/01/2016
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I., Zhu, Jiajun
• Primary Examiner(s): Cheung, Calvin
• Assistant Examiner(s): Ingram, Thomas
• Application Number:
  US13/630,054
• Time in Patent Office: 1,250 Days
• Field of Search: 701/300, 701/301, 701/302, 701 292- 334
• US Class Current: 1/1
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60W 10/06 including control of combustion engines
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/14 Adaptive cruise control
• CPC Class Codes: B60W 50/0205 : Diagnosing or detecting failures; Failure detection models
• CPC Class Codes: B60W 50/029 : Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
• CPC Class Codes: B60W 60/00186 : related to the vehicle
• CPC Class Codes: B60W 60/0053 : from vehicle to occupant
• CPC Class Codes: B60W 60/0059 : Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
• CPC Class Codes: G01C 21/28 : with correlation of data from several navigational instruments
• CPC Class Codes: G01S 13/58 : Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/931 : of land vehicles
• CPC Class Codes: G01S 17/66 : Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 : Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 : Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 2007/4975 : of sensor obstruction by, e.g. dirt- or ice-coating, e.g. by reflection measurement on front-screen
• CPC Class Codes: G01S 7/4004 : of parts of a radar system
• CPC Class Codes: G01S 7/4039 : of sensor or antenna obstruction, e.g. dirt- or ice-coating
• CPC Class Codes: G01S 7/497 : Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4972 : Alignment of sensor
• CPC Class Codes: G01S 7/52004 : Means for monitoring or calibrating short-range imaging G01S7/5205
• CPC Class Codes: G05D 1/00 : Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 : extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0255 : using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/0257 : using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0274 : using mapping information stored in a memory device navigation using map-matching G01C21/30

Abstract

Methods and systems are disclosed for determining sensor degradation by actively controlling an autonomous vehicle. Determining sensor degradation may include obtaining sensor readings from a sensor of an autonomous vehicle, and determining baseline state information from the obtained sensor readings. A movement characteristic of the autonomous vehicle, such as speed or position, may then be changed. The sensor may then obtain additional sensor readings, and second state information may be determined from these additional sensor readings. Expected state information may be determined from the baseline state information and the change in the movement characteristic of the autonomous vehicle. A comparison of the expected state information and the second state information may then be performed. Based on this comparison, a determination may be made as to whether the sensor has degraded.

Claims

1. An apparatus comprising:

a computer-readable memory that stores a change in a movement characteristic of an autonomous vehicle;

a sensor configured to detect an object in a driving environment of the autonomous vehicle; and

one or more processors, in communication with the sensor and the computer-readable memory, the one or more processors configured to;

receive a first sensor reading from the sensor based on a first detection of the object in the driving environment;

determine first state information for the object based on the received first sensor reading;

change the movement characteristic of the autonomous vehicle based on the stored change in the movement characteristic of the autonomous vehicle;

determine expected state information based on the first state information and the change in the movement characteristic of the autonomous vehicle;

receive a second sensor reading from the sensor based on a second detection of the object in the driving environment;

determine second state information for the object based on the received second sensor reading;

determine sensor degradation in the sensor by comparing the second state information with the expected state information;

determine a level of severity of the sensor degradation based on one or more types of state information, wherein in determining the level of severity the one or more processors are further configured to;

determine a deviation value from comparing the second state information with the expected state information;

compare the determined deviation value with a deviation threshold value; and

determine at least one of a magnitude by which the determined deviation value exceeds the deviation threshold value, and a number of deviation thresholds that are exceeded for different types of state information;

determine whether the level of severity of the sensor degradation meets a first predetermined level or a second predetermined level;

perform a first action if the severity of the sensor degradation is at the first level; and

perform a different action if the severity of the sensor degradation is at the second level.

2. The apparatus of claim 1, wherein the change in the movement characteristic of the autonomous vehicle comprises one of changing the autonomous vehicle'"'"'s position, changing the autonomous vehicle'"'"'s speed, or changing the autonomous vehicle'"'"'s heading.

3. The apparatus of claim 1, wherein the sensor comprises one of a camera, a radar detection unit, or a laser sensor.

4. The apparatus of claim 1, wherein the processor is configured to change the movement characteristic of the autonomous vehicle by increasing a speed of the autonomous vehicle by a predetermined amount.

5. The apparatus of claim 1, wherein the one or more processors are configured to change the movement characteristic of the autonomous vehicle by changing a heading of the autonomous vehicle.

6. The apparatus of claim 1, wherein the first state information comprises a distance from the object relative to the autonomous vehicle.

7. The apparatus of claim 1, wherein the one or more processors are further configured to:

correlate the determined sensor degradation with map information;

determine where or when the sensor began degrading based on the correlation; and

determine, based on the map information, whether environmental factors affected operation of the sensor.

8. The apparatus of claim 1, wherein the sensor is a laser sensor, and wherein:

receiving the first sensor reading comprises receiving a first position and a first intensity of a laser point on the object;

determining expected state information comprises determining an expected position and an expected intensity;

receiving the second sensor reading comprises receiving a second position and a second intensity of the laser point; and

determining the sensor degradation in the sensor includes comparing at least the expected position and the expected intensity with the second position and the second intensity.

9. The apparatus of claim 1, wherein the sensor is a radar sensor, and wherein:

receiving the first sensor reading comprises receiving baseline information for other vehicles relative to the autonomous vehicle;

determining expected state information comprises determining expected information for other vehicles relative to the autonomous vehicle;

receiving the second sensor reading comprises receiving second information for other vehicles relative to the autonomous vehicle; and

determining the sensor degradation in the sensor includes comparing at the expected information with the second information relative to the autonomous vehicle.

10. A method comprising:

receiving, with one or more processors in communication with a sensor, a first sensor reading from the based on a first detection of an object in a driving environment;

determining, with the one or more processors, first state information for the object based on the received first sensor reading;

changing a movement characteristic of the autonomous vehicle based on the stored change in the movement characteristic of the autonomous vehicle;

determining, with the one or more processors, expected state information based on the first state information and the change in the movement characteristic of the autonomous vehicle;

receiving, with the one or more processors, a second sensor reading from the sensor based on a second detection of the object in the driving environment;

determining, with the one or more processors, second state information for the object based on the received second sensor reading;

determining, with the one or more processors, sensor degradation in the sensor by comparing the second state information with the expected state information;

determining, with the one or more processors, a level of severity of the sensor degradation based on one or more types of state information, wherein determining the level of severity comprises;

determining a deviation value from comparing the second state information with the expected state information;

comparing the determined deviation value with a deviation threshold value; and

determining at least one of a magnitude by which the determined deviation value exceeds the deviation threshold value, and a number of deviation thresholds that are exceeded for different types of state information;

determining, with the one or more processors, whether the level of severity of the sensor degradation meets a first predetermined level or a second predetermined level;

performing, with the one or more processors, a first action if the severity of the sensor degradation is at the first level; and

performing, with the one or more processors, a different action if the severity of the sensor degradation is at the second level.

11. The method of claim 10, wherein the change in the movement characteristic of the autonomous vehicle comprises one of changing the autonomous vehicle'"'"'s position, changing the autonomous vehicle'"'"'s speed, or changing the autonomous vehicle'"'"'s heading.

12. The method of claim 10, wherein the sensor comprises one of a camera, a radar detection unit, or a laser sensor.

13. The method of claim 10, further comprising:

changing the movement characteristic of the autonomous vehicle by increasing a speed of the autonomous vehicle by a predetermined amount.

14. The method of claim 10, further comprising:

changing the movement characteristic of the autonomous vehicle by changing a heading of the autonomous vehicle.

15. The method of claim 10, wherein the first state information comprises a distance from the object relative to the autonomous vehicle.

16. The method of claim 10, further comprising:

correlating the determined sensor degradation with map information;

determining where or when the sensor began degrading based on the correlation; and

determining, based on the map information, whether environmental factors affected operation of the sensor.