Predictive sensor array configuration system for an autonomous vehicle

• US 9,840,256 B1
• Filed: 12/16/2015
• Issued: 12/12/2017
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated (Aurora Innovation, Inc.)
• Original Assignee: Uber Technologies, Inc.
• Inventors: Valois, Jean-Sebastien
• Primary Examiner(s): Brushaber, Frederick
• Application Number:
  US14/971,850
• Time in Patent Office: 727 Days
• CPC Class Codes: B60W 40/02 related to ambient conditions
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/40 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/021 specially adapted to land vehicles
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04W 4/38 for collecting sensor information
• CPC Class Codes: H04W 4/48 for in-vehicle communication
• CPC Class Codes: H04W 84/18 Self-organising networks, e.g. ad-hoc networks or sensor networks
• CPC Class Codes: Y04S 40/18 Network protocols supporting networked applications, e.g. including control of end-device applications over a network

Abstract

An autonomous vehicle (AV) can include a predictive sensor configuration system that can dynamically determine imminent lighting conditions for a sensor array of the AV as the AV travels a current route. The predictive sensor configuration system can dynamically determine one or more configurations for the sensor array to compensate for the imminent lighting conditions, and preemptively execute the one or more configurations for the sensor array as the AV travels the current route.

Claims

1. A predictive sensor array configuration system for an autonomous vehicle (AV) comprising:

one or more processors; and

one or more memory resources storing instructions that, when executed by the one or more processors, cause the predictive sensor array configuration system to;

as the AV travels a current route, dynamically determine, from a sensor array of the AV, imminent lighting conditions for one or more cameras of the sensor array;

dynamically determine one or more configurations for the one or more cameras to compensate for the imminent lighting conditions; and

preemptively execute the one or more configurations for the one or more cameras as the AV travels the current route to preemptively compensate for the imminent lighting conditions.

2. The predictive sensor array configuration system of claim 1, wherein the executed instructions further cause the predictive sensor array configuration system to:

maintain a lookup table comprising a plurality of sensor configurations for each respective one of a plurality of sensor systems included in the sensor array, wherein the one or more configurations are included in the plurality of sensor configurations.

3. The predictive sensor array configuration system of claim 2, wherein the executed instructions cause the predictive sensor array configuration system to dynamically determine the one or more configurations by performing a lookup in the lookup table based on the imminent lighting conditions.

4. The predictive sensor array configuration system of claim 1, wherein the executed instructions cause the predictive sensor array configuration system to dynamically determine the one or more configurations by performing an optimization utilizing a plurality of possible configurations for each of the one or more cameras based on the imminent lighting conditions, and wherein an output of the optimization comprises the one or more configurations.

5. The predictive sensor array configuration system of claim 1, wherein the executed instructions cause the predictive sensor array configuration system to dynamically determine the imminent light conditions for the sensor array by (i) receiving sensor data from one or more sensors of the sensor array, the sensor data from the one or more sensors indicating the current route traveled by the AV, and (ii) identifying a number of route features affecting the imminent lighting conditions in the current route using the sensor data from the one or more sensors.

6. The predictive sensor array configuration system of claim 5, wherein the route features comprise one or more of an overpass, a tunnel, or shadows across the current route.

7. The predictive sensor array configuration system of claim 5, wherein the route features comprise reflective light from one or more surface features along the current route, the reflective light affecting an operational performance of the one or more cameras of the sensor array.

8. The predictive sensor array configuration system of claim 1, wherein the executed instructions further cause the predictive sensor array configuration system to:

determine a current time of day;

wherein the executed instructions further cause the predictive sensor array configuration system to dynamically determine the one or more configurations for the one or more cameras based on the current time of day.

9. The predictive sensor array configuration system of claim 1, wherein the executed instructions further cause the predictive sensor array configuration system to:

maintain a sub-map database comprising three-dimensional (3D) surface data of an operational region of the AV; and

identify, using a current position of the AV, a correlated sub-map from the sub-map database that provides 3D surface data surrounding the current position of the AV;

wherein the executed instructions cause the predictive sensor array configuration system to dynamically determine the imminent lighting conditions for the one or more cameras by comparing current sensor data from the sensor array with lighting conditions provided by the correlated sub-map.

10. The predictive sensor array configuration system of claim 9, wherein the current sensor data is received via the one or more cameras of the sensor array, and wherein the one or more preemptively executed configurations include executed aperture settings for the one or more cameras as the AV travels along the current route.

11. The predictive sensor array configuration system of claim 9, wherein the sensor array provides the current sensor data to an on-board data processing system of the AV to enable the AV to maneuver through road traffic along the current route.

12. The predictive sensor array configuration system of claim 11, wherein the on-board data processing system dynamically compares the current sensor data from the sensor array to a respective current sub-map stored in the sub-map database to maneuver through road traffic along the current route.

13. An autonomous vehicle (AV) comprising:

a sensor array to continuously detect a situational environment of the AV and generate sensor data indicating the situational environment;

an on-board data processing system to process the sensor data by comparing the sensor data to stored sub-maps that provide detailed surface data of a current route traveled by the AV;

an acceleration, braking, and steering system;

an AV control system to utilize the processed sensor data to operate the acceleration, braking, and steering system along the current route through road traffic; and

a predictive sensor array configuration system to;

as the AV travels the current route, dynamically determine imminent lighting conditions for the sensor array of the AV;

dynamically determine one or more configurations for the sensor array to compensate for the imminent lighting conditions; and

preemptively execute the one or more configurations for the sensor array as the AV travels the current route.

14. The AV of claim 13, further comprising:

a memory resource to store the sub-maps and a lookup table comprising a plurality of sensor configurations for each respective one of a plurality of sensor systems included in the sensor array, wherein the one or more configurations are included in the plurality of sensor configurations.

15. The AV of claim 14, wherein the predictive sensor array configuration system dynamically determines the one or more configurations by performing a lookup in the lookup table based on the imminent lighting conditions.

16. The AV of claim 13, wherein the predictive sensor array configuration system dynamically determines the one or more configurations by performing an optimization utilizing a plurality of possible configurations for each of the plurality of sensor systems based on the imminent lighting conditions, and wherein an output of the optimization comprises the one or more configurations.

17. A non-transitory computer readable medium storing instructions for predictively configuring a sensor array of an autonomous vehicle (AV), wherein the instructions, when executed by one or more processors of the AV, cause the one or more processors to:

as the AV travels a current route, dynamically determine, from the sensor array, imminent lighting conditions for one or more cameras of the sensor array;

dynamically determine one or more configurations for the one or more cameras to compensate for the imminent lighting conditions; and

preemptively execute the one or more configurations for the one or more cameras as the AV travels the current route to preemptively compensate for the imminent lighting conditions.

18. The non-transitory computer readable medium of claim 17, wherein the executed instructions further cause the one or more processors to:

maintain a lookup table comprising a plurality of sensor configurations for each respective one of a plurality of sensor systems included in the sensor array, wherein the one or more configurations are included in the plurality of sensor configurations.

19. The non-transitory computer readable medium of claim 18, wherein the executed instructions cause the one or more processors to dynamically determine the one or more configurations by performing a lookup in the lookup table based on the imminent lighting conditions.

20. The non-transitory computer readable medium of claim 17, wherein the executed instructions cause the one or more processors to dynamically determine the one or more configurations by performing an optimization utilizing a plurality of possible configurations for each of the one or more cameras based on the imminent lighting conditions, and wherein an output of the optimization comprises the one or more configurations.