Predictive sensor array configuration system for an autonomous vehicle

• US 10,220,852 B2
• Filed: 09/01/2017
• Issued: 03/05/2019
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Original Assignee: Uber Technologies, Inc.
• Inventors: Valois, Jean-Sebastien
• Primary Examiner(s): Brushaber, Frederick M
• Application Number:
  US15/694,493
• Publication Number:
  US 20180009441A1
• Time in Patent Office: 550 Days
• CPC Class Codes: B60W 40/02 related to ambient conditions
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/40 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/021 specially adapted to land vehicles
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04W 4/38 for collecting sensor information
• CPC Class Codes: H04W 4/48 for in-vehicle communication
• CPC Class Codes: H04W 84/18 Self-organising networks, e.g. ad-hoc networks or sensor networks
• CPC Class Codes: Y04S 40/18 Network protocols supporting networked applications, e.g. including control of end-device applications over a network

Abstract

An autonomous vehicle (AV) can include a set of sensors generating sensor data corresponding to a surrounding environment of the AV. The AV can further include a control system that determines imminent lighting conditions for one or more cameras of the set of sensors, and executes a set of configurations for the one or more cameras to preemptively compensate for the imminent lighting conditions.

Claims

1. An autonomous vehicle (AV) comprising:

a set of sensors generating sensor data corresponding to a surrounding environment of the AV;

a control system comprising one or more processors executing an instruction set, causing the control system to;

based on the sensor data generated by the set of sensors, determine imminent lighting conditions for one or more cameras of the set of sensors; and

execute a set of configurations for the one or more cameras to preemptively compensate for the imminent lighting conditions.

2. The AV of claim 1, wherein the executed instructions further cause the control system to:

store a set of sub-maps comprising recorded surface data of a given region upon which the AV operates;

wherein the executed instructions cause the control system to determine the imminent lighting conditions by analyzing a current sub-map from the stored set of sub-maps.

3. The AV of claim 2, wherein the stored set of sub-maps identify road features that affect the imminent lighting conditions, the road features comprising at least one of a tunnel, an overpass, or proximate buildings.

4. The AV of claim 2, wherein each sub-map in the set of sub-maps comprises at least one of recorded LIDAR data or recorded image data.

5. The AV of claim 1, further comprising:

acceleration, braking, and steering systems;

wherein the executed instructions further cause the control system to;

dynamically process the sensor data from the set of sensors to autonomously operate the acceleration, braking, and steering systems along a current route;

wherein the executed instructions cause the control system to determine the imminent lighting conditions and execute the set of configurations dynamically as the AV travels along the current route.

6. The AV of claim 1, wherein the executed instructions further cause the control system to:

dynamically determine a brightness differential between current lighting conditions and the imminent lighting conditions;

wherein executed instructions cause the control system to determine the set of configurations based on the brightness differential.

7. The AV of claim 1, wherein executing the set of configurations for the one or more cameras comprises preemptively adjusting aperture settings of the one or more cameras based on the imminent lighting conditions.

8. The AV of claim 1, wherein the executed instructions cause the control system to determine the imminent lighting conditions by analyzing the sensor data from the set of sensors.

9. The AV of claim 1, wherein changes in the imminent lighting conditions correspond to at least one of shadows, lights, the Sun, or solar reflections.

10. The AV of claim 1, wherein the executed instructions cause the control system to determine the imminent light conditions by receiving data indicating the imminent lighting conditions from forward operating AVs in relation to the AV.

11. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of an autonomous vehicle (AV) cause the one or more processors to:

based on sensor data generated by one or more sensors of the AV, determine imminent lighting conditions for one or more cameras of a set of sensors of the AV, wherein the sensor data corresponds to a surrounding environment of the AV; and

execute a set of configurations for the one or more cameras to preemptively compensate for the imminent lighting conditions.

12. The non-transitory computer-readable medium of claim 11, wherein the executed instructions further cause the one or more processors to:

store a set of sub-maps comprising recorded surface data of a given region upon which the AV operates;

wherein the executed instructions cause the one or more processors to determine the imminent lighting conditions by analyzing a current sub-map from the stored set of sub-maps.

13. The non-transitory computer-readable medium of claim 12, wherein the stored set of sub-maps identify road features that affect the imminent lighting conditions, the road features comprising at least one of a tunnel, an overpass, or proximate buildings.

14. The non-transitory computer-readable medium of claim 12, wherein each sub-map in the set of sub-maps comprises at least one of recorded LIDAR data or recorded image data.

15. The non-transitory computer-readable medium of claim 11, wherein the AV further comprises acceleration, braking, and steering systems, and wherein the executed instructions further cause the one or more processors to:

dynamically process the sensor data from the set of sensors to autonomously operate the acceleration, braking, and steering systems along a current route;

wherein the executed instructions cause the one or more processors to determine the imminent lighting conditions and execute the set of configurations dynamically as the AV travels along the current route.

16. The non-transitory computer-readable medium of claim 11, wherein the executed instructions further cause the one or more processors to:

dynamically determine a brightness differential between current lighting conditions and the imminent lighting conditions;

wherein executed instructions cause the one or more processors to determine the set of configurations based on the brightness differential.

17. The non-transitory computer-readable medium of claim 11, wherein executing the set of configurations for the one or more cameras comprises preemptively adjusting aperture settings of the one or more cameras based on the imminent lighting conditions.

18. The non-transitory computer-readable medium of claim 11, wherein the executed instructions cause the one or more processors to determine the imminent lighting conditions by analyzing the sensor data from the set of sensors.

19. The non-transitory computer-readable medium of claim 11, wherein changes in the imminent lighting conditions correspond to at least one of shadows, lights, the Sun, or solar reflections.

20. A computer-implemented method of preemptively configuring sensors of an autonomous vehicle (AV), the method being performed by one or more processors of the AV and comprising:

based on sensor data generated by one or more sensors of the AV, determining imminent lighting conditions for one or more cameras of a set of sensors of the AV, wherein the sensor data corresponds to a surrounding environment of the AV; and

executing a set of configurations for the one or more cameras to preemptively compensate for the imminent lighting conditions.