Controlling vehicle sensors based on road configuration

• US 10,509,127 B2
• Filed: 10/31/2018
• Issued: 12/17/2019
• Est. Priority Date: 12/13/2017
• Status: Active Grant

• Current Assignee: Luminar Technologies, Inc.
• Original Assignee: Luminar Technologies, Inc.
• Inventors: Englard, Benjamin, Danziger, Eric C., Augenbraun, Joseph, Russell, Austin K.
• Primary Examiner(s): Lang, Michael D
• Application Number:
  US16/176,607
• Publication Number:
  US 20190179025A1
• Time in Patent Office: 412 Days
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/15 Road slope, i.e. the inclination of a road segment in the longitudinal direction
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2555/40 Altitude
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 2720/12 Lateral speed
• CPC Class Codes: B60W 2720/125 Lateral acceleration
• CPC Class Codes: B60W 30/18145 Cornering
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 50/06 Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 13/426 Scanning radar, e.g. 3D radar G01S13/66 takes precedence
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 13/931 : of land vehicles
• CPC Class Codes: G01S 17/66 : Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 : Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 : of land vehicles
• CPC Class Codes: G01S 2013/9323 : Alternative operation using light waves
• CPC Class Codes: G01S 2013/9324 : Alternative operation using ultrasonic waves
• CPC Class Codes: G01S 7/40 : Means for monitoring or calibrating
• CPC Class Codes: G01S 7/4026 : Antenna boresight
• CPC Class Codes: G01S 7/403 : in azimuth, i.e. in the horizontal plane
• CPC Class Codes: G01S 7/4034 : in elevation, i.e. in the vertical plane
• CPC Class Codes: G01S 7/417 : involving the use of neural networks
• CPC Class Codes: G01S 7/4808 : Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/4817 : relating to scanning
• CPC Class Codes: G01S 7/4972 : Alignment of sensor
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 : involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0221 : involving a learning process
• CPC Class Codes: G05D 1/0231 : using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0248 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0257 : using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G06F 18/214 : Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2163 : Partitioning the feature space
• CPC Class Codes: G06F 18/217 : Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24 : Classification techniques
• CPC Class Codes: G06F 18/24143 : Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06N 3/045 : Combinations of networks
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06N 3/086 : using evolutionary algorithms, e.g. genetic algorithms or genetic programming
• CPC Class Codes: G06N 5/01 : Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06T 15/08 : Volume rendering
• CPC Class Codes: G06T 17/05 : Geographic models
• CPC Class Codes: G06T 17/20 : Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 : Training; Learning
• CPC Class Codes: G06T 2207/20084 : Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4007 : based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/50 : Depth or shape recovery
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 10/776 : Validation; Performance evaluation
• CPC Class Codes: G06V 10/82 : using neural networks
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/59 : inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions

Abstract

A method for controlling a first sensor configured to sense an environment through which a vehicle is moving includes receiving sensor data generated by one or more sensors of the vehicle as the vehicle moves through the environment, identifying, by one or more processors and based on at least a portion of the received sensor data, one or more road portions along which the vehicle is expected to travel, and determining, by one or more processors, a configuration of the identified road portions, at least in part by determining a slope of at least one of the identified road portions. The method also includes determining, by one or more processors analyzing at least the determined configuration, an elevation of a field of regard of the first sensor that satisfies one or more visibility criteria, and causing the first sensor to be adjusted in accordance with the determined elevation.

Claims

1. A method for controlling a first sensor configured to sense an environment through which a vehicle is moving, the method comprising:

receiving sensor data generated by one or more sensors of the vehicle as the vehicle moves through the environment;

identifying, by one or more processors and based on at least a portion of the received sensor data, one or more road portions along which the vehicle is expected to travel;

determining, by one or more processors, a configuration of the identified road portions, at least in part by determining a slope of at least one of the identified road portions;

determining, by one or more processors analyzing at least the determined configuration, an elevation angle of a center of a field of regard of the first sensor that satisfies one or more visibility criteria;

causing the first sensor to be adjusted in accordance with the determined elevation angle;

determining, by one or more processors, that, at the determined elevation angle, a sensing distance of the first sensor is less than a threshold distance; and

in response to determining that the sensing distance is less than the threshold distance, causing, by one or more processors, a speed of the vehicle to decrease.

2. The method of claim 1, wherein determining the elevation angle of the center of the field of regard that satisfies one or more visibility criteria includes:

determining the elevation angle of the center of the field of regard that maximizes the sensing distance of the first sensor in a direction along which the vehicle is expected to travel.

3. The method of claim 1, wherein the first sensor is a lidar device included in the one or more sensors, and wherein the sensor data includes a point cloud.

4. The method of claim 3, wherein identifying one or more road portions includes:

partitioning the point cloud into a plurality of portions in accordance with probable boundaries between separate physical objects; and

classifying one or more of the plurality of portions as the one or more road portions.

5. The method of claim 1, further comprising:

determining, by one or more processors analyzing at least the determined configuration, a horizontal position of the field of regard of the first sensor.

6. The method of claim 1, wherein the one or more sensors include two or more selected from the group consisting of (i) an inertial measurement unit, (ii) a lidar device, and (iii) a camera.

7. The method of claim 1, wherein:

the one or more sensors include an inertial measurement unit;

the method further includes determining, by one or more processors analyzing a portion of the sensor data corresponding to the inertial measurement unit, a slope of a road portion on which the vehicle is currently moving; and

determining the elevation angle of the center of the field of regard of the first sensor further includes one or more processors analyzing the slope of the road portion on which the vehicle is currently moving.

8. The method of claim 1, wherein causing the first sensor to be adjusted in accordance with the determined elevation angle comprises causing one or more mirrors of the first sensor to move the center of the field of regard to the determined elevation angle.

9. A non-transitory computer-readable medium storing thereon instructions executable by one or more processors to implement a sensor control architecture for controlling at least a first sensor of one or more sensors that are configured to sense an environment through which a vehicle is moving, the sensor control architecture comprising:

a perception component configured to;

receive sensor data generated by the one or more sensors as the vehicle moves through the environment, and

generate, based on the received sensor data, signals descriptive of a current state of the environment; and

a sensor control component configured to;

identify, based on the signals descriptive of the current state of the environment, one or more road portions along which the vehicle is expected to travel,

determine a configuration of the identified road portions, at least in part by determining a slope of at least one of the identified road portions,

determine, by analyzing at least the determined configuration, an elevation angle of a center of a field of regard of the first sensor that satisfies one or more visibility criteria,

cause the first sensor to be adjusted in accordance with the determined elevation angle,

determine that, at the determined elevation angle, a sensing distance of the first sensor is less than a threshold distance, and

in response to determining that the sensing distance is less than the threshold distance, cause a speed of the vehicle to decrease.

10. The non-transitory computer-readable medium of claim 9, wherein the one or more visibility criteria include one or more criteria that maximize the sensing distance of the first sensor in a direction along which the vehicle is expected to travel.

11. The non-transitory computer-readable medium of claim 9, wherein the first sensor is a lidar device included in the one or more sensors, and wherein the sensor data includes a point cloud.

12. The non-transitory computer-readable medium of claim 11, wherein the sensor control component is configured to identify the one or more road portions at least by:

partitioning the point cloud into a plurality of portions in accordance with probable boundaries between separate physical objects; and

classifying one or more of the plurality of portions as the one or more road portions.

13. The non-transitory computer-readable medium of claim 9, wherein the sensor control component is further configured to determine, by analyzing at least the determined configuration, a horizontal position of the field of regard of the first sensor.

14. The non-transitory computer-readable medium of claim 9, wherein:

the one or more sensors include an inertial measurement unit; and

the sensor control component is further configured to;

determine, by analyzing a portion of the sensor data corresponding to the inertial measurement unit, a slope of a road portion on which the vehicle is currently moving, and

determine the elevation angle of the center of the field of regard of the first sensor by also analyzing the slope of the road portion on which the vehicle is currently moving.

15. The non-transitory computer-readable medium of claim 9, wherein the one or more sensors include two or more selected from the group consisting of (i) an inertial measurement unit, (ii) a lidar device, and (iii) a camera.

16. A vehicle comprising:

a first sensor configured to sense an environment through which the vehicle is moving; and

a computing system configured to;

receive sensor data generated by one or more sensors of the vehicle as the vehicle moves through the environment,

identify, based on at least a portion of the received sensor data, one or more road portions along which the vehicle is expected to travel,

determine a configuration of the identified road portions, at least in part by determining a slope of at least one of the identified road portions,

determine, by analyzing at least the determined configuration, an elevation angle of a center of a field of regard of the first sensor that satisfies one or more visibility criteria,

cause the first sensor to be adjusted in accordance with the determined elevation angle,

determine that, at the determined elevation angle, a sensing distance of the first sensor is less than a threshold distance, and

in response to determining that the sensing distance is less than the threshold distance, cause a speed of the vehicle to decrease.

17. The vehicle of claim 16, wherein the one or more visibility criteria include one or more criteria that maximize the sensing distance of the first sensor in a direction along which the vehicle is expected to travel.

18. The vehicle of claim 16, wherein:

the one or more sensors include an inertial measurement unit; and

the computing system is further configured to;

determine, by analyzing a portion of the sensor data corresponding to the inertial measurement unit, a slope of a road portion on which the vehicle is currently moving, and

determine the elevation angle of the center of the field of regard of the first sensor by also analyzing the slope of the road portion on which the vehicle is currently moving.

19. The vehicle of claim 16, wherein the first sensor is a lidar device included in the one or more sensors, and wherein the sensor data includes a point cloud.

20. The vehicle of claim 16, wherein the one or more sensors include two or more selected from the group consisting of (i) an inertial measurement unit, (ii) a lidar device, and (iii) a camera.