Minimizing incorrect sensor data associations for autonomous vehicles

• US 10,054,678 B2
• Filed: 07/30/2015
• Issued: 08/21/2018
• Est. Priority Date: 07/30/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Mei, Xue, Nagasaka, Naoki, Okumura, Bunyo
• Primary Examiner(s): Brainard, Timothy A
• Application Number:
  US14/813,429
• Publication Number:
  US 20170031015A1
• Time in Patent Office: 1,118 Days
• Field of Search: 342 54
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 13/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G05D 1/0238 using obstacle or wall sensors G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931

Abstract

Minimizing incorrect associations of sensor data for an autonomous vehicle are described. A driving environment of the autonomous vehicle includes a stationary object and a dynamic object. Such objects can be detected by radar sensors and/or lidar sensors. In one example, a history of radar observation can be used to minimize incorrect sensor data associations. In such case, the location of a stationary object in the driving environment can be determined. When a dynamic object passes by the stationary object, lidar data of the dynamic object is prevented from being associated with radar data obtained substantially at the determined location of the stationary object. In another example, identifiers assigned to radar data can be used to minimize incorrect sensor data associations. In such case, lidar data of an object can be associated with radar data having a particular identifier.

Claims

1. A method of minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the method comprising:

acquiring, using at least a radar sensor, a first sensor observation of the forward portion of the driving environment in which the dynamic object is located away from the stationary object, the first sensor observation including radar data for the dynamic object and the stationary object, acquiring the first sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

determining the location of the stationary object in the driving environment based on the radar data;

acquiring, using at least the radar sensor and a lidar sensor, a second sensor observation of the forward portion of the driving environment in which the dynamic object is located substantially near the determined location of the stationary object, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation, acquiring the second sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

transmitting one or more lidar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted lidar signals, whereby the one or more lidar signals are reflected off of objects in the driving environment; and

for the second sensor observation, preventing the lidar data of the dynamic object from being associated with radar data obtained substantially at the determined location of the stationary object.

2. The method of claim 1, wherein acquiring a first sensor observation of the forward portion of the driving environment in which the dynamic object is located away from the stationary object includes:

acquiring a plurality of sensor observations of the forward portion of the driving environment in which the dynamic object is located away from the stationary object;

wherein the plurality of sensor observations includes radar data for the dynamic object and the stationary object; and

for the plurality of sensor observations, determining the location of the stationary object in the driving environment based at least in part on the radar data.

3. The method of claim 1, wherein the radar data includes speed data and location data, and wherein preventing the lidar data of the dynamic object from being associated with radar data obtained substantially at the determined location of the stationary object includes preventing the lidar data of the dynamic object from being associated with at least the speed data from the radar data obtained near the determined location of the stationary object.

4. The method of claim 1, further including:

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object that is prevented from being associated with radar data obtained substantially at the determined location of the stationary object.

5. A method of minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the method comprising:

acquiring a first sensor observation of the forward portion of the driving environment in which the dynamic object is located away from the stationary object, the first sensor observation including radar data for the dynamic object and the stationary object;

determining the location of the stationary object in the driving environment based on the radar data;

acquiring a second sensor observation of the forward portion of the driving environment in which the dynamic object is located substantially near the determined location of the stationary object, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation;

for the second sensor observation, preventing the lidar data of the dynamic object from being associated with radar data obtained substantially at the determined location of the stationary object;

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object that is prevented from being associated with radar data obtained substantially at the determined location of the stationary object; and

causing the autonomous vehicle to implement the determined driving maneuver.

6. The method of claim 5, wherein acquiring the first sensor observation of the forward portion of the driving environment is performed using at least a radar sensor, and wherein acquiring the second sensor observation of the forward portion of the driving environment is performed using at least a radar sensor and a lidar sensor.

7. A system for minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the system comprising:

a sensor system including a radar sensor and a lidar sensor, the sensor system being configured to;

acquire, using at least the radar sensor, a first sensor observation of a forward portion of the driving environment in which the dynamic object is located away from the stationary object, the first sensor observation including radar data for the dynamic object and the stationary object, acquiring the first sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment; and

acquire, using at least the radar sensor and the lidar sensor, a second sensor observation of the forward portion of the driving environment in which the dynamic object is located substantially near the stationary object, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation, acquiring the second sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

transmitting one or more lidar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted lidar signals, whereby the one or more lidar signals are reflected off of objects in the driving environment; and

a processor operatively connected to the sensor system, the processor being programmed to initiate executable operations including;

for the first sensor observation, determining the location of the stationary object in the driving environment based on the radar data; and

for the second sensor observation, preventing the lidar data of the dynamic object from being associated with radar data obtained substantially at the determined location of the stationary object.

8. The system of claim 7, wherein acquiring a first sensor observation of the forward portion of the driving environment in which the dynamic object is located away from the stationary object includes:

acquiring a plurality of sensor observations of the forward portion of the driving environment in which the dynamic object is located away from the stationary object;

wherein the plurality of sensor observations includes radar data for the dynamic object and the stationary object; and

for the plurality of sensor observations, determining the location of the stationary object in the driving environment based at least in part on the radar data.

9. The system of claim 7, wherein the radar data includes speed data and location data, and wherein preventing the lidar data of the dynamic object from being associated with radar data obtained substantially at the determined location of the stationary object includes preventing the lidar data of the dynamic object from being associated with at least the speed data from the radar data obtained substantially at the determined location of the stationary object.

10. The system of claim 7, wherein the executable operations further include:

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object that is prevented from being associated with radar data obtained substantially at the determined location of the stationary object.

11. A system for minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the system comprising:

a sensor system including a radar sensor and a lidar sensor, the sensor system being configured to;

acquire a first sensor observation of a forward portion of the driving environment in which the dynamic object is located away from the stationary object, the first sensor observation including radar data for the dynamic object and the stationary object; and

acquire a second sensor observation of the forward portion of the driving environment in which the dynamic object is located substantially near the stationary object, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation; and

a processor operatively connected to the sensor system, the processor being programmed to initiate executable operations including;

for the first sensor observation, determining the location of the stationary object in the driving environment based on the radar data;

for the second sensor observation, preventing the lidar data of the dynamic object from being associated with radar data obtained substantially at the determined location of the stationary object;

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object that is prevented from being associated with radar data obtained substantially at the determined location of the stationary object; and

causing the autonomous vehicle to implement the determined driving maneuver.

12. The system of claim 11, wherein acquiring the first sensor observation of the forward portion of the driving environment is performed using at least a radar sensor, and wherein acquiring the second sensor observation of the forward portion of the driving environment is performed using at least a radar sensor and a lidar sensor.

13. A method of minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the method comprising:

acquiring, using at least a radar sensor and a lidar sensor, a first sensor observation of the forward portion of the driving environment, the first sensor observation including radar data and lidar data for the dynamic object and the stationary object, a first identifier being assigned to the radar data for the dynamic object and a second identifier being assigned to the radar data for the stationary object, acquiring the first sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

transmitting one or more lidar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted lidar signals, whereby the one or more lidar signals are reflected off of objects in the driving environment;

for the first sensor observation, associating the lidar data of the dynamic object with the radar data having the first identifier assigned thereto;

for the first sensor observation, associating the lidar data of the stationary object with the radar data having the second identifier assigned thereto;

acquiring, using at least the radar sensor and the lidar sensor, a second sensor observation of the forward portion of the driving environment, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation, the first identifier being assigned to the radar data for the dynamic object and the second identifier being assigned to the radar data for the stationary object, acquiring the second sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

transmitting one or more lidar signals into the forward portion of the driving environment; and

receiving one or more reflections of the one or more transmitted lidar signals, whereby the one or more lidar signals are reflected off of objects in the driving environment; and

for the second sensor observation, associating the lidar data of the dynamic object with only radar data with the first identifier assigned thereto.

14. The method of claim 13, further including:

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object with only radar data with the first identifier assigned thereto.

15. The method of claim 14, wherein the driving maneuver is a movement of the autonomous vehicle in the lateral direction relative to the dynamic object.

16. A method of minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the method comprising:

acquiring a first sensor observation of the forward portion of the driving environment, the first sensor observation including radar data and lidar data for the dynamic object and the stationary object, a first identifier being assigned to the radar data for the dynamic object and a second identifier being assigned to the radar data for the stationary object;

for the first sensor observation, associating the lidar data of the dynamic object with the radar data having the first identifier assigned thereto;

for the first sensor observation, associating the lidar data of the stationary object with the radar data having the second identifier assigned thereto;

acquiring a second sensor observation of the forward portion of the driving environment, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation, the first identifier being assigned to the radar data for the dynamic object and the second identifier being assigned to the radar data for the stationary object;

for the second sensor observation, associating the lidar data of the dynamic object with only radar data with the first identifier assigned thereto;

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object with only radar data with the first identifier assigned thereto; and

causing the autonomous vehicle to implement the determined driving maneuver.

17. A system for minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the system comprising:

a sensor system including a radar sensor and a lidar sensor, the sensor system being configured to;

acquire, using at least the radar sensor and the lidar sensor, a first sensor observation of the forward portion of the driving environment, the first sensor observation including radar data and lidar data for the dynamic object and the stationary object, a first identifier being assigned to the radar data for the dynamic object and a second identifier being assigned to the radar data for the stationary object, acquiring the first sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

transmitting one or more lidar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted lidar signals, whereby the one or more lidar signals are reflected off of objects in the driving environment;

acquire, using at least the radar sensor and the lidar sensor, a second sensor observation of the forward portion of the driving environment, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation, the first identifier being assigned to the radar data for the dynamic object and the second identifier being assigned to the radar data for the stationary object, acquiring the second sensor observation including;

transmitting one or more radar signals into the forward portion of the driving environment;

receiving one or more reflections of the one or more transmitted radar signals, whereby the one or more radar signals are reflected off of objects in the driving environment;

transmitting one or more lidar signals into the forward portion of the driving environment; and

receiving one or more reflections of the one or more transmitted lidar signals, whereby the one or more lidar signals are reflected off of objects in the driving environment; and

a processor operatively connected to the sensor system, the processor being programmed to initiate executable operations comprising;

for the first sensor observation, associating the lidar data of the dynamic object with the radar data having the first identifier assigned thereto;

for the first sensor observation, associating the lidar data of the stationary object with the radar data having the second identifier assigned thereto; and

for the second sensor observation, associating the lidar data of the dynamic object with only radar data with the first identifier assigned thereto.

18. The system of claim 17, wherein the executable operations further include:

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object and the radar data with the first identifier assigned thereto.

19. The system of claim 18, wherein the driving maneuver is a movement of the autonomous vehicle in a lateral direction relative to the dynamic object.

20. A system for minimizing incorrect associations of sensor data for an autonomous vehicle, the autonomous vehicle operating in a driving environment, a stationary object and a dynamic object being located in a forward portion of the driving environment, the system comprising:

a sensor system including a radar sensor and a lidar sensor, the sensor system being configured to;

acquire a first sensor observation of the forward portion of the driving environment, the first sensor observation including radar data and lidar data for the dynamic object and the stationary object, a first identifier being assigned to the radar data for the dynamic object and a second identifier being assigned to the radar data for the stationary object;

acquire a second sensor observation of the forward portion of the driving environment, the second sensor observation including radar data and lidar data for the dynamic object and the stationary object, the second sensor observation being subsequent to the first sensor observation, the first identifier being assigned to the radar data for the dynamic object and the second identifier being assigned to the radar data for the stationary object;

a processor operatively connected to the sensor system, the processor being programmed to initiate executable operations comprising;

for the first sensor observation, associating the lidar data of the dynamic object with the radar data having the first identifier assigned thereto;

for the first sensor observation, associating the lidar data of the stationary object with the radar data having the second identifier assigned thereto;

for the second sensor observation, associating the lidar data of the dynamic object with only radar data with the first identifier assigned thereto;

determining a driving maneuver for the autonomous vehicle based at least partially on the lidar data of the dynamic object and the radar data with the first identifier assigned thereto; and

causing the autonomous vehicle to implement the determined driving maneuver.