Tracking on-road vehicles with sensors of different modalities

• US 9,255,989 B2
• Filed: 07/24/2012
• Issued: 02/09/2016
• Est. Priority Date: 07/24/2012
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Joshi, Avdhut S., James, Michael R., Samples, Michael E.
• Primary Examiner(s): Algahaim, Helal A
• Assistant Examiner(s): SMITH, JELANI A
• Application Number:
  US13/556,802
• Publication Number:
  US 20140032012A1
• Time in Patent Office: 1,295 Days
• Field of Search: 244/3.16, 340/539.13, 342/145, 342/357.29, 342/357.74, 342/424, 342/70, 342/95, 375/150, 701/1, 701/2, 701/3, 701/25, 701/301, 701/466, 701/468, 701/494, 701/50, 705/14.1, 705/75
• US Class Current: 1/1
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 7/2955 Means for determining the position of the radar coordinate system for evaluating the position data of the target in another coordinate system G01S7/24 takes precedence; sighting devices adapted for indirect laying of fire F41G3/16; inertial navigation G01C21/16
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A vehicle system includes a first sensor and a second sensor, each having, respectively, different first and second modalities. A controller includes a processor configured to: receive a first sensor input from the first sensor and a second sensor input from the second sensor; detect, synchronously, first and second observations from, respectively, the first and second sensor inputs; project the detected first and second observations onto a graph network; associate the first and second observations with a target on the graph network, the target having a trajectory on the graph network; select either the first or the second observation as a best observation based on characteristics of the first and second sensors; and estimate a current position of the target by performing a prediction based on the best observation and a current timestamp.

Claims

1. A controller for a vehicle, comprising a processor configured to:

receive a first sensor input from a first sensor and a second sensor input from a second sensor, the first and second sensors having, respectively, different first and second modalities and the first and second sensors being local to the vehicle;

detect, synchronously, first and second observations of a target from, respectively, the first and second sensor inputs, the first and second observations being from a perspective of the vehicle;

generate a graph network from a road map of a road network of an area in which the vehicle is present and an acquired position of the vehicle, wherein the graph network includes one dimensional lanes as links between nodes;

project the detected first and second observations of the target onto the graph network;

associate the first and second observations with the target on the graph network, the target having a trajectory on the graph network, wherein the trajectory of the target is based on one of the one dimensional lanes;

select either the first or the second observation of the target as a best observation of the target based on characteristics of the first and second sensors; and

estimate a current position of the target by performing a prediction based on the best observation and a current timestamp.

2. The controller according to claim 1, wherein the first and second observations are associated with the target by generating a new target on the graph network when the first and second observations are not within a predefined distance of a currently tracked target.

3. The controller according to claim 1, wherein the first sensor is a lidar sensor, and the second sensor is a radar sensor.

4. The controller according to claim 1, wherein the current position of the target is estimated by performing a Kalman filter prediction process.

5. The controller according to claim 1, wherein the characteristics of the first and second sensors include a time delay for processing the sensor data and an accuracy in detecting valid obstacles by the processing of the sensor data.

6. The controller according to claim 1, wherein the target is tracked, by current position updates, through successive observations by the first and second sensors, and the target is removed from being tracked when no observation is observed for the target for a defined time period.

7. The controller according to claim 1, wherein the detected first and second observations are projected onto the graph network by transforming respective coordinate systems of the first and second sensor inputs into a common coordinate system of the vehicle.

8. The controller according to claim 1, wherein the controller stores a list of targets in a target list in a memory of the controller, and the target list is queried to associate observations with the targets.

9. The controller according to claim 1, wherein synchronous detection of observations is restricted to points on or within a margin of error from a lane of the graph network.

10. The controller according to claim 1, wherein tracking of targets is restricted to points on or within a margin of error from a lane of the graph network.

11. The controller according to claim 1, wherein the detected first and second observations are projected onto the graph network by correcting the detected first and second observations to reflect a current timestamp, adjusting a position of the first and second observations on the graph network to account for a time delay between acquiring respective sensor data and projecting resulting observations onto the graph network.

12. A vehicle system, comprising:

a first sensor and a second sensor, each having, respectively, different first and second modalities, the first and second sensors being local to the vehicle; and

a controller including a processor configured to;

receive a first sensor input from the first sensor and a second sensor input from the second sensor;

detect, synchronously, first and second observations of a target from, respectively, the first and second sensor inputs, the first and second observations being from a perspective of the vehicle;

generate a graph network from a road map of a road network of an area in which the vehicle is present and an acquired position of the vehicle, wherein the graph network includes one dimensional lanes as links between nodes;

project the detected first and second observations onto the graph network;

associate the first and second observations with the target on the graph network, the target having a trajectory on the graph network, wherein the trajectory of the target is based on one of the one dimensional lanes;

select either the first or the second observation of the target as a best observation of the target based on characteristics of the first and second sensors; and

estimate a current position of the target by performing a prediction based on the best observation and a current timestamp.

13. A method, comprising:

receiving a first sensor input from a first sensor and a second sensor input from a second sensor, the first and second sensors having, respectively, different first and second modalities and the first and second sensors being local to a vehicle;

detecting, synchronously, first and second observations of a target from, respectively, the first and second sensor inputs, the first and second observations being from a perspective of the vehicle;

generating a graph network from a road map of a road network of an area in which the vehicle is present and an acquired position of the vehicle, wherein the graph network includes one dimensional lanes as links between nodes;

projecting the detected first and second observations onto a graph network;

associating the first and second observations with the target on the graph network, the target having a trajectory on the graph network, wherein the trajectory of the target is based on one of the one dimensional lanes;

selecting either the first or the second observation of the target as a best observation of the target based on characteristics of the first and second sensors; and

estimating a current position of the target by performing a prediction based on the best observation and a current timestamp.