Multi-surface model-based tracking

• US 9,466,215 B2
• Filed: 03/26/2013
• Issued: 10/11/2016
• Est. Priority Date: 03/26/2012
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH, Robert Bosch LLC (Robert Bosch GmbH)
• Original Assignee: Robert Bosch GmbH
• Inventors: Luo, Yun, Zhu, Xavier
• Primary Examiner(s): Jabr, Fadey
• Assistant Examiner(s): HEINLE, COURTNEY D
• Application Number:
  US13/850,829
• Publication Number:
  US 20130253796A1
• Time in Patent Office: 1,295 Days
• Field of Search: 701/96
• US Class Current: 1/1
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 in the back of the vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Systems and methods for detecting a vehicle. One system includes a controller. The controller is configured to receive images from a camera mounted on a first vehicle, identify a surface of a second vehicle located around the first vehicle based on the images, and generate a three-dimensional model associated with the second vehicle. The model includes a first plane and a second plane approximately perpendicular to the first plane. The first plane of the model is associated with the identified surface of the second vehicle. The controller is further configured to track a position of the second vehicle using the three-dimensional model after the identified surface falls at least partially outside of a field-of-view of the at least one camera.

Claims

1. A vehicle mounted system for detecting a vehicle, the system comprising:

a controller including a processor, a computer readable media in communication with the processor, and an input/output interface, the processor configured to;

receive images from a camera mounted on a moving first vehicle,

identify a surface of a second vehicle located around the first vehicle based on the images by;

detecting motion in the images,

extracting features from the images based on the detected motion,

generating a surface based on the extracted features, and

comparing the surface to a side surface of at least one vehicle model;

generate a three-dimensional model associated with the second vehicle, the three-dimensional model including a first plane and a second plane approximately perpendicular to the first plane, wherein the first plane is associated with the identified surface of the second vehicle, and

track a position of the second vehicle using the three-dimensional model after the identified surface falls at least partially outside of a field-of-view of the camera by;

receiving updated images from the camera,

extracting features from the updated images,

determining if the extracted features fit the first plane of the three-dimensional model,

when the extracted features do not fit the first plane of the three-dimensional model, determining if the extracted features fit the second plane of the three-dimensional model, and

when the extracted features fit the second plane of the three-dimensional model, updating the three-dimensional model based on the extracted features, and

wherein the controller is further configured to perform at least one automatic vehicle control operation based on the tracked position of the second vehicle.

2. The system of claim 1, wherein the controller is further configured to receive data from at least one radar sensor mounted on the first vehicle.

3. The system of claim 1, wherein the surface of the second vehicle is a back surface of the second vehicle.

4. The system of claim 1, wherein the surface of the second vehicle is a side surface of the second vehicle.

5. The system of claim 1, wherein the controller is configured to identify the surface of the second vehicle based on features extracted from the images.

6. The system of claim 1, wherein the controller is configured to generate the three-dimensional model by:

classifying the second vehicle into a vehicle category based on characteristics of the identified surface of the second vehicle,

defining a bounding box representing the identified surface of the second, wherein the size of the bounding box is based on the vehicle category of the second vehicle, and

setting the bounding box as the first plane of the three-dimensional model.

7. The system of claim 1, wherein the camera is mounted to a front of the first vehicle.

8. The system of claim 1, wherein the at least one automatic vehicle control operation includes at least one adaptive cruise control and forward collision warning.

9. The system of claim 1, wherein the controller determines turning intention of the second vehicle from rotation of the second vehicle before a vehicle turn of the second vehicle occurs.

10. A method for detecting and avoiding collision with a vehicle, the method comprising:

receiving, at at least one controller, images from at least one camera mounted on a first vehicle;

identifying, by the at least one controller, a surface of a second vehicle located around the first vehicle based on the images by;

detecting motion in the images,

extracting features from the images based on the detected motion,

generating a surface based on the extracted features, and

comparing the surface to a side surface of at least one vehicle model;

generating, by the at least one controller, a three-dimensional model representing the second vehicle, the three-dimensional model including a first plane and a second plane approximately perpendicular to the first plane, wherein the first plane represents the identified surface of the second vehicle;

determining, by the at least one controller, a position of the second vehicle using the model and updated data from the at least one camera after the identified surface of the second vehicle falls at least partially outside of a field-of-view of the at least one camera, and

performing, by the at least one controller, at least one automatic vehicle control operation based on the tracked position of the second vehicle.

11. The method of claim 10, wherein detecting a surface of a second vehicle includes detecting a back surface of a second vehicle.

12. The method of claim 10, wherein detecting a surface of the second vehicle includes detecting a side surface of a second vehicle.

13. The method of claim 10, wherein determining a position of the second vehicle includes

receiving updated images from the at least one camera;

extracting features from the updated images;

determining if the extracted features fit the first plane of the three-dimensional model;

when the extracted features do not fit the first plane of the three-dimensional model, determining if the extracted features fit the second plane of the three-dimensional model; and

when the extracted features fit the second plane of the three-dimensional model, updating the three-dimensional model based on the extracted features.

14. The method of claim 10, wherein generating a three-dimensional model includes

classifying the second vehicle into a vehicle category based on characteristics of the identified surface of the second vehicle,

defining a bounding box representing the identified surface of the second, wherein the size of the bounding box is based on the vehicle category of the second vehicle, and

setting the bounding box as the first plane of the three-dimensional model.

15. The method of claim 10, further comprising transmitting the position of the second vehicle to at least one of an automatic cruise control system and a forward collision warning system.

16. The method of claim 10, wherein the controller determines turning intention of the second vehicle from rotation of the second vehicle before a vehicle turn of the second vehicle occurs.

17. A method for detecting a vehicle, the method comprising:

receiving images from a camera mounted on a moving first vehicle;

detecting a surface of a second vehicle located around the first vehicle based on the images;

generating, by a controller, a three-dimensional model representing the second vehicle, the three-dimensional model including a first plane and a second plane approximately perpendicular to the first plane, wherein the first plane represents the identified surface of the second vehicle;

determining a position of the second vehicle using the model after the identified surface of the second vehicle falls at least partially outside of a field-of-view of the camera, wherein determining the position of the second vehicle includes;

receiving updated images from the camera;

extracting features from the updated images;

determining if the extracted features fit the first plane of the three-dimensional model;

when the extracted features do not fit the first plane of the three-dimensional model, determining if the extracted features fit the second plane of the three-dimensional model; and

when the extracted features fit the second plane of the three-dimensional model, updating the three-dimensional model based on the extracted features, and

performing at least one automatic vehicle control operation based on the tracked position of the second vehicle.

18. The method of claim 17, wherein the controller determines turning intention of the second vehicle from rotation of the second vehicle before a vehicle turn of the second vehicle occurs.