Video based intelligent vehicle control system

• US 9,165,468 B2
• Filed: 04/12/2010
• Issued: 10/20/2015
• Est. Priority Date: 04/12/2010
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH, Robert Bosch LLC (Robert Bosch GmbH)
• Original Assignee: Robert Bosch GmbH
• Inventors: Luo, Yun, Hoetzer, Dieter
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): OH, HARRY Y
• Application Number:
  US12/758,394
• Publication Number:
  US 20110251768A1
• Time in Patent Office: 2,017 Days
• Field of Search: 701/70
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 60/00 Drive control systems specially adapted for autonomous road vehicles
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A system for longitudinal and lateral control of a vehicle. The system includes a camera generating a plurality of images representative of a field of view in front of the vehicle, and a controller receiving the plurality of images. The controller detects an object in the plurality of images, extracts a plurality of features from the plurality of images, generates a reference feature depiction based on the plurality of features from one or more of the plurality of images, generates a comparison feature depiction based on the plurality of features from one of the plurality of images, compares the reference feature depiction and the comparison feature depiction, and determines a longitudinal position of the vehicle relative to the object based on differences between the reference feature depiction and the comparison feature depiction.

Claims

1. A system for longitudinal and lateral control of a vehicle, the system comprising:

a camera generating a plurality of images representative of a field of view in front of the vehicle;

a human-machine interface including a video display and configured to receive an input; and

a controller receiving the plurality of images and

detecting an object in the plurality of images,

extracting a plurality of features from the plurality of images,

generating a plurality of edges connecting pairs of the plurality of features, a length of the edges representing a spatial distance between the plurality of features;

generating a reference feature depiction based on the plurality of features and the plurality of edges from one or more of the plurality of images,

generating a comparison feature depiction based on the plurality of features and the plurality of edges from one of the plurality of images,

comparing the reference feature depiction and the comparison feature depiction,

determining a longitudinal position of the vehicle relative to the object based on differences between the reference feature depiction and the comparison feature depiction,

adjusting a scale of the reference feature depiction based on the input received by the human-machine interface, and

modifying a distance the vehicle maintains from the detected object such that a scale of the comparison feature depiction is about equal to the adjusted scale of the reference feature depiction.

2. The system of claim 1, wherein the controller determines a lateral position of the vehicle relative to the object based on a position of the comparison feature depiction in the plurality of images.

3. The system of claim 2, wherein the same images are used to determine the longitudinal position and the lateral position.

4. The system of claim 2, wherein the controller adjusts a speed and a direction of the vehicle to maintain the longitudinal position and the lateral position of the vehicle relative to the object.

5. The system of claim 1, wherein the reference feature depiction is generated using the plurality of features extracted from two or more of the plurality of images.

6. The system of claim 5, wherein the reference feature depiction includes the plurality of features extracted from two or more of the plurality of images have a level of confidence of corresponding features exceeds a threshold.

7. The system of claim 1, wherein the plurality of features used in the comparison feature depiction is reduced until the plurality of features of the comparison feature depiction match the plurality of features of the reference feature depiction.

8. The system of claim 7, further comprising determining that the object is not in the plurality of images when the plurality of features of the comparison feature depiction that match the plurality of features of the reference feature depiction are less than a threshold.

9. The system of claim 1, wherein a quantity of features for the reference feature depiction is chosen such that the quantity of features is large enough to ensure that the object is accurately identified and that the quantity of features is small enough to allow for computational efficiency.

10. The system of claim 1, wherein the longitudinal position of the vehicle relative to the object is determined based on a change of scale between the comparison feature depiction and the reference feature depiction.

11. The system of claim 10, wherein the controller uses a weighted measurement of changes in the length of the plurality of edges between the reference feature depiction and the comparison feature depiction to determine the change of scale.

12. The system of claim 1, wherein the human-machine interface comprises a rocker switch, the controller increasing the scale of the reference feature depiction when the rocker switch is pressed upward and decreasing the scale of the reference feature depiction when the rocker switch is pressed downward.

13. The system of claim 1, wherein the human-machine interface comprises a roller switch, the controller increasing the scale of the reference feature depiction when the roller switch is rolled in an upward direction and decreasing the scale of the reference feature depiction when the roller switch is rolled in a downward direction.

14. The system of claim 1, wherein the input is at least one of an accelerator of the vehicle, a brake pedal of the vehicle, a button, a voice command, and a touch screen.

15. The system of claim 1, wherein a status of the system includes an object to track has not been identified, an object to track been identified but not selected, an object is being tracked, and the vehicle is too close to an object.

16. The system of claim 15, further comprising a button, the button being lit in one of a plurality of colors to indicate the status of the system.

17. A method of controlling a vehicle using a forward looking camera, the method comprising:

detecting an object in a plurality of images generated by a camera,

displaying the object on an human machine interface;

extracting a plurality of features from the plurality of images,

generating a plurality of edges connecting pairs of the plurality of features, a length of the edges representing a spatial distance between the plurality of features;

generating a reference feature depiction based on the plurality of features and the plurality of edges from one or more of the plurality of images,

generating a comparison feature depiction based on the plurality of features and the plurality of edges from one of the plurality of images,

comparing the reference feature depiction and the comparison feature depiction,

determining a longitudinal position of the vehicle relative to the object based on differences between the reference feature depiction and the comparison feature depiction,

receiving an input from the human-machine interface,

adjusting a scale of the reference feature depiction based on the input received by the human-machine interface, and

modifying a distance the vehicle maintains from the detected object such that a scale of the comparison feature depiction is about equal to the adjusted scale of the reference feature depiction.

18. The method of claim 17, further comprising determining a lateral position of the vehicle relative to the object based on a position of the comparison feature depiction in the plurality of images.

19. The method of claim 18, further comprising adjusting a speed and a direction of the vehicle to maintain the longitudinal position and the lateral position of the vehicle relative to the object.

20. The method of claim 17, further comprising reducing the plurality of features used in the comparison feature depiction until the plurality of features of the comparison feature depiction match the plurality of features of the reference feature depiction.

21. The method of claim 20, further comprising determining that the object is not in the plurality of images when the plurality of features of the comparison feature depiction that match the plurality of features of the reference feature depiction are less than a threshold.

22. The method of claim 17, further comprising using a weighted measurement of changes in the length of the plurality of edges between the reference feature depiction and the comparison feature depiction to determine the change of scale.