Object detection and tracking system

• US 8,027,029 B2
• Filed: 11/07/2008
• Issued: 09/27/2011
• Est. Priority Date: 11/07/2007
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Higgins-Luthman, Michael J., Lu, Yuesheng
• Primary Examiner(s): Alsomiri; Isam
• Application Number:
  US12/266,656
• Publication Number:
  US 20100328644A1
• Time in Patent Office: 1,054 Days
• Field of Search: 356/4.01, 356/5.01, 356/5.15, 356/28, 382/103, 382/106, 382/107
• US Class Current: 356/4.01
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/4865 Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak peak detection in noise, signal conditioning G01S7/487
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/81 : for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An object detection system for a vehicle includes an imaging device, an image processor and a Lidar device. The imaging device is operable to capture image data representative of a scene exterior the vehicle. The image processor processes image data to detect an object of interest in the field of view of the imaging device. The Lidar device is responsive to the image processing so as to have its line of sight guided to point toward the detected object of interest. The Lidar device is operable to measure the distances between the host vehicle and the detected object of interest.

Claims

1. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of a scene exterior the host vehicle;

an image processor processing image data to detect an object of interest in the field of view of said imaging device; and

a ranging device emitting a steerable beam, wherein said steerable beam of said ranging device is steerable responsive to said image processing so as to have its line of sight guided toward the detected object of interest, and wherein said object detection system is operable to direct said steerable beam toward a detected object of interest for a dwell time and wherein said dwell time of said steerable beam is variable in accordance with a priority score.

2. The object detection system of claim 1, wherein said imaging device comprises a pixelated two-dimensional array of photosensing pixels.

3. The object detection system of claim 1, wherein said ranging device comprises a detector.

4. The object detection system of claim 3, wherein said ranging device includes a steering device that steers an aim of said steerable beam in response to said image processing.

5. The object detection system of claim 4, wherein said ranging device is operable to measure the distance to the detected object of interest via at least one of (i) comparing a modulation phase and (ii) calculating a time of flight of the light between the host vehicle and the detected object.

6. The object detection system of claim 1, wherein said system detects multiple objects of interest in the field of view of said imaging device.

7. The object detection system of claim 6, wherein said system prioritizes the detected objects based on at least one of (i) proximity of the detected objects to the host vehicle, (ii) speed of the detected objects relative to the host vehicle, and (iii) location of the detected objects relative to the host vehicle.

8. The object detection system of claim 7, wherein said system guides said line of sight of said steerable beam of said ranging device toward the detected objects in accordance with the prioritization of the detected objects.

9. The object detection system of claim 8, wherein said system guides said line of sight of said steerable beam of said ranging device toward a first detected object for a first period of time and said system guides said line of sight of said steerable beam of said ranging device toward a second detected object for a second period of time.

10. The object detection system of claim 9, wherein said first detected object has a higher priority than said second detected object and said first period of time is greater than said second period of time.

11. The object detection system of claim 1, wherein said object detection system comprises at least one of (a) an adaptive cruise control system, (b) a collision warning system, (c) a pedestrian detection system, (d) a stop-and-go system, (e) a side object detection system, (f) a lane change aid systems, (g) a side blind zone detection system, (h) a forward object detection system, (i) a rear blind zone object detection system, and (j) a parking assist system.

12. The object detection system of claim 1, wherein, responsive to a high priority score assigned to a detected object, said steerable beam has a longer dwell time at an object that is assigned a high priority score.

13. The object detection system of claim 1, wherein said steerable beam of said ranging device is steerable to at least one of (a) track a detected object while the detected object moves in the field of view of said imaging device and (b) stay on a detected object while the detected object moves in the field of view of said imaging device.

14. The object detection system of claim 1, wherein priority scores are assigned to objects detected in the field of view of said imaging device in accordance with at least one of (a) a danger of the detected objects and (b) an importance of the detected objects.

15. The object detection system of claim 1, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

16. The object detection system of claim 1, wherein said ranging device comprises a Lidar device.

17. The object detection system of claim 1, wherein said steerable beam of said ranging device is mechanically steerable responsive to said image processing.

18. The object detection system of claim 1, wherein said priority score is representative of the importance of the object detected in the field of view.

19. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of a scene exterior the host vehicle;

an image processor processing image data to detect an object in the field of view of said imaging device;

a ranging device emitting a steerable beam, wherein said steerable beam of said ranging device is steerable responsive to said image processing so as to have its line of sight guided toward the detected object, and wherein said object detection system is operable to direct said steerable beam toward the detected object; and

wherein a priority score is assigned to the detected object in accordance with at least one of (a) a danger of the detected object and (b) an importance of the detected object, and wherein said object detection system is operable to direct said steerable beam toward the detected object at least in part responsive to the assigned priority score.

20. The object detection system of claim 19, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

21. The object detection system of claim 19, wherein said object detection system guides said line of sight of said steerable beam of said ranging device toward the detected object in accordance with the prioritization of the detected object, wherein said object detection system guides said line of sight of said steerable beam of said ranging device toward a first detected object for a first period of time and said object detection system guides said line of sight of said steerable beam of said ranging device toward a second detected object for a second period of time, and wherein said first detected object has a higher priority than said second detected object and said first period of time is greater than said second period of time.

22. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of a scene exterior the host vehicle;

an image processor processing image data to detect an object in the field of view of said imaging device;

a ranging device emitting a steerable beam, wherein said steerable beam of said ranging device is steerable responsive to said image processing so as to have its line of sight guided toward the detected object, and wherein said object detection system is operable to direct said steerable beam toward a detected object; and

wherein said object detection system detects multiple objects in the field of view of said imaging device, and wherein said object detection system prioritizes the detected objects based on at least one of (i) proximity of the detected objects to the host vehicle, (ii) speed of the detected objects relative to the host vehicle, and (iii) location of the detected objects relative to the host vehicle, and wherein said object detection system is operable to direct said steerable beam toward at least one of the detected objects at least in part responsive to the prioritization of the objects.

23. The object detection system of claim 22, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

24. The object detection system of claim 22, wherein said object detection system guides said line of sight of said steerable beam of said ranging device toward a first detected object for a first period of time and said object detection system guides said line of sight of said steerable beam of said ranging device toward a second detected object for a second period of time, and wherein said first detected object has a higher priority than said second detected object and said first period of time is greater than said second period of time.