Object detection system

• US 8,767,186 B2
• Filed: 09/23/2011
• Issued: 07/01/2014
• Est. Priority Date: 11/07/2007
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lu, Yuesheng, Higgins-Luthman, Michael J.
• Primary Examiner(s): Diacou, Ari M
• Application Number:
  US13/242,991
• Publication Number:
  US 20120008129A1
• Time in Patent Office: 1,012 Days
• Field of Search: 701/36, 701/301, 356/4.01
• US Class Current: 356/4.01
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/4865 Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak peak detection in noise, signal conditioning G01S7/487
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/81 : for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An object detection system for a host vehicle includes an imaging device, a ranging device and a processor. The imaging device has a field of view exterior of the host vehicle and is operable to capture image data representative of an object exterior the host vehicle. The ranging device has a field of view that sweeps at least the field of view of the imaging device and that detects the object exterior the host vehicle. The processor processes image data from the imaging device and processes data from the ranging device in order to determine if the detected object is an object of interest. The determination that the detected object is an object of interest is made in accordance with a priority score.

Claims

1. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and o able to capture image data representative of an object exterior the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image e data from said image in device and processes data from said ran device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score; and

wherein said field of view of said ranging device is adjustable.

2. The object detection system of claim 1, wherein said field of view of said ranging device is adjustable responsive to processing of image data from said imaging device, and wherein said object detection system is operable to adjust said adjustable field of view of said ranging device toward the detected object.

3. The object detection system of claim 1, wherein the field of view of said ranging device is adjustable responsive to said image processing.

4. The object detection system of claim 3, wherein said ranging device is operable to measure the distance to the detected object of interest via at least one of (i) comparing a modulation phase and (ii) calculating a time of flight of light between the host vehicle and the detected object.

5. The object detection system of claim 1, wherein the field of view of said ranging device is adjustable to at least one of (a) track a detected object, (b) track a detected object while the detected object moves in the field of view of said imaging device, (c) dwell on a detected object and (d) dwell on a detected object while the detected object moves in the field of view of said imaging device.

6. The object detection system of claim 1, wherein said imaging device is associated with at least one of (i) headlamp control, (ii) lane departure warning, (iii) traffic sign recognition, (iv) night vision and (v) adaptive front lighting.

7. The object detection system of claim 1, wherein said imaging device comprises a pixelated two-dimensional array of photosensing pixels.

8. The object detection system of claim 1, wherein said ranging device comprises at least one of (i) a radar device, (ii) a Lidar device and (iii) a stereo vision device.

9. The object detection system of claim 1, wherein said system determines multiple objects of interest in the field of view of said imaging device.

10. The object detection system of claim 9, wherein said system prioritizes detected objects based on at least one of (i) proximity of detected objects to the host vehicle, (ii) speed of detected objects relative to the host vehicle, and (iii) location of detected objects relative to the host vehicle.

14. The object detection system of claim 1, wherein said object detection system comprises at least one of (a) an adaptive cruise control system of the host vehicle, (b) a collision warning system of the host vehicle, (c) a pedestrian detection system of the host vehicle, (d) a stop-and-go system of the host vehicle, (e) a side object detection system of the host vehicle, (f) a lane change aid system of the host vehicle, (g) a side blind zone detection system of the host vehicle, (h) a forward object detection system of the host vehicle, (i) a rear blind zone object detection system of the host vehicle, and (j) a parking assist system of the host vehicle.

17. The object detection system of claim 1, wherein priority scores are assigned to objects detected in the field of view of said imaging device in accordance with at least one of (a) a danger of the detected objects and (b) an importance of the detected objects.

18. The object detection system of claim 1, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

19. The object detection system of claim 1, wherein said ranging device comprises a Lidar device.

20. The object detection system of claim 1, wherein said priority score is representative of the importance of the object detected in the field of view.

11. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score;

wherein said system determines multiple objects of interest in the field of view of said imaging device;

wherein said system prioritizes detected objects based on at least one of (i) proximity, of detected objects to the host vehicle, (ii) seed of detected objects relative to the host vehicle, and (iii) location of detected objects relative to the host vehicle; and

wherein said system adjusts the field of view of said ranging device toward the detected objects in accordance with the prioritization of the detected objects.

12. The object detection system of claim 11, wherein said system adjusts the field of view of said ranging device toward a first detected object for a first period of time and said system adjusts field of view of said ranging device toward a second detected object for a second period of time.

13. The object detection system of claim 12, wherein said first detected object has a higher priority than said second detected object and said first period of time is greater than said second period of time.

15. An object detection system for a host vehicle, said object detection system comprising;

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score; and

wherein, responsive to the priority score assigned to a detected object, an interrogation dwell time at the detected object is extended.

16. The object detection system of claim 15, wherein an extended interrogation dwell time is associated with a higher priority score assigned to a detected object.

21. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score;

wherein said ranging device at least one of (a) tracks a detected object, (b) tracks a detected object while the detected object moves in the field of view of said imaging device, (c) dwells on a detected object and (d) dwells on a detected object while the detected object moves in the field of view of said imaging device; and

wherein said system determines an interrogation dwell time in accordance with the prioritization of detected objects.

22. The object detection system of claim 21, wherein said system dwells at a first detected object for a first interrogation dwell time and said system dwells at a second detected object for a second interrogation dwell time.

23. The object detection system of claim 21, wherein said imaging device comprises a pixelated two-dimensional array of photosensing pixels, and wherein said system determines multiple objects of interest in the field of view of said imaging device.

24. The object detection system of claim 21, wherein said ranging device comprises at least one of (i) a radar device, (ii) a Lidar device and (iii) a stereo vision device.

25. The object detection system of claim 21, wherein said ranging device is operable to measure the distance to the detected object of interest via at least one of (i) comparing a modulation phase and (ii) calculating a time of flight of light between the host vehicle and the detected object.

26. The object detection system of claim 21, wherein said system prioritizes detected objects based on at least one of (i) proximity of detected objects to the host vehicle, (ii) speed of detected objects relative to the host vehicle, and (iii) location of detected objects relative to the host vehicle.

27. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score;

wherein said ranging device at least one of (a) tracks a detected object, (b) tracks a detected object while the detected object moves in the field of view of said imaging device, (c) dwells on a detected object and (d) dwells on a detected object while the detected object moves in the field of view of said imaging device;

wherein said system prioritizes detected objects based on at least one of (i) proximity of detected objects to the host vehicle, (ii) seed of detected objects relative to the host vehicle, and (iii) location of detected objects relative to the host vehicle; and

wherein said system dwells at the detected objects in accordance with the prioritization of detected objects.

28. The object detection system of claim 27, wherein said system dwells at a first detected object for a first interrogation dwell time and said system dwells at a second detected object for a second interrogation dwell time, and wherein said first detected object has a higher priority than said second detected object and said first interrogation dwell time is greater than said second interrogation dwell time.

29. The object detection system of claim 27, wherein said object detection system comprises at least one of (a) an adaptive cruise control system of the host vehicle, (b) a collision warning system of the host vehicle, (c) a pedestrian detection system of the host vehicle, (d) a stop-and-go system of the host vehicle, (e) a side object detection system of the host vehicle, (f) a lane change aid system of the host vehicle, (g) a side blind zone detection system of the host vehicle, (h) a forward object detection system of the host vehicle, (i) a rear blind zone object detection system of the host vehicle, and (j) a parking assist system of the host vehicle.

30. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ran ,in. device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score;

wherein said ranging device at least one of (a) tracks a detected object, (b) tracks a detected ob ct while the detected object moves in the field of view of said imaging device, (c) dwells on a detected object and (d) dwells on a detected object while the detected object moves in the field of view of said imaging device; and

wherein, responsive to the priority score assigned to a detected object, an interrogation dwell time at the detected object is extended, and wherein an extended interrogation dwell time is associated with a higher priority score assigned to a detected object.

31. The object detection system of claim 30, wherein priority scores are assigned to objects detected in the field of view of said imaging device in accordance with at least one of (a) a danger of the detected objects and (b) an importance of the detected objects.

32. The object detection system of claim 30, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

33. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle, wherein said imaging device comprises a pixelated two-dimensional array of photosensing pixels;

a ranging device having a field of view that encompasses at least the field of view of said image in device and that detects the object exterior the host vehicle wherein said ranging device comprises at least one of (i) a radar device, (ii) a Lidar device and (iii) a stereo vision device;

a processor that processes image data from said imaging device and processes data from said ranging, device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score, and wherein said system determines multiple objects of interest in the field of view of said imaging device ; and

wherein said system determines an interrogation dwell time in accordance with the prioritization of detected objects, and wherein said system dwells at a first detected object for a first interrogation dwell time and said system dwells at a second detected object for a second interrogation dwell time.

34. The object detection system of claim 33, wherein said ranging device at least one of (a) tracks a detected object, (b) tracks a detected object while the detected object moves in the field of view of said imaging device, (c) dwells on a detected object and (d) dwells on a detected object while the detected object moves in the field of view of said imaging device.

35. The object detection system of claim 33, wherein said ranging device is operable to measure the distance to the detected object of interest via at least one of (i) comparing a modulation phase and (ii) calculating a time of flight of light between the host vehicle and the detected object.

36. An object detection system for a host vehicle said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle, wherein said imaging device comprises a pixelated two-dimensional array of photosensing pixels;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle, wherein said ranging device comprises at least one of (i) a radar device, (ii) a Lidar device and (iii) a stereo vision device;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score, and wherein said system determines multiple objects of interest in the field of view of said imaging device; and

wherein said system dwells at the detected objects in accordance with the prioritization of detected objects, and wherein said system prioritizes detected objects based on at least one of (i) proximity of detected objects to the host vehicle, (ii) speed of detected objects relative to the host vehicle, and (iii) location of detected objects relative to the host vehicle.

37. The object detection system of claim 36, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

38. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle, wherein said imaging device comprises a forward facing imaging device viewing forward through a windshield of the host vehicle;

wherein said imaging device provides image data for at least one of (i) an automatic headlamp control system of the host vehicle, (ii) a collision avoidance system of the host vehicle, (iii) a stop-and-go system of the host vehicle, (iv) a vehicle detection system of the host vehicle (v) a predictive brake assist s stem of the host vehicle, (vi) an adaptive cruise control system of the host vehicle, (vii) a traffic sign recognition system of the host vehicle and (viii) a lane departure warning system of the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score; and

wherein said system determines an interrogation dwell time in accordance with the prioritization of detected objects.

39. The object detection system of claim 38, wherein said ranging device at least one of (a) tracks a detected object, (b) tracks a detected object while the detected object moves in the field of view of said imaging device, (c) dwells on a detected object and (d) dwells on a detected object while the detected object moves in the field of view of said imaging device.

40. The object detection system of claim 38, wherein said system dwells at a first detected object for a first interrogation dwell time and said system dwells at a second detected object for a second interrogation dwell time.

41. The object detection system of claim 38, wherein said imaging device comprises a pixelated two-dimensional array of photosensing pixels, and wherein said system determines multiple objects of interest in the field of view of said imaging device.

42. The object detection system of claim 38, wherein said ranging device comprises at least one of (i) a radar device, (ii) a Lidar device and (iii) a stereo vision device.

43. The object detection system of claim 38, wherein said system prioritizes detected objects based on at least one of (i) proximity of detected objects to the host vehicle, (ii) speed of detected objects relative to the host vehicle, and (iii) location of detected objects relative to the host vehicle.

45. The object detection system of claim 38, wherein priority scores are assigned to objects detected in the field of view of said imaging device in accordance with at least one of (a) a danger of the detected objects and (b) an importance of the detected objects.

46. The object detection system of claim 38, wherein a higher priority score is assigned to a detected object if the detected object is determined to be at least one of (a) a vehicle that is close to the host vehicle, (b) a vehicle that is closing to the host vehicle in relative speed, (c) a vehicle that is closing to the host vehicle in relative speed and collision will happen in pre-determined time, (d) a vehicle that is in the same lane of the host vehicle, (e) a vehicle that is moving to the same lane of the host vehicle, (f) a pedestrian and (g) a pedestrian moving toward the center of the field of view of said imaging device.

44. An object detection system for a host vehicle, said object detection system comprising:

an imaging device having a field of view exterior of the host vehicle and operable to capture image data representative of an object exterior the host vehicle, wherein said imaging device comprises a forward facing imaging device viewing forward through a windshield of the host vehicle;

wherein said imaging device provides image data for at least one of (i) an automatic headlamp control system of the host vehicle, (ii) a collision avoidance system of the host vehicle, (iii) a stop-and-go system of the host vehicle. (iv) a vehicle detection system of the host vehicle a predictive brake assist system of the host vehicle, (vi) an adaptive cruise control s stem of the host vehicle (vii) a traffic sign recognition system of the host vehicle and (viii) a lane departure warning system of the host vehicle;

a ranging device having a field of view that encompasses at least the field of view of said imaging device and that detects the object exterior the host vehicle;

a processor that processes image data from said imaging device and processes data from said ranging device in order to determine if the detected object is an object of interest;

wherein said determination that the detected object is an object of interest is made in accordance with a priority score;

wherein said system prioritizes detected objects based on at least one of (i) proximity of detected objects to the host vehicle (ii) speed of detected objects relative to the host vehicle and (iii) location of detected objects relative to the host vehicle; and

wherein said system dwells at the detected objects in accordance with the prioritization of detected objects, and wherein said system dwells at a first detected object for a first interrogation dwell time and said system dwells at a second detected object for a second interrogation dwell time, and wherein said first detected object has a higher priority than said second detected object and said first interrogation dwell time is greater than said second interrogation dwell time.