Vision system for vehicle

• US 10,099,614 B2
• Filed: 11/27/2012
• Issued: 10/16/2018
• Est. Priority Date: 11/28/2011
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Diessner, Horst Daniel
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): Jiang, Zaihan
• Application Number:
  US14/359,340
• Publication Number:
  US 20140313339A1
• Time in Patent Office: 2,149 Days
• Field of Search: 348148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G02B 3/14 of variable focal length
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A vision system for a vehicle includes an image sensor disposed at a subject vehicle and having a field of view exterior of the subject vehicle. A control is operable to process image data captured by the image sensor to detect an object exterior of the subject vehicle. The control is operable to process captured image data to detect points of interest present in the field of view of the image sensor and, responsive to processing of captured image data, the control is operable to determine movement of the detected points of interest. The control is operable to process captured image data to determine movement vectors and, responsive to processing of captured image data, the control is operable to determine an object of interest in the field of view of the image sensor and exterior of the subject vehicle.

Claims

1. A vision system for a vehicle, said vision system comprising:

an image sensor disposed at a subject vehicle and having an exterior field of view exterior of the subject vehicle;

a control, wherein said control comprises a processor that processes image data captured by said image sensor to detect objects present exterior of the subject vehicle and in the field of view of said image sensor;

wherein said control processes captured image data to detect points of interest of an object present in the field of view of said image sensor, and wherein detected points of interest of the detected object comprise detected corner features of the detected object that is present in the field of view of said image sensor, and wherein the detected object has a height above ground, and wherein the detected corner features of the detected object are disposed at respective heights of the detected object above ground;

wherein said control processes captured image data to determine movement vectors of the detected points of interest of the detected object, and wherein, responsive to said processing of captured image data, said control determines whether determined movement vectors of detected points of interest of the detected object present in the field of view of said image sensor are indicative of an object of interest present in the field of view of said image sensor;

wherein a parabolic region of interest in which objects exterior of the subject vehicle are likely objects of interest is defined by a parabolic boundary having an apex located centrally within the field of view of said image sensor and above a middle region of the field of view of said image sensor; and

wherein determined movement vectors that are less than a threshold pixel length are not considered indicative of an object of interest, and wherein the detected object is not considered an object of interest when the detected corner features of the detected object are above and outside the parabolic boundary of the parabolic region of interest.

2. The vision system of claim 1, wherein said control determines an object of interest by utilizing a first and a second detection method in combination.

3. The vision system of claim 2, wherein said first method for determining objects of interest comprises at least one of Gaussian blurring and consecutive Hessian blob detection.

4. The vision system of claim 2, wherein said second method for determining objects of interest comprises a FAST9 corner detector.

5. The vision system of claim 1, wherein said control operates in one of a plurality of stages.

6. The vision system of claim 5, wherein said plurality of stages has at least four different stages comprising (i) vehicle stationary and object stationary, (ii) vehicle stationary and object moving, (iii) vehicle moving and object stationary and (iv) vehicle moving and object moving.

7. The vision system of claim 6, wherein said control determines objects of interest that have been detected in a different stage earlier when the control is in the stage of vehicle stationary and object stationary.

8. The vision system of claim 6, wherein said control determines objects of interest by analyzing movement vectors when said control is in the stage vehicle stationary and object moving or in the stage of vehicle moving and object moving.

9. The vision system of claim 8, wherein said control employs a prediction algorithm to continue tracking of objects which have been detected in previous sequential frames even when these are not detectable in a present frame by movement vector interpolation.

10. The vision system of claim 1, wherein said control determines that detected points of interest are part of an object of interest when at least one movement vector is greater than a threshold amount.

11. The vision system of claim 1, wherein said image sensor comprises part of a camera module.

12. The vision system of claim 11, wherein said camera module includes said control.

13. The vision system of claim 12, wherein said camera module is configured for use as a rear backup camera of a vehicle.

14. The vision system of claim 1, wherein said image sensor is disposed at a rear portion of the subject vehicle and has an exterior field of view rearward of the subject vehicle, and wherein said control is operable to determine an object of interest in the field of view of said image sensor and rearward of the subject vehicle.

15. A vision system for a vehicle, said vision system comprising:

an image sensor disposed at a rear portion of a subject vehicle and having an exterior field of view rearward of the subject vehicle;

wherein said image sensor comprises part of a camera module, and wherein said camera module is configured for use as a rear backup camera of a vehicle;

a control, wherein said control comprises a processor that processes image data captured by said image sensor to detect objects present exterior of the subject vehicle and in the field of view of said image sensor;

wherein said camera module includes said control;

wherein said control processes captured image data to detect points of interest of an object present in the field of view of said image sensor, and wherein detected points of interest of the detected object comprise detected corner features of the detected object that is present in the field of view of said image sensor, and wherein the detected object has a height above ground, and wherein the detected corner features of the detected object are disposed at respective heights of the detected object above ground;

wherein said control processes captured image data to determine movement vectors of detected points of interest of the detected object, and wherein, responsive to said processing of captured image data, said control determines whether determined movement vectors of detected points of interest of the detected object present in the field of view of said image sensor are indicative of an object of interest present in the field of view of said image sensor and rearward of the subject vehicle;

wherein a parabolic region of interest in which objects exterior of the subject vehicle are likely objects of interest is defined by a parabolic boundary having an apex located centrally within the field of view of said image sensor and above a middle region of the field of view of said image sensor; and

wherein determined movement vectors that are less than a threshold pixel length are not considered indicative of an object of interest, and wherein the detected object is not considered an object of interest when the detected corner features of the detected object are above and outside the parabolic boundary of the parabolic region of interest.

16. The vision system of claim 15, wherein said control determines that detected points of interest are part of an object of interest when at least one movement vector is greater than a threshold amount.

17. The vision system of claim 15, wherein said control operates in one of at least four different stages comprising (i) vehicle stationary and object stationary, (ii) vehicle stationary and object moving, (iii) vehicle moving and object stationary and (iv) vehicle moving and object moving.

18. The vision system of claim 17, wherein at least one of (i) said control determines objects of interest that have been detected in a different stage earlier when the control is in the stage of vehicle stationary and object stationary and (ii) said control determines objects of interest by analyzing movement vectors when said control is in the stage vehicle stationary and object moving or in the stage of vehicle moving and object moving.

19. A vision system for a vehicle, said vision system comprising:

an image sensor disposed at a subject vehicle and having an exterior field of view exterior of the subject vehicle;

wherein said image sensor comprises part of a camera module;

a control, wherein said control comprises a processor that processes image data captured by said image sensor to detect objects present exterior of the subject vehicle and in the field of view of said image sensor;

wherein said camera module includes said control;

wherein said control processes captured image data to detect points of interest of an object present in the field of view of said image sensor, and wherein detected points of interest of the detected object comprise detected corner features of the detected object that is present in the field of view of said image sensor, and wherein the detected object has a height above ground, and wherein the detected corner features of the detected object are disposed at respective heights of the detected object above ground;

wherein said control processes captured image data to determine movement vectors of the detected points of interest of the detected object, and wherein, responsive to said processing of captured image data, said control determines whether determined movement vectors of detected points of interest of the detected object present in the field of view of said image sensor are indicative of an object of interest present in the field of view of said image sensor;

wherein a parabolic region of interest in which objects exterior of the subject vehicle are likely objects of interest is defined by a parabolic boundary having an apex located centrally within the field of view of said image sensor and above a middle region of the field of view of said image sensor;

wherein said control determines that detected points of interest are part of an object of interest when at least one determined movement vector associated with a detected point of interest is greater than a threshold amount; and

wherein determined movement vectors that are less than a threshold pixel length are not considered indicative of an object of interest, and wherein the detected object is not considered an object of interest when the detected corner features of the detected object are above and outside the parabolic boundary of the parabolic region of interest.

20. The vision system of claim 19, wherein said camera module is configured for use as a rear backup camera of a vehicle, wherein said camera module is disposed at a rear portion of the subject vehicle and said image sensor has an exterior field of view rearward of the subject vehicle, and wherein said control is operable to determine an object of interest in the field of view of said image sensor and rearward of the subject vehicle.