Vehicle vision system with object detection

• US 10,204,279 B2
• Filed: 07/28/2016
• Issued: 02/12/2019
• Est. Priority Date: 07/30/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil, Zhang, Liang
• Primary Examiner(s): Saini, Amandeep
• Application Number:
  US15/221,912
• Publication Number:
  US 20170032198A1
• Time in Patent Office: 929 Days
• Field of Search: 382104
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vision system of a vehicle includes at least one camera disposed at a vehicle and having a field of view exterior of the vehicle, and an image processor operable to process image data captured by the camera. Responsive to image processing of captured image data, the image processor determines objects present in the field of view of the camera. The vision system processes additional frames of captured image data to enhance determination of objects of interest. The vision system initially detects objects present in the field of view of the camera and conducts hypotheses filtering and hypotheses merging and, responsive to the hypotheses filtering and hypotheses merging, the system predicts hypotheses and evaluates the predicted hypotheses to determine objects of interest from objects not of interest.

Claims

1. A vision system for a vehicle, said vision system comprising:

at least one camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing by said image processor of captured image data, objects present in the field of view of said camera are detected;

wherein, after detection of objects via image processing by said image processor of captured image data, said vision system processes additional frames of captured image data to enhance determination of objects of interest;

wherein enhancement of determination of objects of interest by said vision system comprises hypotheses filtering and hypotheses merging; and

wherein enhancement of determination of an object of interest by said vision system comprises hypotheses prediction based at least in part on an output of hypotheses merging.

2. The vision system of claim 1, wherein said system repeats the hypotheses merging and hypotheses prediction to enhance determination of objects of interest.

3. The vision system of claim 1, wherein hypotheses merging and hypotheses prediction comprises use of Kalman filtering.

4. A vision system for a vehicle, said vision system comprising:

at least one camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing by said image processor of captured image data, objects present in the field of view of said camera are detected;

wherein, after detection of objects via image processing by said image processor of captured image data, said vision system processes additional frames of captured image data to enhance determination of objects of interest;

wherein enhancement of determination of objects of interest by said vision system comprises hypotheses filtering and hypotheses merging; and

wherein enhancement of determination of objects of interest by said vision system comprises (i) hypotheses filtering, which is responsive to an initial detection of objects via image processing of captured image data, (ii) hypotheses merging, which is responsive to an output of hypotheses filtering, (iii) hypotheses prediction, which is responsive to an output of hypotheses merging, and (iv) hypotheses evaluation, which is responsive to an output of hypotheses prediction.

5. A vision system for a vehicle, said vision system comprising:

at least one camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing by said image processor of captured image data, objects present in the field of view of said camera are detected;

wherein, after detection of objects via image processing by said image processor of captured image data, said vision system processes additional frames of captured image data to enhance determination of objects of interest;

wherein enhancement of determination of objects of interest by said vision system comprises hypotheses filtering and hypotheses merging; and

wherein hypotheses filtering comprises tracking of detected objects over multiple frames of captured image data.

6. The vision system of claim 5, wherein said camera comprises a fisheye lens.

8. The vision system of claim 5, wherein, responsive to said hypotheses filtering and hypotheses merging, said system predicts hypotheses and evaluates the predicted hypotheses to determine objects of interest from objects not of interest.

9. The vision system of claim 5, wherein enhancement of determination of objects of interest by said vision system comprises hypotheses evaluation based at least in part on predicted hypotheses.

10. The vision system of claim 5, wherein detection of objects comprises use of a low pass filter of captured image data prior to image processing of captured image data.

7. A vision system for a vehicle, said vision system comprising:

at least one camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing by said image processor of captured image data, objects present in the field of view of said camera are detected;

wherein, after detection of objects via image processing by said image processor of captured image data, said vision system processes additional frames of captured image data to enhance determination of objects of interest;

wherein enhancement of determination of objects of interest by said vision system comprises hypotheses filtering and hypotheses merging; and

wherein hypotheses merging comprises comparing outputs of hypotheses filtering with predicted hypotheses derived from previous frames of captured image data.

11. The vision system of claim 7, wherein said vision system processes multiple frames of captured image data to track an object determined to be present in the field of view of said camera.

12. The vision system of claim 7, wherein filtered hypotheses and predicted hypotheses are combined to generate an assigned hypotheses.

13. The vision system of claim 12, wherein the assigned hypotheses is input into a hypotheses prediction module, and wherein an output of the hypotheses prediction module is (i) input to hypotheses merging and (ii) input into a hypotheses evaluation module.

14. A vision system of a vehicle, said vision system comprising:

at least one camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a fisheye lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing by said image processor of captured image data, objects present in the field of view of said camera are detected;

wherein, after detection of objects via image processing by said image processor of captured image data, said vision system processes additional frames of captured image data to enhance determination of objects of interest;

wherein enhancement of determination of objects of interest by said vision system comprises use of (i) a hypotheses filtering module, which is responsive to an initial detection of objects via image processing of captured image data, (ii) a hypotheses merging module, which is responsive to an output of the hypotheses filtering module, (iii) a hypotheses prediction module, which is responsive to an output of the hypotheses merging module, and (iv) a hypotheses evaluation module, which is responsive to an output of the hypotheses prediction module; and

wherein the output of said hypotheses prediction module is input to the hypotheses merging module and to the hypotheses evaluation module, and wherein the hypotheses merging module compares the output of the hypotheses filtering module and the output of the hypotheses prediction module, and wherein the output of the hypotheses filtering module and the output of the hypotheses prediction module are combined by the hypotheses merging module to generate a subsequent output of the hypotheses merging module, which is input to the hypotheses prediction module.

15. The vision system of claim 14, wherein said vision system processes multiple frames of captured image data to track an object determined to be present in the field of view of said camera.

16. The vision system of claim 14, wherein, responsive to the output of said hypotheses merging module, said hypotheses prediction module predicts hypotheses and evaluates the predicted hypotheses to determine objects of interest from objects not of interest.

17. A vision system of a vehicle, said vision system comprising:

at least one camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing by said image processor of captured image data, objects present in the field of view of said camera are detected;

wherein said vision system processes multiple frames of captured image data to track an object determined to be present in the field of view of said camera;

wherein, after detection of objects via image processing by said image processor of captured image data, said vision system enhances determination of objects of interest by use of (i) a hypotheses filtering module, which is responsive to an initial detection of objects via image processing of captured image data, (ii) a hypotheses merging module, which is responsive to an output of the hypotheses filtering module, (iii) a hypotheses prediction module, which is responsive to an output of the hypotheses merging module, and (iv) a hypotheses evaluation module, which is responsive to an output of the hypotheses prediction module; and

wherein the output of said hypotheses prediction module is input to the hypotheses merging module and to the hypotheses evaluation module, and wherein the hypotheses merging module compares the output of the hypotheses filtering module and the output of the hypotheses prediction module, and wherein the output of the hypotheses filtering module and the output of the hypotheses prediction module are combined by the hypotheses merging module to generate a subsequent output of the hypotheses merging module, which is input to the hypotheses prediction module.