Vehicle vision system with 3D registration for distance estimation

• US 10,713,506 B2
• Filed: 12/15/2015
• Issued: 07/14/2020
• Est. Priority Date: 12/18/2014
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil, Okouneva, Galina, Zhang, Liang
• Primary Examiner(s): Pendleton, Brian T
• Assistant Examiner(s): Lotfi, Kyle M
• Application Number:
  US14/969,527
• Publication Number:
  US 20160180182A1
• Time in Patent Office: 1,673 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vision system of a vehicle includes at least one camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle. Responsive to image processing of captured image data and with the at least one camera disposed at the vehicle, the image processor determines a three dimensional object present in the field of view of the camera and determines a point of interest on the determined object. The vision system uses triangulation to determine an estimated location in three dimensional space of the determined point of interest. The vision system processes additional frames of captured image data to enhance the estimation of the location in three dimensional space of the determined point of interest. The image processor is operable to estimate a distance to the determined object by comparing multiple frames of captured image data.

Claims

1. A vision system for a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing of captured image data and with said camera disposed at the vehicle, said image processor determines a three dimensional object present in the field of view of said camera;

wherein multiple frames of captured image data are processed, as the vehicle moves relative to the determined object, to determine a point of interest, present in each frame of captured image data, on the determined object;

wherein said vision system, via processing of the multiple frames of image data captured by said camera as the vehicle moves relative to the determined object, uses triangulation based on the determined point of interest present in each frame of the multiple frames of image data captured by said camera to determine an estimated location in three dimensional space of the determined point of interest;

wherein, responsive to the determined estimated location in three dimensional space of the determined point of interest, said vision system processes additional frames of captured image data to enhance the estimation of the location in three dimensional space of the determined point of interest; and

wherein said image processor estimates distance to the determined point of interest on the determined object by comparing, one to another, frames of captured image data where there is movement of the determined point of interest of the determined object relative to said camera disposed at the vehicle.

2. The vision system of claim 1, wherein said system repeats the processing to enhance the estimated location in three dimensional space of the determined point of interest until a difference between subsequent estimations is less than a threshold level.

3. The vision system of claim 1, wherein determined points of interest in two or more image frames are compared to determine if they match.

4. The vision system of claim 1, wherein said vision system is operable to detect points of interest in each frame of captured image data and perform correspondence analysis to find matched point pairs in two or more frames of captured image data.

5. The vision system of claim 1, wherein said vision system utilizes triangulation to determine an initial location of a determined point of interest from a corresponding pair of points of interest in two or more frames of image data.

6. The vision system of claim 5, wherein a third frame of image data is processed to refine the determined initial location of the point of interest.

7. The vision system of claim 1, wherein the estimation of the location in three dimensional space of the determined point of interest is enhanced responsive to processing of additional frames of image data.

8. The vision system of claim 7, wherein, responsive to processing of frames of captured image data, said vision system (a) refines correspondent feature locations by comparing similarities between projected and detected points of interest, (b) selecting corresponding features according to their determined similarities, and (c) repeating triangulation on newly selected corresponding pairs of points of interest.

9. The vision system of claim 1, wherein said camera comprises a wide angle lens and wherein said camera, when disposed at the vehicle, has a wide angle field of view exterior of the vehicle.

10. A vision system for a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

wherein said camera comprises a wide angle lens and wherein said camera, when disposed at the vehicle, has a wide angle field of view exterior of the vehicle;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing of captured image data and with said camera disposed at the vehicle, said image processor determines a three dimensional object present in the field of view of said camera;

wherein multiple frames of captured image data are processed, as the vehicle moves relative to the determined object, to determine a point of interest, present in each frame of captured image data, on the determined object;

wherein said vision system, via processing of the multiple frames of image data captured by said camera as the vehicle moves relative to the determined object, uses triangulation based on the determined point of interest present in each frame of the multiple frames of image data captured by said camera to determine an estimated location in three dimensional space of the determined point of interest;

wherein, responsive to the determined estimated location in three dimensional space of the determined point of interest, said vision system processes additional frames of captured image data to enhance the estimation of the location in three dimensional space of the determined point of interest;

wherein said image processor is operable to estimate a distance to the determined point of interest on the determined object by comparing, one to another, frames of captured image data where there is movement of the determined point of interest of the determined object relative to said camera disposed at the vehicle; and

wherein said system repeats the processing to enhance the estimated location in three dimensional space of the determined point of interest until a difference between subsequent estimations is less than a threshold level.

11. The vision system of claim 10, wherein determined points of interest in two or more image frames are compared to determine if they match.

12. The vision system of claim 10, wherein said vision system is operable to detect points of interest in each frame of captured image data and perform correspondence analysis to find matched point pairs in two or more frames of captured image data.

13. The vision system of claim 10, wherein said vision system utilizes triangulation to determine an initial location of a point of interest from a corresponding pair of points of interest in two or more frames of image data.

14. The vision system of claim 13, wherein a third frame of image data is processed to refine the determined initial location of the point of interest.

15. The vision system of claim 10, wherein the estimation of the location in three dimensional space of the determined point of interest is enhanced responsive to processing of additional frames of image data.

16. The vision system of claim 15, wherein, responsive to processing of frames of captured image data, said vision system (a) refines correspondent feature locations by comparing similarities between projected and detected points of interest, (b) selecting corresponding features according to their determined similarities, and (c) repeating triangulation on newly selected corresponding pairs of points of interest.

17. A vision system for a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

wherein said camera comprises a wide angle lens and wherein said camera, when disposed at the vehicle, has a wide angle field of view exterior of the vehicle;

an image processor operable to process image data captured by said camera;

wherein, responsive to image processing of captured image data and with said camera disposed at the vehicle, said image processor determines a three dimensional object present in the field of view of said camera;

wherein multiple frames of captured image data are processed, as the vehicle moves relative to the determined object, to determine a point of interest, present in each frame of captured image data, on the determined object;

wherein said vision system, via processing of the multiple frames of image data captured by said camera as the vehicle moves relative to the determined object, uses triangulation based on the determined point of interest present in each frame of the multiple frames of image data captured by said camera to determine an estimated location in three dimensional space of the point of interest;

wherein, responsive to the determined estimated location in three dimensional space of the determined point of interest, said vision system processes additional frames of captured image data to enhance the estimation of the location in three dimensional space of the point of interest;

wherein said image processor is operable to estimate a distance to the determined point of interest on the determined object by comparing, one to another, at least three frames of captured image data where there is movement of the determined point of interest of the determined object relative to said camera disposed at the vehicle; and

wherein said system repeats the processing to enhance the estimated location in three dimensional space of the determined point of interest until a difference between subsequent estimations is less than a threshold level.

18. The vision system of claim 17, wherein said vision system is operable to detect points of interest in each frame of captured image data and perform correspondence analysis to find matched point pairs in two or more frames of captured image data.

19. The vision system of claim 17, wherein the estimation of the location in three dimensional space of the determined point of interest is enhanced responsive to processing of additional frames of image data.

20. The vision system of claim 19, wherein, responsive to processing of frames of captured image data, said vision system (a) refines correspondent feature locations by comparing similarities between projected and detected points of interest, (b) selecting corresponding features according to their determined similarities, and (c) repeating triangulation on newly selected corresponding pairs of points of interest.