Vehicular vision system with image processing

• US 10,474,909 B2
• Filed: 02/18/2019
• Issued: 11/12/2019
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Sabeti, Leila
• Primary Examiner(s): Chio, Tat C
• Assistant Examiner(s): Nguyen, Kathleen V
• Application Number:
  US16/278,294
• Publication Number:
  US 20190180114A1
• Time in Patent Office: 267 Days
• Field of Search: 348148
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A vehicular vision system includes a camera and an image processor that processes frames of captured image data to determine a path of travel of the vehicle. The system determines some image data points to be ground and tracks image data of a three dimensional object present in the field of view of the camera to determine movement of the image data of the object relative to the determined ground. Responsive to a determined ground location of a determined bicycle and responsive to the known height of the camera, the system determines distance to the bicycle. At least in part responsive to the bicycle being within a threshold distance to the vehicle and in the determined path of travel of the vehicle, the system generates an output indicative of determination of the bicycle being in the determined path of travel of the vehicle.

Claims

1. A vehicular vision system, said vehicular vision system comprising:

a CMOS camera disposed at a windshield of a vehicle and having a field of view at least forward and through the windshield of the vehicle;

wherein said camera comprises a two dimensional imaging array of photosensing elements arranged in at least 1200 columns of photosensing elements and at least 800 rows of photosensing elements;

wherein said camera, when disposed at the vehicle, is at a known height relative to a ground surface at which the vehicle is located;

an image processor operable to process frames of image data captured by said camera;

wherein said image processor, using a kinematics model of the vehicle, processes two or more frames of captured image data to determine a path of travel of the vehicle;

wherein said vehicular vision system uses ground points segmentation and the kinematics model of the vehicle to determine distance traveled by the vehicle between frames of captured image data;

wherein said vehicular vision system determines image data points that travel the same distance as the vehicle but in a direction opposite to that of the vehicle to be ground;

wherein said vehicular vision system tracks image data of a three dimensional object present in the field of view of said camera to determine movement of the image data of the three dimensional object relative to the determined ground;

wherein, responsive to tracking the image data of the three dimensional object, said vehicular vision system determines presence of a bicycle ahead of the vehicle and in the field of view of said camera;

wherein, responsive to processing of captured image data by said image processor, said vehicular vision system determines a distal portion of the determined bicycle relative to the vehicle and determines a ground location at which the determined distal portion of the determined bicycle is located;

wherein, responsive to the determined ground location of the determined distal portion of the determined bicycle and responsive to the known height of said camera relative to the ground surface, said vehicular vision system determines distance to the determined bicycle; and

wherein, at least in part responsive to the determined bicycle being within a threshold distance to the vehicle, and at least in part responsive to the determined bicycle being in the determined path of travel of the vehicle, said vehicular vision system generates an output indicative of determination of the bicycle being in the determined path of travel of the vehicle.

2. The vehicular vision system of claim 1, wherein the output comprises an alert to a driver of the vehicle.

3. The vehicular vision system of claim 1, wherein said vehicular vision system is operable to determine distance to the bicycle using two dimensional information.

4. The vehicular vision system of claim 1, wherein the distance determination uses foreground segmentation utilizing a Gaussian mixture model and wherein said Gaussian mixture model has 3 Gaussian distributions.

5. The vehicular vision system of claim 1, wherein said vehicular vision system is operable to determine the presence of the bicycle responsive at least in part to a determination of a sudden illumination change via processing of captured image data.

6. The vehicular vision system of claim 5, wherein, responsive to a determination of the sudden illumination change, said vehicular vision system merges an illumination invariant edge based technique.

7. The vehicular vision system of claim 1, wherein said image processor tracks the determined bicycle over multiple frames of captured image data.

8. The vehicular vision system of claim 1, wherein the bicycle is determined at least in part using block determination.

9. The vehicular vision system of claim 1, wherein said vehicular vision system is operable to (i) use a first algorithm to rapidly find the three dimensional object and (ii) use a second algorithm to increase accuracy while choosing how and where to search to enhance rapid determination of a location of the three dimensional object.

10. The vehicular vision system of claim 9, wherein the second algorithm selects where to search to determine the location of the three dimensional object.

11. A vehicular vision system, said vehicular vision system comprising:

a CMOS camera disposed at a vehicle and having a field of view at least sideward and to the rear of the vehicle;

wherein said camera comprises a two dimensional imaging array of photosensing elements arranged in at least 1200 columns of photosensing elements and at least 800 rows of photosensing elements;

wherein said camera, when disposed at the vehicle, is at a known height relative to a ground surface at which the vehicle is located;

an image processor operable to process frames of image data captured by said camera;

wherein said image processor, using a kinematics model of the vehicle, processes two or more frames of captured image data to determine motion of the vehicle;

wherein said vehicular vision system uses ground points segmentation and the kinematics model of the vehicle to determine distance traveled by the vehicle between frames of captured image data;

wherein said vehicular vision system determines image data points that travel the same distance as the vehicle but in a direction opposite to that of the vehicle to be ground;

wherein said vehicular vision system tracks image data of a three dimensional object present in the field of view of said camera to determine movement of the image data of the three dimensional object relative to the determined ground;

wherein, responsive to tracking the image data of the three dimensional object, said vehicular vision system determines presence of a bicycle sideward and rearward of the vehicle and in the field of view of said camera;

wherein, responsive to processing of captured image data by said image processor, said vehicular vision system determines a distal portion of the determined bicycle relative to the vehicle and determines a ground location at which the determined distal portion of the determined bicycle is located;

wherein, responsive to the determined ground location of the determined distal portion of the determined bicycle and responsive to the known height of said camera relative to the ground surface, said vehicular vision system determines distance to the determined bicycle; and

wherein, at least in part responsive to the determined bicycle being within a threshold distance to the vehicle, and at least in part responsive to the determined bicycle approaching the vehicle from the rear, said vehicular vision system generates an output indicative of determination of the bicycle approaching the vehicle from the rear.

12. The vehicular vision system of claim 11, wherein the output comprises an alert to a driver of the vehicle.

13. The vehicular vision system of claim 11, wherein said camera is disposed at a side of the vehicle.

14. The vehicular vision system of claim 11, wherein said image processor tracks the determined bicycle over multiple frames of captured image data.

15. The vehicular vision system of claim 11, wherein said vehicular vision system is operable to (i) use a first algorithm to rapidly find the three dimensional object and (ii) use a second algorithm to increase accuracy while choosing how and where to search to enhance rapid determination of a location of the three dimensional object, and wherein the second algorithm selects where to search to determine the location of the three dimensional object.

16. A vehicular vision system, said vehicular vision system comprising:

a CMOS camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera comprises a two dimensional imaging array of photosensing elements arranged in at least 1200 columns of photosensing elements and at least 800 rows of photosensing elements;

wherein said camera, when disposed at the vehicle, is at a known height relative to a ground surface at which the vehicle is located;

an image processor operable to process frames of image data captured by said camera;

wherein said image processor, using a kinematics model of the vehicle, processes two or more frames of captured image data to determine motion of the vehicle;

wherein said vehicular vision system uses ground points segmentation and the kinematics model of the vehicle to determine distance traveled by the vehicle between frames of captured image data;

wherein said vehicular vision system determines image data points that travel the same distance as the vehicle but in a direction opposite to that of the vehicle to be ground;

wherein said vehicular vision system tracks image data of a three dimensional object present in the field of view of said camera to determine movement of the image data of the three dimensional object relative to the determined ground;

wherein, responsive to tracking the image data of the three dimensional object, said vehicular vision system determines presence of a bicycle in the field of view of said camera;

wherein said image processor tracks the determined bicycle over multiple frames of captured image data;

wherein, responsive to processing of captured image data by said image processor, said vehicular vision system determines a distal portion of the determined bicycle relative to the vehicle and determines a ground location at which the determined distal portion of the determined bicycle is located;

wherein, responsive to the determined ground location of the determined distal portion of the determined bicycle and responsive to the known height of said camera relative to the ground surface, said vehicular vision system determines distance to the determined bicycle; and

wherein, at least in part responsive to the determined bicycle being within a threshold distance to the vehicle, and at least in part responsive to the determined bicycle approaching the vehicle, said vehicular vision system generates an output indicative of determination of the bicycle approaching the vehicle.

17. The vehicular vision system of claim 16, wherein said vehicular vision system is operable to (i) use a first algorithm to rapidly find the three dimensional object and (ii) use a second algorithm to increase accuracy while choosing how and where to search to enhance rapid determination of a location of the three dimensional object.

18. The vehicular vision system of claim 17, wherein the second algorithm selects where to search to determine the location of the three dimensional object.

19. The vehicular vision system of claim 18, wherein said camera is disposed at a windshield of a vehicle and has a field of view at least forward and through the windshield of the vehicle, and wherein, at least in part responsive to the determined bicycle being in a determined path of travel of the vehicle, said vehicular vision system generates an output indicative of determination of the bicycle being in the determined path of travel of the vehicle.

20. The vehicular vision system of claim 18, wherein said camera is disposed at the vehicle so as to have a field of view at least sideward and to the rear of the vehicle, and wherein, at least in part responsive to the determined bicycle approaching the vehicle from the rear, said vehicular vision system generates an output indicative of determination of the bicycle approaching the vehicle from the rear.