Vehicle vision system with image processing

• US 10,210,399 B2
• Filed: 12/16/2014
• Issued: 02/19/2019
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Sabeti, Leila
• Primary Examiner(s): Atala, Jamie J
• Assistant Examiner(s): Nguyen, Kathleen V
• Application Number:
  US14/572,017
• Publication Number:
  US 20150175072A1
• Time in Patent Office: 1,526 Days
• Field of Search: 348148
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A vision system of a vehicle includes a camera and an image processor operable to process image data captured by the camera. Responsive to processing of captured image data by the image processor, the image processor determines the presence of a pedestrian in the field of view of the camera, with the determination using foreground segmentation utilizing a Gaussian mixture model. Responsive to processing of captured image data by the image processor, the vision system determines that a ground location of a distal portion of the determined pedestrian. Responsive to the determined ground location of the distal portion of the determined pedestrian, the vision system determines a distance to the determined pedestrian. At least in part responsive to the determined pedestrian being within a threshold distance to the vehicle, the vision system generates an alert to a driver of the vehicle.

Claims

1. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera, when disposed at the vehicle, is at a known height relative to a ground surface at which the vehicle is located;

an image processor operable to process image data captured by said camera;

wherein said image processor, using a kinematics model of the vehicle, processes two or more frames of captured image data to determine a path of travel of the vehicle;

wherein said system uses ground points segmentation and the kinematics model of the vehicle to determine distance traveled by the vehicle between two or more frames of captured image data;

wherein said system identifies as ground points image data points that travel the same distance as the vehicle but in a direction opposite to that of the vehicle, and wherein said system identifies other image data points as points of a three dimensional object;

wherein said system monitors the determined points of a three dimensional object to determine their movement, while not monitoring the determined ground points;

wherein, responsive to processing of captured image data by said image processor and monitoring the determined points of a three dimensional object, said image processor determines presence of a pedestrian in the field of view of said camera;

wherein, responsive to processing of captured image data by said image processor, said vision system determines a distal portion of the determined pedestrian relative to the vehicle and determines a ground location at which the determined distal portion of the determined pedestrian is located;

wherein, responsive to the determined ground location of the determined distal portion of the determined pedestrian and responsive to the known height of said camera relative to the ground surface, said vision system determines a distance to the determined pedestrian; and

wherein, at least in part responsive to the determined pedestrian being within a threshold distance to the vehicle, and at least in part responsive to the determined pedestrian being in the determined path of the vehicle, said vision system generates an alert to a driver of the vehicle.

2. The vision system of claim 1, wherein said camera is disposed at a side of the vehicle.

3. The vision system of claim 1, wherein said vision system is operable to determine the presence of the pedestrian responsive at least in part to a determination of a sudden illumination change in captured image data.

4. The vision system of claim 3, wherein, responsive to a determination of a sudden illumination change, said vision system merges an illumination invariant edge based technique.

5. The vision system of claim 1, wherein said vision system is operable to determine distance to the pedestrian using two dimensional information.

6. The vision system of claim 1, wherein said determination uses foreground segmentation utilizing a Gaussian mixture model and wherein said Gaussian mixture model has 3 Gaussian distributions.

7. The vision system of claim 1, wherein said camera comprises a two dimensional imaging array of photosensing elements and wherein said imaging array comprises a matrix array of photosensing elements comprising at least 1200 columns of photosensing elements and at least 800 rows of photosensing elements.

8. The vision system of claim 1, wherein said image processor is operable to detect and track an object over multiple frames of captured image data by determining the object using a block determination.

9. The vision system of claim 8, wherein said vision system is operable to use (i) a first algorithm to rapidly find the object, (ii) a second algorithm to increase the accuracy while checking the search area and choosing how and where to search to enhance rapid determination of a location of the object, and (iii) a third algorithm that looks at a larger area for faster moving objects.

10. The vision system of claim 9, wherein said second algorithm selects where to search, so with just a few search points said vision system can determine the location of the object.

11. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera, when disposed at the vehicle, is at a known height relative to a ground surface at which the vehicle is located;

wherein said camera comprises a two dimensional matrix array of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said image processor, using a kinematics model of the vehicle, processes two or more frames of captured image data to determine a path of travel of the vehicle;

wherein said system uses ground points segmentation and the kinematics model of the vehicle to determine distance traveled by the vehicle between two or more frames of captured image data;

wherein said system identifies as ground points image data points that travel the same distance as the vehicle but in a direction opposite to that of the vehicle, and wherein said system identifies other image data points as points of a three dimensional object;

wherein said system monitors the determined points of a three dimensional object to determine their movement, while not monitoring the determined ground points;

wherein, responsive to processing of captured image data by said image processor and monitoring the determined points of a three dimensional object, said image processor determines presence of a pedestrian in the field of view of said camera;

wherein, responsive to processing of captured image data by said image processor, said vision system determines a distal portion of the determined pedestrian relative to the vehicle and determines a ground location at which the determined distal portion of the determined pedestrian is located;

wherein, responsive to the determined ground location of the determined distal portion of the determined pedestrian and responsive to the known height of said camera relative to the ground surface, said vision system determines a distance to the determined pedestrian;

wherein, at least in part responsive to the determined pedestrian being within a threshold distance to the vehicle, and at least in part responsive to the determined pedestrian being in the determined path of the vehicle, said vision system generates an alert to a driver of the vehicle; and

wherein said vision system is operable to use (i) a first algorithm to rapidly find an object present in the field of view of said camera, (ii) a second algorithm to increase the accuracy while checking the search area and choosing how and where to search to enhance rapid determination of a location of the object, and (iii) a third algorithm that looks at a larger area for faster moving objects.

12. The vision system of claim 11, wherein said matrix array comprises at least 1200 columns of photosensing elements and at least 800 rows of photosensing elements, and wherein said determination of the presence of the pedestrian uses foreground segmentation utilizing a Gaussian mixture model.

13. The vision system of claim 11, wherein said second algorithm selects where to search, so with just a few search points said vision system can determine the location of the object.

14. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera, when disposed at the vehicle, is at a known height relative to a ground surface at which the vehicle is located;

wherein said camera comprise a two dimensional matrix array of photosensing elements, and wherein said matrix array of photosensing elements comprises at least 800 rows of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said image processor, using a kinematics model of the vehicle, processes two or more frames of captured image data to determine a path of travel of the vehicle;

wherein said system uses ground points segmentation and the kinematics model of the vehicle to determine distance traveled by the vehicle between two or more frames of captured image data;

wherein said system identifies as ground points image data points that travel the same distance as the vehicle but in a direction opposite to that of the vehicle, and wherein said system identifies other image data points as points of a three dimensional object;

wherein said system monitors the determined points of a three dimensional object to determine their movement, while not monitoring the determined ground points;

wherein, responsive to processing of captured image data by said image processor and monitoring the determined points of a three dimensional object, said image processor determines presence of a pedestrian in the field of view of said camera;

wherein, responsive to processing of captured image data by said image processor, said vision system determines a distal portion of the determined pedestrian relative to the vehicle and determines a ground location at which the determined distal portion of the determined pedestrian is located;

wherein, responsive to the determined ground location of the determined distal portion of the determined pedestrian and responsive to the known height of said camera relative to the ground surface, said vision system determines a distance to the determined pedestrian;

wherein said vision system is operable to determine distance to the pedestrian using two dimensional information; and

wherein, at least in part responsive to the determined pedestrian being within a threshold distance to the vehicle, and at least in part responsive to the determined pedestrian being in the determined path of the vehicle, said vision system generates an alert to a driver of the vehicle.

15. The vision system of claim 14, wherein said image processor is operable to detect and track an object over multiple frames of captured image data by determining the object using a block determination, and wherein said vision system is operable to use (i) a first algorithm to rapidly find the object, (ii) a second algorithm to increase the accuracy while checking the search area and choosing how and where to search to enhance rapid determination of a location of the object, and (iii) a third algorithm that looks at a larger area for faster moving objects.