Vehicle vision system with road line sensing algorithm and lane departure warning

• US 10,449,899 B2
• Filed: 05/05/2016
• Issued: 10/22/2019
• Est. Priority Date: 05/08/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil, Corral-Soto, Eduardo R.
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US15/147,301
• Publication Number:
  US 20160325682A1
• Time in Patent Office: 1,265 Days
• Field of Search: 348148
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A vehicle vision system includes a plurality of cameras disposed at a vehicle and having respective fields of view exterior of the vehicle. An image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird'"'"'s-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of the cameras. A display displays the bird'"'"'s-eye view image for viewing by a driver of the vehicle. The image processor processes synthesized captured image data using a lane marker sensing algorithm that utilizes recursive temporal stabilization to determine lane markings in the field of view of at least the rear camera. The system determines when the vehicle is departing from its being-traveled lane and, responsive to determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle.

Claims

1. A surround view vision system for a vehicle, said surround view vision system comprising:

a plurality of cameras disposed at a vehicle each having a respective field of view exterior of the vehicle that encompasses a respective ground region adjacent to the vehicle, said plurality of cameras comprising at least (i) a front camera disposed at a front portion of the vehicle and having at least a forward and downward field of view encompassing a ground region forward of the vehicle and (ii) a rear camera disposed at a rear portion of the vehicle and having at least a rearward and downward field of view encompassing a ground region rearward of the vehicle;

wherein each of said cameras comprises a lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by at least some of said cameras;

wherein said image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird'"'"'s-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of said cameras;

a display that displays the bird'"'"'s-eye view image for viewing by a driver of the vehicle while operating the vehicle;

wherein said image processor processes synthesized captured image data using a lane marker sensing algorithm to determine lane markings in the field of view of at least said rear camera;

wherein said image processor determines lane markers by (a) extracting low level image features from synthesized captured image data and (b) extracting line information from synthesized captured image data via pixel classification and line fitting;

wherein said lane marker sensing algorithm utilizes recursive temporal stabilization;

wherein said lane marker sensing algorithm predicts lane markers when they are no longer within the field of view of said rear camera;

wherein prediction of lane markers when they are no longer within the field of view of said rear camera is based at least in part on said recursive temporal stabilization; and

wherein said system, responsive to processing of synthesized captured image data using said lane marker sensing algorithm, determines when the vehicle is departing from its being-traveled lane and, responsive to a determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle.

2. The surround view vision system of claim 1, wherein said image processor processes synthesized captured image data and determines lane markings via classification and image processing techniques.

3. The surround view vision system of claim 1, wherein a principal axis of the field of view of said rear camera is downward at an angle of about 45 degrees.

4. The surround view vision system of claim 1, wherein said lens comprises a fish-eye lens.

5. The surround view vision system of claim 1, wherein said image processor determines lane markers by stabilizing line information extracted from synthesized captured image data.

6. The surround view vision system of claim 1, wherein said lane marker sensing algorithm utilizes said recursive temporal stabilization to reduce jitter of detected lines in synthesized captured image data.

7. The surround view vision system of claim 1, wherein said lane marker sensing algorithm utilizes geometric road constraints to enhance determination of lane markers.

8. The surround view vision system of claim 1, wherein said lane marker sensing algorithm utilizes synthesized captured image data to determine low-level features including image gradients and pixel-level statistics, and wherein said lane marker sensing algorithm utilizes a probabilistic pixel classification stage to classify data captured by each pixel of said rear camera as being part of a lane marker or not part of a lane marker.

9. The surround view vision system of claim 8, wherein said lane marker sensing algorithm applies spatial line fitting to pixels that are determined to be part of a lane marker to determine lines in synthesized captured image data.

10. The surround view vision system of claim 9, wherein said lane marker sensing algorithm applies road geometric constraints including lane marker width and gap size between lane markers.

11. The surround view vision system of claim 1, wherein said alert comprises a visible alert generated in the displayed bird'"'"'s-eye view image.

12. The surround view vision system of claim 11, wherein said visible alert comprises a linear overlay that shows deviation of the vehicle'"'"'s path of travel relative to the lane markers displayed in the displayed bird'"'"'s-eye view image.

13. A surround view vision system for a vehicle, said surround view vision system comprising:

a plurality of cameras disposed at a vehicle each having a respective field of view exterior of the vehicle that encompasses a respective ground region adjacent to the vehicle, said plurality of cameras comprising at least (i) a rear camera disposed at a rear portion of the vehicle and having at least a rearward and downward field of view encompassing a ground region rearward of the vehicle, (ii) a driver side camera disposed at a driver-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle and (iii) a passenger side camera disposed at a passenger-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle;

wherein each of said cameras comprises a fish-eye lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by at least some of said cameras;

wherein said image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird'"'"'s-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of said cameras;

a display that displays the bird'"'"'s-eye view image for viewing by a driver of the vehicle while operating the vehicle;

wherein said image processor processes synthesized captured image data using a lane marker sensing algorithm to determine lane markings in the field of view of at least said rear camera;

wherein said image processor determines lane markers by (a) extracting low level image features from synthesized captured image data and (b) extracting line information from synthesized captured image data via pixel classification and line fitting;

wherein said lane marker sensing algorithm utilizes recursive temporal stabilization;

wherein said lane marker sensing algorithm predicts lane markers when they are no longer within the field of view of said rear camera;

wherein prediction of lane markers when they are no longer within the field of view of said rear camera is based at least in part on said recursive temporal stabilization;

wherein said system, responsive to processing of synthesized captured image data using said lane marker sensing algorithm, determines when the vehicle is departing from its being-traveled lane and, responsive to a determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle; and

wherein said alert comprises a visible alert generated in the displayed bird'"'"'s-eye view image, and wherein said visible alert comprises a linear overlay that shows deviation of the vehicle'"'"'s path of travel relative to the lane markers displayed in the displayed bird'"'"'s-eye view image.

14. The surround view vision system of claim 13, wherein said lane marker sensing algorithm utilizes said recursive temporal stabilization to reduce jitter of detected lines in synthesized captured image data.

15. A surround view vision system for a vehicle, said surround view vision system comprising:

a plurality of cameras disposed at a vehicle each having a respective field of view exterior of the vehicle that encompasses a respective ground region adjacent to the vehicle, said plurality of cameras comprising at least (i) a rear camera disposed at a rear portion of the vehicle and having at least a rearward and downward field of view encompassing a ground region rearward of the vehicle, (ii) a driver side camera disposed at a driver-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle, (iii) a passenger side camera disposed at a passenger-side portion of the vehicle and having at least a sideward and downward field of view encompassing a ground region sideward of the vehicle, and (iv) a front camera disposed at a front portion of the vehicle and having at least a forward and downward field of view encompassing a ground region forward of the vehicle;

wherein each of said cameras comprises a fish-eye lens and a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by at least some of said cameras;

wherein said image processor is operable, responsive to image processing of captured image data, to synthesize captured image data to generate a reduced distortion bird'"'"'s-eye view image of a region exterior and near the vehicle and encompassed by the fields of view of said cameras;

a display that displays the bird'"'"'s-eye view image for viewing by a driver of the vehicle while operating the vehicle;

wherein said image processor processes synthesized captured image data using a lane marker sensing algorithm to determine lane markings in the field of view of at least said rear camera;

wherein said image processor determines lane markers by (a) extracting low level image features from synthesized captured image data and (b) extracting line information from synthesized captured image data via pixel classification and line fitting;

wherein said lane marker sensing algorithm utilizes recursive temporal stabilization;

wherein said lane marker sensing algorithm utilizes said recursive temporal stabilization to reduce jitter of detected lines in synthesized captured image data;

wherein said lane marker sensing algorithm predicts lane markers when they are no longer within the field of view of said rear camera;

wherein prediction of lane markers when they are no longer within the field of view of said rear camera is based at least in part on said recursive temporal stabilization;

wherein said system, responsive to processing of synthesized captured image data using said lane marker sensing algorithm, determines when the vehicle is departing from its being-traveled lane and, responsive to a determination that the vehicle is departing from its being-traveled lane, generates an alert to the driver of the vehicle;

wherein said alert comprises a visible alert generated in the displayed bird'"'"'s-eye view image, and wherein said visible alert comprises a linear overlay that shows deviation of the vehicle'"'"'s path of travel relative to the lane markers displayed in the displayed bird'"'"'s-eye view image; and

wherein said alert further comprises an audible alert in the vehicle.

16. The surround view vision system of claim 15, wherein said lane marker sensing algorithm utilizes geometric road constraints to enhance determination of lane markers, and wherein the utilized road geometric constraints include lane marker width and gap distance between markers.