Method and apparatus for distortion correction and image enhancing of a vehicle rear viewing system

• US 20090021609A1
• Filed: 07/16/2007
• Published: 01/22/2009
• Est. Priority Date: 07/16/2007
• Status: Active Grant

• Current Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Luo, Yun, John, Chris St., Prainito, John
• Granted Patent: US 8,233,045 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

An image enhancing system (12) for a vehicle (10) comprises a display unit (14) for displaying modified images (38) and an imaging device (20) for receiving captured images (36) enhanced by the image enhancing system (12). The system (12) further includes an image enhancing module (26) in communication with the display unit (14) and the imaging device (20) such that pixels located in the captured images (36) are enhanced by repositioning the pixels from a first position to a second position via a transfer operation.

Claims

1. An image enhancing system for a vehicle comprising:

a display unit for displaying modified images enhanced by the image enhancing system;

an imaging device for receiving captured images enhanced by said image enhancing system;

an image enhancing module in communication with said display unit and said imaging device such that pixels located in said captured images are enhanced by repositioning said pixels from a first position to a second position through a transfer operation to form said modified images.

2. The image enhancing system of claim 1 wherein said transfer operation includes at least one coefficient that is adjusted by analyzing said captured images.

3. The image enhancing system of claim 2 wherein said analyzing the captured images includes a calibration procedure where coefficients in the transfer operation are defined by locating substantially horizontal and/or substantially vertical lines in said captured images.

4. The image enhancing system of claim 3 wherein said substantially vertical and horizontal lines include lines forming a center of distortion about said captured image.

5. The image enhancing system of claim 1 wherein said first position is expressed by (X_a, Y_a) and said second position is expressed by (X_c, Y_c) and said transfer operation is performed in accordance with the formulas:

X_c=s*cos φ

*X_a*(1+k₁ρ

²+k₂ρ

⁴) and

Y_c=(s*sin φ

*X_a+cos φ

*Y_a)(1+k₁ρ

²+k₂ρ

⁴).

6. The image enhancing system of claim 2 wherein said first position is expressed by (X_a, Y_a) and said second position is expressed by (X_c, Y_c) and said transfer operation is performed in accordance with the formulas:

X_c=s*cos φ

*X_a*(1+k₁ρ

²+k₂ρ

⁴) and

Y_c=(s*sin φ

*X_a+cos φ

*Y_a)(1+k₁ρ

²+k₂ρ

⁴).

7. The image enhancing system of claim 1 wherein said transfer operation further comprises an inverse perspective mapping transform.

8. The image enhancing system of claim 2 wherein said transfer operation further comprises an inverse perspective mapping transform.

9. The image enhancing system of claim 6 wherein said transfer operation further comprises an inverse perspective mapping transform.

10. The image enhancing system of claim 1 wherein said transfer operation further comprises remapping said pixels from said first position to said second position.

11. The image enhancing system of claim 1 wherein said wherein said second position is expressed by (X_p, Y_p) and said transfer operation is performed in accordance with the formulas:

x_p=ƒ

X/Z and y_p=ƒ

Y/Z.

12. The image enhancing system of claim 9 wherein said second position is expressed by (X_p, Y_p) and said transfer operation is performed in accordance with the formulas:

x_p=ƒ

X/Z and y_p=ƒ

Y/Z.

13. The image enhancing system of claim 1 wherein said captured images are real-time continuous images received by said imaging device.

14. The image enhancing system of claim 1 wherein said imaging device includes at least one CMOS or CCD sensor based camera.

15. The image enhancing system of claim 1 wherein said image enhancing module generates said modified images comprising a plan view and a side view relative to said vehicle on said display unit such that the distance from a rear location of the vehicle to an object located at any distance the imaging device field-of-view is linearly proportional in said modified images.

16. The image enhancing system of claim 1 wherein said transfer operation further comprises remapping ground and non-ground images to form said modified images.

17. The image enhancing system of claim 1 wherein said transfer operation further comprises analyzing vehicle steering angle information for calculating a backup path illustrated in said modified images.

18. A method of enhancing an image in a vehicle imaging system comprising the steps of:

receiving captured images by at least one imaging device located on the vehicle;

communicating said captured images to an image enhancing module;

enhancing said captured images such that pixels located in said captured images are repositioned from a first position to a second position by a transfer operation to form modified images; and

communicating said modified images from said enhancing module to a display unit located in said vehicle.

19. The method of claim 18 further comprising the step of adjusting at least one coefficient in the transfer operation by analyzing said captured images.

20. The method of claim 19 wherein the step of analyzing said captured images further comprises a calibration step where coefficients in the transfer operation are defined by locating substantially horizontal and/or substantially vertical lines in said captured images.

21. The method of claim 18 wherein the step of performing a transfer operation to form modified images is further achieved by executing an inverse perspective transform.

22. The method of claim 19 wherein the step of performing a transfer operation to form modified images is further achieved by executing an inverse perspective transform.

23. An image enhancing system for improving images received by an imaging device located on a vehicle, the image enhancing system comprising:

at least one camera located on said vehicle, the camera having a pixel array for receiving real-time captured images within the camera field of view;

a computing unit having an image enhancing module in communication with said camera for improving said captured images by repositioning pixels in said captured images from a first position to a second position in accordance with a transfer operation performed by the image enhancing module such that the repositioning of the pixels form enhanced images; and

a display unit in communication with said computing unit and located within said vehicle for displaying said enhanced images.

24. The image enhancing system of claim 23 wherein at least one coefficient in said transfer operation is adjusted by analyzing said captured images.

25. The image enhancing system of claim 24 wherein said first position is expressed by (X_a, Y_a) and said second position is expressed by (X_c, Y_c) and said transfer operation is performed in accordance with the formulas:

X_c=s*cos φ

*X_a*(1+k₁ρ

²+k₂ρ

⁴)and

Y_c=(s*sin φ

*X_a+cos φ

*Y_a)(1+k₁ρ

²+k₂ρ

⁴).

26. The image enhancing system of claim 23 wherein said transfer operation further comprises an inverse perspective mapping transform.

27. The image enhancing system of claim 23 wherein said second position is expressed by (X_p, Y_p) and said transfer operation is performed in accordance with the formulas:

x_p=ƒ

X/Z and y_p=ƒ

Y/Z.

28. A rearview image enhancing system for a vehicle comprising:

a display unit for displaying modified images enhanced by the image enhancing system;

an imaging device for receiving captured images enhanced by said image enhancing system;

an image enhancing module in communication with said display unit and said imaging device such that pixels located in said captured images are clustered and segmented to form at least one area of interest by referencing the pixels from a ground plane in the captured images to form said modified images.

29. The rearview image enhancing system of claim 28 wherein said clustered and segmented pixels used to form at least one area of interest are formed by evaluating the pixels in the captured image for similar image cues.

30. The rearview image enhancing system of claim 29 wherein said similar image cues include color, texture, and/or pixel intensity.

31. The rearview image enhancing system of claim 28 wherein said clustered and segmented pixels are used to separate ground and non-ground regions in obtaining the area of interest.

32. The rearview image enhancing system of claim 31 wherein said area of interest form pop-up objects in the modified images displayed on the display unit.

33. The rearview image enhancing system of claim 32 wherein said pop-up objects are located in the modified images relative to the ground plane.

34. The rearview image enhancing system of claim 32 further comprising a computing unit capable of predicting a collision with said pop-up objects based on a steering angle of the vehicle and the location of the pop-up objects in the modified image.

35. The rearview image enhancing system of claim 34 wherein said modified image includes a scale for indicating the distance between said pop-up objects and the vehicle.