Camera auto-calibration by horizon estimation

• US 8,259,174 B2
• Filed: 02/06/2009
• Issued: 09/04/2012
• Est. Priority Date: 02/06/2009
• Status: Expired due to Fees

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Naik, Sanjeev M.
• Primary Examiner(s): Shaw, Peling
• Assistant Examiner(s): Cloud, Joiya M
• Application Number:
  US12/367,305
• Publication Number:
  US 20100201814A1
• Time in Patent Office: 1,306 Days
• Field of Search: 348/148
• US Class Current: 348/148
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20068 Projection on vertical or horizontal image axis
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis

Abstract

A method for estimating the horizon in an image of a camera to provide camera auto-calibration in the pitch direction. The method includes taking an image of a scene and generating a texture map of the image using horizontal edge detection analysis to locate horizontal edges in the scene. The method also includes providing a motion map by providing image differencing between subsequent images to find areas in the scene with no motion while the vehicle is moving. The texture map and the motion map are combined to identify the areas in the image that do not move and contain horizontal edges. A horizontal projection is generated from the combined map by counting white dots in the map in the horizontal direction. The horizontal projection is smoothed to eliminate noise and the location of the horizon is estimated in the image by identifying the maximum peak in the horizontal projection.

Claims

1. A method for estimating a horizon on an image from a camera, said method comprising:

providing first and second images from the camera;

providing a texture map from one of the images that locates horizontal edges in the image;

providing a motion map from a difference between the first and second images that locates areas in the images where no motion is occurring;

combining the texture map and the motion map to identify locations in the combined map that include a horizontal edge and do not move;

providing a horizontal projection that includes a count of identified locations in the horizontal direction in the combined map that included horizontal edges and do not move; and

identifying a maximum peak in the horizontal projection that identifies the estimated horizon in the images.

2. The method according to claim 1 further providing a smoothing of the horizontal projection to reduce noise in the horizontal projection.

3. The method according to claim 1 wherein providing a texture mg includes using horizontal edge detection analysis that includes using Sobel filtering.

4. The method according to claim 1 wherein the edges in the texture map, the locations in the combined map and the locations in the horizontal projection are dots.

5. The method according to claim 2 wherein smoothing the horizontal projection includes using a Gaussian smoothing filter.

6. The method according to claim 1 wherein the horizontal projection is a horizontal histogram.

7. The method according to claim 1 wherein the camera is a vehicle camera.

8. The method according to claim 1 further comprising using the horizon estimation to update the camera'"'"'s pitch angle.

9. A method for estimating a horizon in an image from a camera, said method comprising:

providing first and second images from the camera;

providing a texture map from one of the images that locates horizontal edges in the image as dots where providing the texture map includes using a horizontal edge detection analysis to locate the horizontal edges;

providing a motion map from a difference between the first and second images that locates areas in the images as dots where no motion is occurring;

combining the texture map and the motion map to identify locations in the combined map as dots that include a horizontal edge and do not move;

providing a horizontal projection that includes a count in the horizontal direction of the white dots in the combined map that include horizontal edges and do not move in the combined map; and

identifying a maximum peak in the horizontal projection that identifies the estimated horizon in the images.

10. The method according to claim 9 wherein providing horizontal edge detection analysis includes using Sobel filtering.

11. The method according to claim 9 further comprising smoothing the horizontal projection to reduce noise in the horizontal projection.

12. The method according to claim 9 wherein the horizontal projection is a horizontal histogram.

13. The method according to claim 9 wherein the camera is a vehicle camera.

14. The method according to claim 9 further comprising using the horizon estimation to update the camera'"'"'s pitch angle.

15. A system for estimating a horizon in an image from a camera, said method comprising:

means for providing first and second images from the camera;

means for providing a texture map from one of the images that locates horizontal edges in the image as dots;

means for providing a motion map from a difference between the first and second images that locates areas in the images as dots where no motion is occurring;

means for combining the texture map and the motion map to identify locations in the combined map as dots that include a horizontal edge and do not move;

means for providing a horizontal projection that includes a count of the dots in the horizontal direction in the combined map that includes a horizontal edge and do not move; and

means for identifying a maximum peak in the horizontal projection that identifies the estimated horizon in the images.

16. The system according to claim 15 wherein the means for providing a texture map uses Sobel filtering.

17. The system according to claim 15 further comprising means for smoothing the horizontal projection to reduce noise in the horizontal projection.

18. The system according to claim 15 wherein the horizontal projection is a horizontal histogram.

19. The system according to claim 15 wherein the camera is a vehicle camera.

20. The system according to claim 15 further comprising means for using the horizon estimation to update the camera'"'"'s pitch angle.