SELF CALIBRATION OF EXTRINSIC CAMERA PARAMETERS FOR A VEHICLE CAMERA

• US 20090290032A1
• Filed: 05/22/2008
• Published: 11/26/2009
• Est. Priority Date: 05/22/2008
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations Incorporated (Motors Liquidation Co. GUC Trust)
• Inventors: Zhang, Wende, Wolski, Mark J.
• Granted Patent: US 8,373,763 B2
• Time in Patent Office: Days
• US Class Current: 348/211.9
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A system and method for calibrating a camera on a vehicle as the vehicle is being driven. The method includes identifying at least two feature points in at least two camera images from a vehicle that has moved between taking the images. The method then determines a camera translation direction between two camera positions. Following this, the method determines a ground plane in camera coordinates based on the corresponding feature points from the images and the camera translation direction. The method then determines a height of the camera above the ground and a rotation of the camera in vehicle coordinates.

Claims

1. A method for calibrating a camera that is moving relative to the ground, said method comprising:

identifying at least two feature points in at least two camera images;

determining a camera translation and rotation between camera coordinates for different positions of the camera at a first position of the camera;

defining a ground plane in camera coordinates;

determining a height of the camera above the ground; and

identifying a rotation of the camera in vehicle coordinates.

2. The method according to claim 1 wherein determining a camera translation and rotation includes identifying a location point of the camera in the at least two images relative to a normalized image plane, identifying a three-dimensional space using the camera location point for the two images of the camera and one of the feature points and determining a camera translation direction between the two images using the three-dimensional space.

3. The method according to claim 2 wherein the translation direction is calculated where the distance between the locations where the two images are taken is known.

4. The method according to claim 2 wherein the translation direction is estimated where the distance between the locations where the two images are taken is unknown.

5. The method according to claim 2 wherein defining a ground plane includes defining a ground plane in camera coordinates based on the at least two feature points and the camera translation direction.

6. The method according to claim 5 wherein defining the ground plane includes using the translation direction, the two points on the ground to determine a mean for the points on the ground, a dominant variation direction of the ground points and a ground norm direction.

7. The method according to claim 1 wherein determining the camera height includes calculating an inner product between the mean of the featured points and the norm of the ground plane where the height of the camera is used to eliminate feature points that are not on the ground.

8. The method according to claim 1 wherein determining the camera rotation in vehicle coordinates includes determining the camera location in vehicle coordinates using a pitch down of the camera and a yaw of the camera.

9. The method according to claim 1 wherein identifying feature points includes identifying feature points when the vehicle is traveling in a straight line.

10. The method according to claim 1 wherein providing at least two camera images includes providing more than two camera images.

11. The method according to claim 1 wherein defining the ground plane includes merging separate ground planes for each feature point detected on the ground.

12. The method according to claim 1 wherein identifying at least two feature points includes identifying two features points for two camera images at one time frame and two feature points for two camera images at another time frame.

13. The method according to claim 12 wherein the camera is moving in a straight line during the one time frame, the camera is turning between the two time frames and the camera is moving in a straight line during the another time frame.

14. The method according to claim 1 wherein the camera is on a vehicle.

15. A method for calibrating a camera on a vehicle that is moving relative to the ground, said method comprising:

identifying at least two feature points in at least two camera images;

determining a camera translation and rotation between camera coordinates for different positions of the camera at a first position of the camera, wherein determining a camera translation and rotation includes identifying a location point of the camera in the at least two images relative to a normalized image plane, identifying a three-dimensional space using the camera location point for the two images of the camera and one of the feature points and determining a camera translation direction between the two images using the three-dimensional space;

defining a ground plane in camera coordinates based on the at least two feature points and the camera translation direction, wherein defining the ground plane includes using the translation direction, the two points on the ground to determine a mean for the points on the ground, a dominant variation direction of the ground points and a ground norm direction;

eliminating feature points that are not on the ground plane;

determining a height of the camera above the ground by calculating an inner product between the mean of the featured points and the norm of the ground plane where the height of the camera is used to eliminate feature points that are not on the ground; and

identifying a rotation of the camera in vehicle coordinates using a pitch down of the camera and a yaw of the camera.

16. The method according to claim 15 wherein the translation direction is calculated where the distance between the locations where the two images are taken is known.

17. The method according to claim 15 wherein the translation direction is estimated where the distance between the locations where the two images are taken is unknown.

18. The method according to claim 15 wherein defining the ground plane includes merging separate ground planes for each feature point detected on the ground.

19. The method according to claim 15 wherein identifying at least two feature points includes identifying two features points for two camera images at one time frame and two feature points for two camera images at another time frame, wherein the camera is moving in a straight line during the one time frame, the camera is turning between the two time frames and the camera is moving in a straight line during the another time frame.

20. A method for calibrating a camera on a vehicle that is moving relative to the ground, said method comprising:

identifying at least two feature points in at least two camera images;

determining a camera translation and rotation between camera coordinates for different positions of the camera at a first position of the camera, wherein determining a camera translation and rotation includes identifying a location point of the camera in the at least two images relative to a normalized image plane, identifying a three-dimensional space using the camera location point for the two images of the camera and one of the feature points and determining a camera translation direction between the two images using the three-dimensional space;

defining a ground plane in camera coordinates based on the at least two feature points and the camera translation direction;

determining a height of the camera above the ground by calculating an inner product between the mean of the featured points and the norm of the ground plane; and

identifying a rotation of the camera in vehicle coordinates.