SURROUND-VIEW CAMERA SYSTEM (VPM) AND VEHICLE DYNAMIC

• US 20150332446A1
• Filed: 05/12/2015
• Published: 11/19/2015
• Est. Priority Date: 05/16/2014
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: WANG, JINSONG, ZHANG, WENDE, LITKOUHI, BAKHTIAR B.
• Granted Patent: US 9,981,605 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2011/0089 around three axes, i.e. universally mounted
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A system and method for correcting the calibration of a plurality of cameras on a mobile platform such as in a surround-view camera system on a vehicle based on changes in vehicle dynamics. The method includes reading measurement values from one or more sensors on the vehicle that identify a change in vehicle dynamics and defining the plurality of cameras and a vehicle body as a single reference coordinate system. The method also includes identifying the measured values as a rotation matrix and a translation vector in the coordinate system, and integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system to provide the calibration correction of the cameras.

Claims

1. A method for correcting a calibration of a plurality of cameras in a surround-view camera system on a vehicle, said method comprising:

reading measurement values from one or more sensors on the vehicle that identify a change in vehicle dynamics;

defining the plurality of cameras and a vehicle body as a single reference coordinate system;

identifying the measured values as a rotation matrix and a translation vector in the reference coordinate system; and

integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system to provide the correction of the camera calibration.

2. The method according to claim 1 wherein integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system includes defining a stationary orientation of the surround-view camera system by:

3. The method according to claim 2 wherein integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system to provide the camera calibration includes correcting the camera calibration using the equation:

(R_cam,T_cam)=f((R_veh,T_veh),(R_cam,T_cam)_stny),

where stny is a designation for stationary, the vehicle dynamics of pitch, roll and height variation are defined by rotation matrix R_veh and translation vector T_veh, R_cam is the rotation matrix in camera coordinates, and T_cam is the translation vector in camera coordinates.

4. The method according to claim 1 wherein reading measurement values from one or more sensors includes reading measurement values from one or more sensors that provide vehicle or camera and vehicle body pitch, roll and height variation dynamics.

5. The method according to claim 1 wherein the surround-view camera system includes four cameras, wherein a first camera is positioned at a front of the vehicle, a second camera is positioned at a back of the vehicle, a third camera is positioned at a left side of the vehicle and a fourth camera is positioned at a right side of the vehicle.

6. The method according to claim 5 wherein images from the first camera overlap with images from the third camera, images from the first camera overlap with images from the fourth camera, images from the second camera overlap with images from the third camera, and images from the second camera overlap with images from the fourth camera.

7. A method for correcting a calibration of a plurality of cameras on a mobile platform, said method comprising:

reading measurement values from one or more sensors on the mobile platform that identify a change in platform dynamics;

defining the plurality of cameras and a platform body as a single reference coordinate system;

identifying the measured values as a rotation matrix and a translation vector in the reference coordinate system; and

integrating the rotation matrix and the translation vector into a relationship between a platform coordinate system and a camera coordinate system to provide correction of the camera calibration.

8. The method according to claim 7 wherein integrating the rotation matrix and the translation vector into a relationship between a platform coordinate system and a camera coordinate system includes defining a stationary orientation of the cameras by:

9. The method according to claim 8 wherein integrating the rotation matrix and the translation vector into a relationship between a platform coordinate system and a camera coordinate system to provide the camera calibration includes correcting the camera calibration using the equation:

(R_cam,T_cam)=f((R_veh,T_veh),(R_cam,T_cam)_stny),

where stny is a designation for stationary, the platform dynamics of pitch, roll and heght variation are defined by rotation matrix R_veh and translation vector T_veh, R_cam is the rotation matrix in camera coordinates, and T_cam is the translationa vector in camera coordinates.

10. The method according to claim 7 wherein reading measurement values from one or more sensors includes reading measurement values from one or more sensors that provide mobile platform or camera and mobile platform pitch, roll and height variation dynamics.

11. The method according to claim 7 wherein the plurality of cameras are part of a surround-view camera system.

12. The method according to claim 11 wherein the surround-view camera system includes four cameras, wherein a first camera is positioned at a front of the platform, a second camera is positioned at a back of the platform, a third camera is positioned at a left side of the platform and a fourth camera is positioned at a right side of the platform.

13. The method according to claim 12 wherein images from the first camera overlap with images from the third camera, images from the first camera overlap with images from the fourth camera, images from the second camera overlap with images from the third camera, and images from the second camera overlap with images from the fourth camera.

14. A calibration system for correcting a calibration of a plurality of cameras in a surround-view camera system on a vehicle, said system comprising:

means for reading measurement values from one or more sensors on the vehicle that identify a change in vehicle dynamics;

means for defining the plurality of cameras and a vehicle body as a single reference coordinate system;

means for identifying the measured values as a rotation matrix and a translation vector in the reference coordinate system; and

means for integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system to provide the correction of the camera calibration.

15. The calibration system according to claim 14 wherein the means for integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system defines a stationary orientation of the surround-view camera system by:

16. The calibration system according to claim 15 wherein the means for integrating the rotation matrix and the translation vector into a relationship between a vehicle coordinate system and a camera coordinate system to provide the camera calibration corrects the camera calibration uses the equation:

(R_cam,T_cam)=f((R_veh,T_veh),(R_cam,T_cam)_stny),

where stny is a designation for stationary, the vehicle dynamics of pitch, roll and height variation are defined by rotation matrix R_veh and translation vector T_veh, R_cam is the rotation matrix in camera coordinates, and T_cam is the translation vector in camera coordinates.

17. The calibration system according to claim 14 wherein the means for reading measurement values from one or more sensors reads measurement values from one or more sensors that provide vehicle or camera and vehicle body pitch, roll and height variation dynamics.

18. The calibration system according to claim 14 wherein the surround-view camera system includes four cameras, wherein a first camera is positioned at a front of the vehicle, a second camera is positioned at a back of the vehicle, a third camera is positioned at a left side of the vehicle and a fourth camera is positioned at a right side of the vehicle.

19. The calibration system according to claim 18 wherein images from the first camera overlap with images from the third camera, images from the first camera overlap with images from the fourth camera, images from the second camera overlap with images from the third camera, and images from the second camera overlap with images from the fourth camera.