Image processing device and image processing method for camera calibration

• US 9,412,168 B2
• Filed: 12/14/2011
• Issued: 08/09/2016
• Est. Priority Date: 06/18/2009
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Shimizu, Seiya
• Primary Examiner(s): TORRENTE, RICHARD T
• Application Number:
  US13/325,703
• Publication Number:
  US 20120081512A1
• Time in Patent Office: 1,700 Days
• Field of Search: 348/36
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

In a calibration device, an image-integration averaging processing unit integrates/averages an image so as to generate an integrated and averaged image, and an edge extracting unit extracts the edges from the integrated and averaged image. A vanishing-point extracting unit calculates vanishing lines by using the edges and identifies the coordinates of the intersection point of the vanishing lines as the coordinates of the vanishing point. An error determining unit compares the coordinates of the current vanishing point with the coordinates of the previous vanishing point so as to determine whether a parameter of the camera needs correcting. In accordance with the result of the determination, a correction processing unit corrects a parameter of the camera.

Claims

1. An image processing device comprising:

an integrating unit configured to acquire, from a camera installed on a movable body, images that are captured by the camera while the movable body is moving and integrates the acquired images so as to generate an integrated image;

a vanishing-point identifying unit configured to extract brightness edges from the integrated image and identifies an intersection point of lines determined by using the brightness edges as a vanishing point;

a determining unit configured to determine whether a parameter of the camera has an appropriate value by using coordinates of the vanishing point identified and reference coordinates, the reference coordinates being coordinates of a reference vanishing point stored; and

a correcting unit configured to correct the parameter of the camera in accordance with a difference between the coordinates of the vanishing point identified and the reference coordinates.

2. The image processing device according to claim 1, wherein the integrating unit is configured to acquire a moving speed of the movable body from a speed sensor and, while the acquired moving speed is equal to or greater than a predetermined speed, integrates images obtained from the camera so as to generate the integrated image.

3. The image processing device according to claim 2, wherein the integrating unit is configured to acquire a steering angle of the movable body from a steering angle sensor and, while the acquired steering angle is less than a predetermined steering angle, integrates images obtained from the camera so as to generate the integrated image.

4. The image processing device according to claim 1, wherein the integrating unit is configured to acquire the integrated image by adding and averaging the brightness value of the acquired images for each pixel.

5. The image processing device according to claim 1, wherein the parameter of the camera includes a pan angle or a depression angle.

6. An image processing method performed by an image processing device, the image processing method comprising:

acquiring, from a camera installed on a movable body, images that are captured by the camera while the movable body is moving and integrating the acquired images so as to generate an integrated image;

extracting brightness edges from the integrated image and identifying an intersection point of lines determined by using the brightness edges as a vanishing point;

determining whether a parameter of the camera has an appropriate value by using the coordinates of the vanishing point identified and reference coordinates, the reference coordinates being coordinates of a reference vanishing point stored; and

correcting the parameter of the camera in accordance with a difference between the coordinates of the vanishing point identified and the reference coordinates.

7. The image processing method according to claim 6, the integrating includes acquiring a moving speed of the movable body from a speed sensor and, while the acquired moving speed is equal to or greater than a predetermined speed, integrating images obtained from the camera so as to generate the integrated image.

8. The image processing method according to claim 7, the integrating includes acquiring a steering angle of the movable body from a steering angle sensor and, while the acquired steering angle is less than a predetermined steering angle, integrating images obtained from the camera so as to generate the integrated image.

9. An image processing device comprising:

a processor; and

a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute;

acquiring, from a camera installed on a movable body, images that are captured by the camera while the movable body is moving and integrating the acquired images so as to generate an integrated image;

extracting brightness edges from the integrated image and identifying an intersection point of lines determined by using the brightness edges as a vanishing point;

determining whether a parameter of the camera has an appropriate value by using the coordinates of the vanishing point identified and reference coordinates, the reference coordinates being coordinates of a reference vanishing point stored; and

correcting the parameter of the camera in accordance with a difference between the coordinates of the vanishing point identified and the reference coordinates.