Camera calibration apparatus and method, image processing apparatus and method, program providing medium, and camera

• US 6,816,187 B1
• Filed: 06/02/2000
• Issued: 11/09/2004
• Est. Priority Date: 06/08/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Iwai, Yoshiaki, Yoshigahara, Takayuki
• Primary Examiner(s): Christensen, Andrew
• Assistant Examiner(s): Long, Heather R.
• Application Number:
  US09/587,246
• Time in Patent Office: 1,621 Days
• Field of Search: 348/135, 348/136, 348/137, 348/42, 348/43, 348/241, 348/627, 348/333.1, 348/187, 348/50, 348/189, 348/180, 348/181, 348/182, 382/154, 382/266, 382/268, 382/275, 382/289, 358/504, 358/406
• US Class Current: 348/187
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

The present invention relates to a high-performance camera calibration apparatus and method capable of accomplishing stable high-accuracy parameter estimation. A pattern whose geometrical configuration is known in advance is photographed by a camera to generate a picked-up image, with the generated picked-up image being temporarily stored as an image input in a frame buffer. In addition, a base image having a pattern univocally corresponding in geometrical configuration definition to the picked-up image is generated according to the CG technology or the like, and is stored in another frame buffer. An image registration is made between the base image and the picked-up image to minimize the luminance error, which enables the parameter estimation.

Claims

1. A camera calibration apparatus for calculating a parameter representative of a characteristic of a camera, comprising:

image inputting means for inputting a picked-up image obtained by taking a photograph of a pattern with a predefined geometrical configuration through the use of a camera;

image holding means for holding a base image comprising said pattern with said predefined geometrical configuration; and

transformation parameter calculating means for calculating a transformation parameter on the basis of the association in pixel between said picked-up image and said base image to minimize luminance error between corresponding pixels of both said picked-up image and base image.

2. A camera calibration apparatus according to claim 1, further comprising image generating means for generating said base image comprising said pattern with said predefined geometrical configuration according to the definition, with said image holding means holding said image generated by said image generating means.

3. A camera calibration apparatus according to claim 1, further comprising image generating means for generating said base image comprising said pattern with said predefined geometrical configuration according to the definition and light projecting means for projecting the generated base image on a generally plain plane, with said image inputting means inputting a picked-up image obtained by photographing the projected image, given by said light projecting means, through the use of said camera.

4. A camera calibration apparatus according to claim 1, wherein said transformation parameter calculating means performs an image conversion of one of said picked-up image and said base image and further associates the converted image with the other image.

5. A camera calibration apparatus according to claim 4, wherein said transformation parameter calculating means derives a projective transformation parameter, performs an image conversion of one of said picked-up image and said base image through the use of the derived projective transformation parameter.

6. A camera calibration apparatus according to claim 5, wherein said transformation parameter calculating means derives a distortion parameter representative of a distortion factor of a picked-up image taking place at image pickup by said camera and performs a projective transformation of said picked-up image from which distortion is removed through the use of the derived distortion parameter, to associate the transformed image with said base image.

7. A camera calibration apparatus according to claim 4, wherein said transformation parameter calculating means derives a distortion parameter representative of a distortion factor of a picked-up image taking place at image pickup by said camera and performs an image conversion of one of said picked-up image and said base image through the use of the derived distortion parameter, and associates the converted image with the other image to minimize a luminance error between corresponding pixels of both said images.

8. A camera calibration apparatus according to claim 1, wherein said transformation parameter calculating means corrects a luminance value of said base image in accordance with a luminance value of said picked-up image.

9. A camera calibration apparatus according to claim 8, wherein said transformation parameter calculating means extracts, from said picked-up image, an area in which luminance values are approximately equal to each other, and obtains an average value of said luminance values in the extracted area to replace a luminance value of the corresponding pixel in said base image with the obtained average value for correction.

10. A camera calibration method of calculating a parameter representative of a characteristic of a camera, comprising:

an image inputting step of inputting a picked-up image obtained by taking a photograph of a pattern with a predefined geometrical configuration through the use of a camera;

an image holding step of holding a base image comprising said pattern with said predefined geometrical configuration; and

a transformation parameter calculating step of calculating a transformation parameter on the basis of the association in pixel between said picked-up image and said base image to minimize luminance error between corresponding pixels of both said picked-up image and base image.

11. A camera calibration method according to claim 10, further comprising an image generating step of generating said base image comprising said pattern with said predefined geometrical configuration according to the definition, with said image holding step holding said image generated by said image generating step.

12. A camera calibration method according to claim 10, further comprising an image generating step of generating said base image comprising said pattern with said predefined geometrical configuration according to the definition and a light projecting step of projecting the generated base image on a generally plain plane, with said image inputting step inputting a picked-up image obtained by photographing the projected image, given by said light projecting step, through the use of said camera.

13. A camera calibration method according to claim 10, wherein said transformation parameter calculating step performs an image conversion of one of said picked-up image and said base image and further associates the converted image with the other image.

14. A camera calibration method according to claim 13, wherein said transformation parameter calculating step derives a projective transformation parameter, performs an image conversion of one of said picked-up image and said base image through the use of the derived projective transformation parameter, and associates the converted image with the other image to minimize a luminance error between corresponding pixels of both said images over the whole of said images.

15. A camera calibration method according to claim 14, wherein said transformation parameter calculating step derives a distortion parameter representative of a distortion factor of a picked-up image taking place at image pickup by said camera and performs a projective transformation of said picked-up image from which distortion is removed through the use of the derived distortion parameter, to associate the transformed image with said base image.

16. A camera calibration method according to claim 13, wherein said transformation parameter calculating means derives a distortion parameter representative of a distortion factor of a picked-up image taking place at image pickup by said camera and performs an image conversion of one of said picked-up image and said base image through the use of the derived distortion parameter, and associates the converted image with the other image to minimize a luminance error between corresponding pixels of both said images.

17. A camera calibration method according to claim 10, wherein said transformation parameter calculating step corrects a luminance value of said base image in accordance with a luminance value of said picked-up image.

18. A camera calibration method according to claim 17, wherein said transformation parameter calculating step extracts, from said picked-up image, an area in which luminance values are approximately equal to each other, and obtains an average value of said luminance values in the extracted area to replace a luminance value of the corresponding pixel in said base image with the obtained average value for correction.

19. An image processing apparatus for processing a plurality of images picked up by a camera, comprising:

image inputting means for inputting a picked-up image obtained by taking a photograph of a pattern with a predefined geometrical configuration through the use of a camera;

image holding means for holding a base image comprising said pattern with said predefined geometrical configuration;

transformation parameter calculating means for calculating a transformation parameter on the basis of the association in pixel between said picked-up image and said base image to minimize luminance error between corresponding pixels of both said picked-up image and base image; and

arithmetic means for obtaining the association in pixel among a plurality of picked-up images obtained by said camera through the use of the calculated transformation parameter.

20. An image processing apparatus according to claim 19, further comprising image generating means for generating said base image comprising said pattern with said predefined geometrical configuration according to the definition, with said image holding means holding said image generated by said image generating means.

21. An image processing apparatus according to claim 19, further comprising image generating means for generating said base image comprising said pattern with said predefined geometrical configuration according to the definition and light projecting means for projecting the generated base image on a generally plain plane, with said image inputting means inputting a picked-up image obtained by photographing the projected image, given by said light projecting means, through the use of a camera.

22. An image processing apparatus according to claim 19, wherein said transformation parameter calculating means performs an image conversion of one of said picked-up image and said base image and further associates the converted image with the other image.

23. An image processing apparatus according to claim 22, wherein said transformation parameter calculating means derives a projective transformation parameter, performs an image conversion of one of said picked-up image and said base image through the use of the derived projective transformation parameter, and associates the converted image with the other image to minimize a luminance error between corresponding pixels of both said images over the whole of said images.

24. An image processing apparatus according to claim 23, wherein said arithmetic means performs, for the association between two picked-up images obtained by said camera, a coordinate transformation using a projective transformation for associating one of said two picked-up images with said base image and an inverse transformation of a projective transformation for associating the other picked-up image with said base image.

25. An image processing apparatus according to claim 24, wherein said transformation parameter calculating means derives a distortion parameter representative of a distortion factor of said picked-up images taking place at image pickup by said camera and performs a projective transformation of said picked-up image, from which distortion is removed through the use of said distortion parameter, to associate the transformed image with said base image.

26. An image processing apparatus according to claim 22, wherein said transformation parameter calculating means derives a distortion parameter representative of a distortion factor of a picked-up image taking place at image pickup by said camera and performs an image conversion of one of said picked-up image and said base image through the use of the derived distortion parameter, and associates the converted image with the other image to minimize a luminance error between corresponding pixels of both said images.

27. An image processing apparatus according to claim 19, wherein said transformation parameter calculating means corrects a luminance value of said base image in accordance with a luminance values of said picked-up images.

28. An image processing apparatus according to claim 27, wherein said transformation parameter calculating means extracts, from said picked-up images, an area in which luminance values are approximately equal to each other and obtains an average value of said luminance values in the extracted area to replace a luminance value of a corresponding pixel in said base image with said average value.

29. An image processing apparatus according to claim 19, wherein said image inputting means inputs a plurality of picked-up images obtained by a plurality of cameras standing in a predetermined positional relationship.

30. An image processing method of processing a plurality of images picked up by a camera, comprising:

an image inputting step of inputting a picked-up image obtained by taking a photograph of a pattern with a predefined geometrical configuration through the use of a camera;

an image holding step of holding a base image comprising said pattern with said predefined geometrical configuration;

transformation parameter calculating step of calculating a transformation parameter on the basis of the association in pixel between said picked-up image and said base image to minimize luminance error between corresponding pixels of both said picked-up image and base image; and

an arithmetic step of obtaining the association in pixel among a plurality of picked-up images obtained by said camera through the use of the calculated transformation parameter.

31. An image processing method according to claim 30, further comprising an image generating step of generating said base image comprising said pattern with said predefined geometrical configuration according to the definition, with said image holding step holding said image generated by said image generating step.

32. An image processing method according to claim 30, further comprising an image generating step of generating said base image comprising said pattern with said predefined geometrical configuration according to the definition and a light projecting step of projecting the generated base image on a generally plain plane, with said image inputting step inputting a picked-up image obtained by photographing the projected image, given by said light projecting step, through the use of a camera.

33. An image processing method according to claim 30, wherein said transformation parameter calculating step performs an image conversion of one of said picked-up image and said base image and further associates the converted image with the other image.

34. An image processing method according to claim 33, wherein said transformation parameter calculating step derives a projective transformation parameter, performs an image conversion of one of said picked-up image and said base image through the use of the derived projective transformation parameter, and associates the converted image with the other image to minimize a luminance error between corresponding pixels of both said images over the whole of said images.

35. An image processing method according to claim 34, wherein said arithmetic step performs, for the association between two picked-up images obtained by said camera, a coordinate transformation using a projective transformation for associating one of said two picked-up images with said base image and an inverse transformation of a projective transformation for associating the other picked-up image with said base image.

36. An image processing method according to claim 35, wherein said transformation parameter calculating step derives a distortion parameter representative of a distortion factor of said picked-up images taking place at image pickup by said camera and performs a projective transformation of said picked-up image, from which distortion is removed through the use of said distortion parameter, to associate the transformed image with said base image.

37. An image processing method according to claim 30, wherein said transformation parameter calculating step corrects a luminance value of said base image in accordance with a luminance values of said picked-up images.

38. An image processing method according to claim 37, wherein said transformation parameter calculating step extracts, from said picked-up images, an area in which luminance values are approximately equal to each other and obtains an average value of said luminance values in the extracted area to replace a luminance value of a corresponding pixel in said base image with said average value.

39. An image processing method according to claim 30, wherein said image inputting step inputs a plurality of picked-up images obtained by a plurality of cameras standing in a predetermined positional relationship.

40. A computer-readable program providing medium for providing, in a material computer-readable form, a computer program for implementing, on a computer system, a camera calibration to calculate a parameter representative of a characteristic of a camera, said computer program comprising:

an image inputting step of inputting a picked-up image obtained by taking a photograph of a pattern with a predefined geometrical configuration through the use of a camera;

an image holding step of holding a base image comprising said pattern with said predefined geometrical configuration; and

a transformation parameter calculating step of calculating a transformation parameter on the basis of the association in pixel between said picked-up image and said base image to minimize luminance error between corresponding pixels of both said picked-up image and base image.

41. A camera which conducts a camera calibration, said camera comprising:

image inputting means for inputting a picked-up image;

an image holding means for holding a base image comprising a pattern with a predefined geometrical configuration; and

transformation parameter calculating means for calculating a transformation parameter on the basis of the association in pixel between said picked-up image and base image to minimize luminance error between corresponding pixels of both said picked-up image and base image, wherein said picked-up image, has a pattern with said predefined geometrical configuration, and is inputted through the image inputting means and said base image.