Three-dimensional calibration using orientation and position sensitive calibration pattern

• US 8,072,654 B2
• Filed: 11/23/2009
• Issued: 12/06/2011
• Est. Priority Date: 12/22/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lefevere, Francois-Marie
• Primary Examiner(s): Safaipour, Houshang
• Application Number:
  US12/624,328
• Publication Number:
  US 20100067072A1
• Time in Patent Office: 743 Days
• Field of Search: 358/474, 358/493, 358/497, 358/488, 358/461, 358/406
• US Class Current: 358/474
• CPC Class Codes: H04N 1/00002 Diagnosis, testing or measuring; Detecting, analysing or monitoring not otherwise provided for error detection, error correction or monitoring in digital computers or digital computer components G06F11/00
• CPC Class Codes: H04N 1/00013 Reading apparatus
• CPC Class Codes: H04N 1/00031 Testing, i.e. determining the result of a trial H04N1/00029 takes precedence; using test signals for transmission mode changing H04N2201/33371
• CPC Class Codes: H04N 1/00045 using a reference pattern designed for the purpose, e.g. a test chart
• CPC Class Codes: H04N 1/00053 out of service, i.e. outside of normal operation
• CPC Class Codes: H04N 1/00063 using at least a part of the apparatus itself, e.g. self-testing
• CPC Class Codes: H04N 1/00087 Setting or calibrating picture signal circuits H04N1/40
• CPC Class Codes: H04N 1/195 the array comprising a two-dimensional array or a combination of two-dimensional arrays
• CPC Class Codes: H04N 1/19594 using a television camera or a still video camera
• CPC Class Codes: H04N 2201/0434 specially adapted for scanning pages of a book
• CPC Class Codes: H04N 2201/0448 for positioning scanning elements not otherwise provided for; Aligning, e.g. using an alignment calibration pattern arrangements for positioning elements within a scanning head H04N2201/02406

Abstract

Systems and methods using an orientation/position-sensitive calibration pattern for three-dimensional calibration of an imaging system, such as one used in a process for scanning documents are disclosed. The method generally includes positioning the pattern on a support, capturing images using cameras to be calibrated, each image containing at least a unique orientation and position sensitive sub-area of the pattern, determining a set of coordinate pairs of corresponding points in the image and the pattern for each image utilizing image data and pattern information, and performing optimization utilizing the sets of coordinate pairs to calibrate relative position, orientation, zoom, and/or lens distortion, etc. of each camera so as to construct a three-dimensional camera calibration model. The pattern is generally comprised of overlapping sub-areas of a minimum portion of the pattern.

Claims

1. A method, comprising:

positioning a calibration pattern relative to one or more cameras, the calibration pattern having a plurality of sub-areas, each sub-area being unique within the calibration pattern;

capturing images using the one or more cameras, each image containing at least one of the sub-areas of the calibration pattern;

determining image data corresponding to each image and calibration data corresponding to the calibration pattern; and

calibrating the one or more cameras based on the image data and the calibration data.

2. The method of claim 1, wherein each sub-area is orientation and position sensitive within the calibration pattern.

3. The method of claim 1, further comprising determining a set of coordinate pairs for each image utilizing the image data and the pattern data, each coordinate pair including coordinates of corresponding points in the image and in the calibration pattern, the calibrating being based on the set of coordinate pairs.

4. The method of claim 3, further comprising performing optimization utilizing the set of coordinate pairs for each image to calibrate at least one of relative position, orientation, zoom, lens distortion to construct a three-dimensional calibration model of each camera of the one or more cameras.

5. The method of claim 3, further comprising:

repositioning the calibration pattern at a different position relative to the one or more cameras; and

repeating the capturing and determining.

6. The method of claim 1, wherein the sub-areas overlap and each sub-area is an area of a predetermined minimum portion of the calibration pattern, each sub-area being position sensitive within the calibration pattern by being patterned differently from other sub-areas in the calibration pattern.

7. The method of claim 1, wherein a support is configured to support a target object to be imaged, the target object being selected from the group consisting of a bound document and an unbound document.

8. The method of claim 1, wherein the calibration pattern comprises features, at least some of the features being marked with an orientation/position marker, a combination of the features and markers within each sub-area rendering the sub-area unique within the calibration pattern and orientation and position sensitive.

9. The method of claim 1, wherein the calibration pattern comprises a grid of features comprising alternating colored blocks, at least some of the features being marked with an orientation/position marker, a combination of the features and markers within each sub-area rendering the sub-area unique within the calibration pattern and orientation and position sensitive.

10. A computer program product embodied on a non-transitory computer readable medium, the computer program product including instructions that, when executed by a processor, cause the processor to perform actions comprising:

receiving image data corresponding to captured images of a calibration pattern, the calibration pattern being positioned on a support relative to one or more cameras that are used to capture the images and having a plurality of sub-areas, each sub-area being unique within the calibration pattern, each image containing at least one of the sub-areas of the calibration pattern;

determining image data corresponding to each image and calibration data corresponding to the calibration pattern; and

calibrating the one or more cameras based on the image data and the calibration data.

11. The computer program product of claim 10, wherein each sub-area is orientation and position sensitive within the calibration pattern.

12. The computer program product of claim 10, wherein the actions further comprise determining a set of coordinate pairs for each image utilizing the image data and the pattern data, each coordinate pair including coordinates of corresponding points in the image and in the calibration pattern, the calibrating being based on the set of coordinate pairs.

13. The computer program product of claim 12, wherein the actions further comprise performing optimization utilizing the set of coordinate pairs for each image to calibrate at least one of relative position, orientation, zoom, lens distortion to construct a three-dimensional calibration model of each camera of the one or more cameras.

14. The computer program product of claim 12, wherein the operations further comprise repeating the capturing and determining after the calibration pattern has been repositioned at a different position relative to the one or more cameras.

15. The computer program product of claim 10, wherein the sub-areas overlap and each sub-area is an area of a predetermined minimum portion of the calibration pattern, each sub-area being position sensitive within the calibration pattern by being patterned differently from other sub-areas in the calibration pattern.

16. The computer program product of claim 10, wherein the calibration pattern comprises features, at least some of the features being marked with an orientation/position marker, a combination of the features and markers within each sub-area rendering the sub-area unique within the calibration pattern and orientation and position sensitive.

17. The computer program product of claim 10, wherein the calibration pattern comprises a grid of features comprising alternating colored blocks, at least some of the features being marked with an orientation/position marker, a combination of the features and markers within each sub-area rendering the sub-area unique within the calibration pattern and orientation and position sensitive.

18. A system, comprising:

one or more cameras to be calibrated;

a calibration pattern that is positioned relative to the one or more cameras, the calibration pattern having a plurality of sub-areas, each sub-area being unique within the calibration pattern, the one or more cameras capturing images of the calibration pattern, each image containing at least one of the sub-areas of the calibration pattern; and

a processor processing the images and the calibration pattern to determine image data corresponding to each image and calibration data corresponding to the calibration pattern, and to calibrate the one or more cameras based on the image data and the calibration data.

19. The system of claim 18, wherein each sub-area is orientation and position sensitive within the calibration pattern.

20. The system of claim 18, wherein the processor determines a set of coordinate pairs for each image utilizing the image data and the pattern data, each coordinate pair including coordinates of corresponding points in the image and in the calibration pattern, the one or more cameras being calibrated being based on the set of coordinate pairs.

21. The system of claim 20, wherein the processor performs optimization utilizing the set of coordinate pairs for each image to calibrate at least one of relative position, orientation, zoom, lens distortion to construct a three-dimensional calibration model of each camera of the one or more cameras.

22. The system of claim 18, wherein the sub-areas overlap and each sub-area is an area of a predetermined minimum portion of the calibration pattern, each sub-area being position sensitive within the calibration pattern by being patterned differently from other sub-areas in the calibration pattern.

23. The system of claim 18, further comprising a support supporting a target object to be imaged, the target object being selected from the group consisting of a bound document and an unbound document.

24. The system of claim 18, wherein the calibration pattern comprises features, at least some of the features being marked with an orientation/position marker, a combination of the features and markers within each sub-area rendering the sub-area unique within the calibration pattern and orientation and position sensitive.

25. The system of claim 18, wherein the calibration pattern comprises a grid of features comprising alternating colored blocks, at least some of the features being marked with an orientation/position marker, a combination of the features and markers within each sub-area rendering the sub-area unique within the calibration pattern and orientation and position sensitive.