Methods and apparatus for calibrating and/or adjusting the arrangement of cameras in a camera pair

• US 10,477,186 B2
• Filed: 01/17/2018
• Issued: 11/12/2019
• Est. Priority Date: 01/17/2018
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Sheridan, Ryan Michael
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/873,893
• Publication Number:
  US 20190222824A1
• Time in Patent Office: 664 Days
• Field of Search: 348136
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/172 image signals comprising non-image signal components, e.g. headers or format information
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/00 Details of television systems scanning details or combination thereof with generation of supply voltages H04N3/00

Abstract

Methods and apparatus for checking the alignment and/or spacing of cameras of a camera pair, e.g., stereoscopic camera pair, are described. Images of a calibration target are captured by first and second cameras of a camera pair being calibrated The captured first and second images, which may correspond to all or a portion of the calibration target, are combined in accordance with the invention to generate a calibration image which is then displayed to an operator of the calibration apparatus including the calibration target. Given the position of at least some different color grid elements in the calibration target, when proper camera spacing and optical axis alignment of the cameras is achieved, the overlaid image will include an overlay of two different color image grid elements resulting in the generated calibration image including grid elements of a different color than that of the overlaid grid elements.

Claims

1. A calibration apparatus, comprising:

a calibration target including a grid pattern, said grid pattern including a plurality of grid elements, each grid element occupying an area of a grid, said grid pattern including subsets of grid elements of different colors;

a camera pair mount for supporting a camera pair facing said calibration target, said camera pair including a first camera and a second camera, said first and second cameras being positioned adjacent one another facing said calibration target;

a display; and

a processor configured to;

generate a calibration image by combining a first image captured by said first camera on a second image captured by the second camera; and

control the display to display the generated calibration image.

2. The calibration apparatus of claim 1 wherein said subsets of grid elements of different colors includes at least four subsets of different color grid elements, said four subsets of grid elements including a first subset of a first color, a second subset of a second color, a third subset of a third color and a fourth subset of a fourth color, said first, second, third and fourth colors being different.

3. The calibration apparatus of claim 2, wherein said first color is white and wherein said first subset of grid elements and said second subset of grid elements form the majority of grid elements in said grid pattern.

4. The calibration apparatus of claim 3, wherein a first intersection point of a first pair of third and fourth grid elements corresponds to an intended intersection point of a first camera optical axis and said calibration target and wherein a second intersection point of a second pair of third and fourth grid elements corresponds to an intended intersection point of a second camera optical axis and said calibration target.

5. The calibration apparatus of claim 4,

wherein every other grid element in each row of grid elements of the calibration target is white;

and

wherein the first and second intersection points correspond to a predetermined desired camera spacing.

6. The calibration apparatus of claim 5, wherein the calibration target includes more grid elements in said first subset than in said second subset.

7. The calibration apparatus of claim 6, wherein said second color is the darkest color used for a grid element, said first and second subsets of grid elements forming a checkerboard pattern over the majority of the calibration target.

8. The calibration apparatus of claim 5,

wherein the first and second intersection points are separated from each other by an even number of horizontal grid element units, where each horizontal grid element unit has a length corresponding to the length of a grid element in the horizontal direction.

9. The calibration apparatus of claim 8, wherein the first intersection point corresponds to a point where a first set of surrounding grid elements meet, said first set of surrounding grid elements including first and second grid elements from said first subset, a first grid element from said third subset and a first grid element from said fourth subset.

10. The calibration apparatus of claim 9, wherein said first grid element from said third subset and said first grid element from said fourth subset are located in said first set of surrounding grid elements on a first diagonal.

11. The calibration apparatus of claim 10, wherein said first and second grid elements from said first subset are located in said first set of surrounding grid elements on a second diagonal.

12. The calibration apparatus of claim 11,

wherein the second intersection point corresponds to a point where a second set of surrounding grid elements meet, said second set of surrounding grid elements including third and fourth grid elements from said first subset, a second grid element from said third subset and a second grid element from said fourth subset;

wherein said second grid element from said third subset and said second grid element from said fourth subset are located in said second set of surrounding grid elements on a third diagonal; and

wherein said third and fourth grid elements from said first subset are located in said second set of surrounding grid elements on a fourth diagonal.

13. The calibration apparatus of claim 12, wherein said first and second sets of surrounding grid elements are surrounded on all four sides by a checkerboard pattern of grid element corresponding to the first and second subsets of grid elements.

14. The calibration apparatus of claim 13, wherein said third and fourth colors combine to form a fifth color when grid elements of said third and fourth subsets overlap in the generated image, said fifth color being different from said first, second, third and fourth colors.

15. The calibration apparatus of claim 14,

wherein said third and fourth colors are primary colors;

wherein said fifth color is a secondary color formed by mixing two primary colors; and

wherein said first and second intersection points are equidistant in the horizontal dimension from a calibration target center point corresponding to an intended horizontal center point of the camera pair, said calibration target center point corresponding to an intersection point of two grid elements of either the third or fourth subsets of grid elements.

16. The calibration apparatus of claim 1, wherein the calibration target is a physical object.

17. A method, comprising:

operating first and second cameras of a camera pair, mounted on a camera mount and facing a calibration target, said calibration target being a physical object, said calibration target including a grid pattern to capture first and second images, respectively, of said grid pattern, said grid pattern including a plurality of grid elements, each grid element occupying an area of a grid, said grid pattern including subsets of grid elements of different colors;

operating a processor to generate a calibration image by combining a first image captured by said first camera on a second image captured by the second camera; and

displaying the generated calibration image.

18. The method of claim 17,

wherein said subsets of grid elements of different colors includes at least four subsets of different color grid elements, said four subsets of grid elements including a first subset of a first color, a second subset of a second color, a third subset of a third color and a fourth subset of a fourth color, said first, second, third and fourth colors being different;

and

wherein generating the calibration image includes generating a grid element having a color which is different from any of said first, second, third and fourth colors when said camera pair is properly aligned and positioned.

19. The method of claim 18, wherein said first color is white and wherein said first subset of grid elements and said second subset of grid elements form the majority of grid elements in said grid pattern;

and

wherein said grid element having a color which is different from any of said first, second, third and fourth colors is a grid element of a secondary color formed by combining said third and fourth colors.

20. The method of claim 18, further comprising, prior to operating the first and second cameras to capture said first and second images,

operating one of said first and second cameras to capture an image of the grid pattern to control camera positioning;

and

operating a camera position controller to control at least one of a vertical positioning motor and a horizontal positioning motor to automatically adjust at least one of a vertical position of the camera pair or a horizontal position of the camera pair based on the image captured by the one of said first and second cameras to align an optical axis of at least one of the first and second cameras with an intended optical axis and grid intersection point prior to generation of the calibration image.

21. An apparatus, comprising:

a calibration target including a grid pattern, said calibration target being a physical object, said grid pattern including a plurality of grid elements, each grid element occupying an area of a grid, said plurality of grid elements including grid elements of a first color, grid elements of a second color, grid elements of a third color and grid elements of a fourth color, said first, second, third and fourth colors being different;

a camera pair mount for supporting a camera pair facing said calibration target, said camera pair including a first camera and a second camera, said first and second cameras being positioned adjacent one another facing said calibration target;

a display; and

a processor configured to;

generate a calibration image by combining a first image captured by said first camera with a second image captured by the second camera by superimposing the first image on the second image; and

control the display to display the generated calibration image.