SYSTEMS, METHODS, AND APPARATUSES FOR STEREOSCOPIC IMAGING

• US 20170064290A1
• Filed: 08/28/2015
• Published: 03/02/2017
• Est. Priority Date: 08/28/2015
• Status: Active Grant

• Current Assignee: ETH Zurich, Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: ETH Zurich, Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Mora, Javier Alonso, Beardsley, Paul
• Granted Patent: US 9,992,482 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 17/061 on-axis systems with at least one of the mirrors having a central aperture
• CPC Class Codes: G02B 27/0012 Optical design, e.g. procedures, algorithms, optimisation routines
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/51 Housings

Abstract

An omnidirectional camera apparatus configured to facilitate omnidirectional stereo imaging is described. The apparatus may include a first convex mirror, a first camera disposed at the first convex mirror, a second convex mirror, and a second camera disposed at the second convex mirror. The first convex mirror and the second convex mirror may be arranged such that a first mirrored surface of the first convex mirror and a second mirrored surface of the second convex mirror may face each other. The first camera may capture imagery reflected off the second convex mirror. The second camera may capture imagery reflected off the first convex mirror. A method of calibrating an omnidirectional camera apparatus is also described.

Claims

1. An omnidirectional stereo camera system, comprising:

a first convex mirror having a first principle axis, a first mirrored surface, and a first central portion disposed along the first principle axis, the first central portion being configured to allow light to pass through the first central portion;

a second convex mirror having a second principle axis, a second mirrored surface, and a second central portion disposed along the second principle axis, the second central portion being configured to allow light to pass through the second central portion, the first convex mirror and the second convex mirror being arranged such that the first principle axis and the second principle axis are aligned and such that the first mirrored surface and the second mirrored surface face each other;

a first camera disposed behind the first convex mirror and aligned with the first principle axis such that light passing through the first central portion of the first convex mirror impinges on the first camera; and

a second camera disposed behind the second convex mirror and aligned with the second principle axis such that light passing through the second central portion of the second convex mirror impinges on the second camera.

2. The omnidirectional stereo camera system of claim 1, wherein the first camera is configured to capture imagery reflected by the second convex mirror, and wherein the second camera is configured to capture imagery reflected by the first convex mirror.

3. The omnidirectional stereo camera system of claim 1, wherein the first central portion is an aperture in the first convex mirror, and wherein the second central portion is an aperture in the second convex mirror.

4. The omnidirectional stereo camera system of claim 1, wherein the first central portion is a transparent or semitransparent portion of the first convex mirror, and wherein the second central portion is a transparent or semitransparent portion of the second convex mirror.

5. The omnidirectional stereo camera of claim 1, wherein the first convex mirror, the first camera, the second convex mirror, and the second camera comprise part of a movable robot.

6. The omnidirectional stereo camera of claim 1, further comprising:

a spherical shell housing at least the first convex mirror, the first camera, the second convex mirror, and the second camera; and

wherein the positioning of the first convex mirror, the first camera, the second convex mirror, and the second camera within the spherical shell is such that the center of gravity of the omnidirectional stereo camera system is offset from a center point of the spherical shell.

7. The omnidirectional stereo camera of claim 1, wherein the first convex mirror, the first camera, the second convex mirror, and the second camera comprise part of a unitary structure within the spherical shell, and wherein the shell rotates independently from the unitary structure.

8. The omnidirectional stereo camera of claim 7 wherein the unitary structure includes a drive mechanism configured to drive rotational motion of the spherical shell independently from the unitary structure.

9. The omnidirectional stereo camera of claim 1, wherein the first convex mirror is defined by a cross-sectional profile, wherein the profile comprises a first portion defined by a hyperbolic function, the first portion encompassing a first area between an edge of the first convex mirror and an intermediate location located between the edge and a central portion of the first convex mirror, and second portion defined by a linear or quadratic function, the second portion encompassing a second area between the intermediate location and the central portion.

10. The omnidirectional stereo camera of claim 1, further comprising one or more physical processors coupled to the first camera and/or second camera, wherein the one or more physical processors are configured by computer-readable instructions to:

receive first information from the first camera and second information from the second camera, the first information describing a first image captured by the first camera, the first image depicting a first object reflected off the second convex mirror, the second information describing a second image captured by the second camera, the second image depicting the first object reflected off the first convex mirror; and

determine a position and/or orientation of the first object within three-dimensional space based on the received first and second information.

11. A method of calibrating an omnidirectional stereo camera apparatus, the omnidirectional stereo camera apparatus comprising a first convex mirror and a first camera disposed at the first convex mirror, a second convex mirror and a second camera disposed at the second convex mirror, the first convex mirror and the second convex mirror being arranged such that a first mirrored surface of the first convex mirror and a second mirrored surface of the second convex mirror face each other, wherein the first camera captures imagery reflected off the second convex mirror and the second camera captures imagery reflected off the first convex mirror, the method being implemented in a computer system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:

estimating a first system model, the first system model comprising estimations of one or more of a first shape of the first convex mirror, a first position of the first convex mirror, a second shape of the second convex mirror, a second position of the second convex mirror, a third position of the first camera, or a fourth position of the second camera;

obtaining imagery captured by the first camera and the second camera depicting a reference object, the reference object being positioned in three-dimensional space adjacent the omnidirectional stereo camera, the imagery including a first image comprising a first representation of the reference object captured by the first camera and a second image comprising a second representation of the reference object captured by the second camera;

determining first position information corresponding to the first representation of the reference object in the first image and second position information corresponding to the second representation of the reference object in the second image;

estimating a reconstruction of the reference object based on the first system model, the first position information, and the second position information;

reprojecting the reconstruction of the reference object onto image planes, such that the reconstruction of the reference object is reprojected onto a first plane and a second plane;

determining third position information corresponding to the reprojected reference object in the first plane and fourth position information corresponding to the reprojected reference object in the second plane; and

updating the system model based on the reprojections, such that the first system model is updated based on one or both of a first difference between the first position information and the third position information, or a second difference between the second position information and the fourth position information.

12. The method of claim 11, additionally comprising:

estimating a second reconstruction of the reference object based on the updated first system model, the first position information, and the second position information;

reprojecting the second reconstruction of the reference object onto image planes, such that the second reconstruction of the reference object is reprojected onto a third plane and a fourth plane;

determining fifth position information corresponding to the second reprojected reference object in the third plane and sixth position information corresponding to the second reprojected reference object in the fourth plane; and

updating the updated first system model based on the second reprojections, such that the updated first system model is further updated based on one or both of a third difference between the first position information and the fifth position information, or a fourth difference between the second position information and the sixth position information.

13. The method of claim 11, wherein the estimations of mirror shapes correspond to mirror shapes having central intersection points.

14. The method of claim 11, wherein the first system model is configured such that the estimated shapes of one or both of the first convex mirror or second convex mirror correspond to a hyperbolic model defined by discrete hyperbolas spaced at discrete steps.

15. The method of claim 11, wherein the first system model is configured such that the estimated shapes of one or both of the first convex mirror or second convex mirror correspond to a polynomial assuming radial symmetry.

16. The method of claim 11, wherein the first system model is configured such that the estimated shapes of one or both of the first convex mirror or second convex mirror correspond to a polynomial assuming radial asymmetry.

17. The method of claim 11, wherein the first system model is configured such that one or more of the first position of the first convex mirror, the second position of the second convex mirror, the third position of the first camera, or the fourth position of the second camera is provided as a fixed origin position of the first system model.