PANORAMIC STEREO CATADIOPTRIC IMAGING

• US 20130208083A1
• Filed: 02/15/2012
• Published: 08/15/2013
• Est. Priority Date: 02/15/2012
• Status: Active Grant

• Current Assignee: City University of Hong Kong
• Original Assignee: City University of Hong Kong
• Inventors: Li, Youfu, Li, Weiming
• Granted Patent: US 9,250,510 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G03B 35/00 Stereoscopic photography
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The disclosed subject matter relates to providing panoramic stereo vision captured with a single camera scheme employing a catadioptric optical path. In an aspect the presently disclosed subject matter can capture a stereo image over a full 360-degree horizontal field of view or portions thereof. Further, the vertical FOV is enlarged compared to conventional cata-fisheye schemes. A system according to the instant subject matter can be calibrated with a computational model that can accommodate a non-single viewpoint imaging model to conduct 3D reconstruction in Euclidean space.

Claims

1. A system, comprising:

a sensor component that detects energy that impinges on the sensor component;

a spatial processor component that determines spatial information relating to the energy detected by the sensor component;

a refractive element component that refracts at least part of the energy to impinge on the sensor component; and

a reflective element component that reflects at least part of the energy to impinge on the sensor component.

2. The system of claim 1, wherein the energy is at least in part visual spectrum energy.

3. The system of claim 1, wherein the spatial information includes three-dimensional spatial information.

4. The system of claim 1, wherein reflective element component reflects at least part of the energy through the refractive element component to impinge on the sensor component.

5. The system of claim 1, wherein the sensor component comprises a digital camera or digital video camera.

6. The system of claim 1, wherein the refractive element is a wide-angle lens or a fisheye lens.

7. The system of claim 1, wherein reflective element is a radially symmetric convex mirror.

8. The system of claim 7, wherein reflective element is conical, spherical, elliptical, parabolic, or hyperbolic.

9. The system of claim 1, wherein the sensor component, the refractive element component, and the reflective element component are coaxially oriented.

10. The system of claim 1, wherein the energy impinging on the sensor component comprises a panoramic stereo image.

11. The system of claim 10, wherein spatial information captured from the panoramic stereo image includes three-dimensional panoramic spatial information comprising a vertical stereo field of view of at least 20-degrees.

12. The system of claim 1, wherein the spatial processor component employs a model comprising at least a design parameter or a performance parameter.

13. The system of claim 12, wherein the at least a design parameter includes a distance between the refractive element component and the reflective element component, a refractive parameter of the refractive component, or a reflective parameter of the reflective component.

14. The system of claim 12, wherein at least the performance parameter includes an entire vertical field of view parameter, a stereo vertical field of view parameter, an image spatial resolution parameter, a baseline length parameter, or an equivalent baseline length parameter.

15. A method, comprising:

receiving a set of imaged scene information related to an image of a scene, the image of the scene comprising at least a portion of a field of view of a refractive element and a portion of a field of view of a reflective element;

determining a first set of image information for a point of the scene, the first set of image information being related to the field of view of the refractive element;

determining a second set of image information for the point of the scene, the second set of image information being related to the field of view of the reflective element; and

determining 3-dimensional spatial information for the point of the scene based on the first set of image information and the second set of image information.

16. The method of claim 15, wherein the determining the 3-dimensional spatial information is a function of, at least in part, a parametric model of a system that generated the received first set of information.

17. The method of claim 16, wherein the determining the 3-dimensional spatial information is a function of, at least in part, a parametric model that includes a parameter for a refractive element and parameters for a rotationally symmetric reflective convex element, the refractive element and the reflective element being coaxially oriented in the parametric model.

18. The method of claim 17, wherein the determining a first set of image information includes determining a refractive field of view, the determining a second set of image information includes determining a reflective field of view, and the determining 3-dimensional spatial information includes determining an overlapping vertical field of view based on the refractive field of view and the reflective field of view.

19. The method of claim 16, wherein the determining 3-dimensional spatial information relies, at least in part, on a parametric model,

the parametric model parameterizing a design parameter that includes:

a distance between a refractive element and a reflective element a,

a refractive parameter of the refractive component, or

a reflective parameter of the reflective component, or

the parametric model parameterizing a performance parameter that includes;

an entire vertical field of view parameter,

a stereo vertical field of view parameter,

an image spatial resolution parameter,

a baseline length parameter, or

an equivalent baseline length parameter.

20. The method of claim 15, wherein the determining 3-dimensional spatial information further comprises calibrating a parametric model of a system with homography.

21. The method of claim 20, wherein the calibrating the parametric model includes employing a known image in a plurality of scene poses.

22. The method of claim 20, wherein the determining 3-dimensional spatial information includes employing at least a first set of viewing cone models to describe image formation geometry of a dioptric image and at least a second set of viewing cone models to describe image formation geometry of a catadioptric image.

23. A computer-readable storage medium having computer-executable instructions that, in response to execution, cause a computing device including a processor to perform operations, comprising:

receiving a scene image information comprising refractive element image information and reflective element image information of a scene;

determining a first set of image information, for a point of the scene, related to the refractive element image information;

determining a second set of image information, for the point of the scene, related to the reflective element image information; and

determining 3-dimensional spatial information for the point of the scene based on the first set of image information and the second set of image information.

24. A system, comprising:

means for detecting energy impinging on a plane;

means for determining 3-dimensional spatial information relating to the energy detected by the means for detecting energy;

means for refracting at least some of the energy to impinge on the means for detecting energy; and

means for reflecting at least some of the energy to impinge on the means for detecting energy.