WIDE-FIELD OF VIEW (FOV) IMAGING DEVICES WITH ACTIVE FOVEATION CAPABILITY

• US 20140218468A1
• Filed: 04/04/2013
• Published: 08/07/2014
• Est. Priority Date: 04/05/2012
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Augmented Vision, Inc.
• Inventors: Gao, Chunyu, Hua, Hong
• Granted Patent: US 9,851,563 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0145 creating an intermediate image
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 25/001 Eyepieces
• CPC Class Codes: G02B 26/00 Optical devices or arrangements for the control of light using movable or deformable optical elements control of light by modification of the optical properties of the media of the elements involved therein G02F1/00
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 27/144 using partially transparent surfaces without spectral selectivity G02B27/147 takes precedence
• CPC Class Codes: G02B 27/283 used for beam splitting or combining
• CPC Class Codes: G02B 5/04 Prisms
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The present invention comprises a foveated imaging system capable of capturing a wide field of view image and a foveated image, where the foveated image is a controllable region of interest of the wide field of view image.

Claims

1. A foveated imaging system (100), capable of capturing a wide field of view image and a foveated image, where the foveated image is a controllable region of interest of the wide field of view image, the system comprising:

a. an objective lens (110), facing an external scene, configured to receive the incoming light from the external scene and to focus the light upon a beamsplitter;

b. a beamsplitter (120), configured to split incoming light from an external scene into a wide field of view imaging pa ⁻

1(125) and a foveated imaging path (135);

c. a wide field of view imaging path (125), the wide field of view imaging path comprising;

i. a first stop (127), which limits the amount of light received in the wide field of view path from the beamsplitter (120);

ii. a wide field-of-view imaging lens (130), configured to receive light from the stop (127) and form a wide field view image on a wide field of view imaging sensor;

iii. a wide field-of-view imaging sensor (140), configured to receive light from the wide field of view imaging lens (130);

d. a foveated view imaging path (135), the foveated view imaging path comprising

i. a second stop (137), which limits the amount of light received in the foveated imaging path from the beamsplitter (120);

ii. a scanning mirror (150), capable of being controlled to reflect the light from the beamsplitter (120);

iv. a foveated imaging lens (160), configured to receive a portion of the light, associated with a region of interest of the external scene, from the scanning mirror (150) and form a foveated image on a foveated imaging sensor; and

v. a foveated imaging sensor (170), configured to receive light from the foveated imaging lens (160);

whereupon the incoming light from the external scene passes through the objective lens (110) to the beamsplitter (120), whereupon the beamsplitter (120) divides the light into the two optical paths, a wide field of view imaging path (125) and a foveated imaging path (135), whereupon the light passes through the first stop (127) to the wide field of view imaging lens (130) along the wide field of view imaging path (125), where the lens focuses the wide field of view image upon the wide field of view imaging sensor (140), whereupon the light passes through the second stop (137) to the scanning mirror (150) along the foveated imaging path (135), whereupon the scanning mirror (150) reflects a region of interest toward the foveated imaging lens (160) through the beam splitter (120), whereupon the foveated imaging lens (160) focuses the foveated image upon the foveated imaging sensor (170);

whereupon the two images are recorded by the sensors, a wide field of view image and a high resolution image of the region of interest within it.

2. The foveated imaging system according to claim 1, wherein the objective lens (110) is disposed on the front of the system, wherein the beamsplitter (120) is disposed adjacent to objective lens receiving light from the objective lens, whereupon the beamsplitter (120) divides the light into the two optical paths, a wide field of view imaging path (125) and a foveated imaging path (135), whereupon the first stop (127) is in optical communication with the beamsplitter (120) along the wide field of view imaging path (125), where upon the second stop (137) is in optical communication with the beamsplitter (120) along the foveated imaging path (135), wherein the scanning mirror (150) is disposed near or at the position of the second stop (137), where it receives light from the beamsplitter (120) along the foveated imaging path (135) and reflects the light back to the beamsplitter (120), wherein the wide-field of view imaging lens (130) is disposed to face the first stop (127) along the wide field of view imaging path (125), where it receives light from the beamsplitter (120) through the first stop (127) along the wide field of view path (125), wherein the foveated imaging lens (160) is disposed to face the beamsplitter (120), where it receives light from the beamsplitter (120) reflected from the scanning mirror (150) along the foveated imaging path (135), wherein the wide-field of view imaging sensor (140) is disposed to face the wide field of view imaging lens (130), and wherein the foveated imaging sensor (170) is disposed to face the foveated imaging lens (160), whereupon the two images are recorded by the sensors, a wide field of view image and a high resolution image of the region of interest within it.

3. The foveated imaging system according to any one of the preceding claims, where the objective lens is a group of rotationally symmetric lenses to capture an umbrella-like or hemispherical shaped field of view.

4. The foveated imaging system according to any one of claims 1-2, where the objective lens utilizes a curved mirror along with necessary rotationally symmetric refractive optical elements to capture a ring-like panoramic field of view.

5. The foveated imaging system according to any one of claims 1-2, where the imaging sensors (140) and (170) are any light sensing device containing an array of light sensing units (pixels) that converts photons into electronic signals, Including, but not limited to, a charge-couple device (CCD), or a complementary metal-oxide-semiconductor (CMOS) or other type of light sensing devices.

6. The foveated imaging system according to any one of claims 1-2, where the scanning mirror (150) is any type of fast moving mirror device whose scanning motion can be electronically controlled, including, but not limited to, voice coil mirror, piezoelectric mirror, Micro-Electro-Mechanical System (MEMS) mirror or other type of scanning mirrors.

7. The foveated imaging system according to any one of claims 1-2, where the scanning mirror is a dual axis scanning unit (252) capable of continuously sampling the wide-FOV through tilting motions (253) and (254) along X and Y axes.

8. The foveated imaging system according to any one of claims 1-2, where the scanning mirror is a rotating single axis scanning unit (255), in which the mirror samples the wide-FOV through a tilt motion (257) along the Y axis and a rotation motion (258) along the Z axis.

9. The foveated imaging system according to any one of claims 1-2, where the aperture of the scanning mirror performs the function of the second stop (137) to limits the amount of light received in the foveated imaging path from the beamsplitter;

10. The foveated imaging system according to any one of claims 1-2, where the beamsplitter (120) is in form of a cube or a plate and could be a non-polarized beamsplitter or a polarized beamsplitter.

11. The foveated imaging system according to any one of claims 1-2, where the beamsplitter is a polarized beamsplitter and a quarter-wave plate is used along with the beamsplitter to increase the light efficiency, where the quarter-wave plate is positioned in the between the beamsplitter (120) and the scanning mirror (150).

12. The foveated imaging system according to any one of claims 1-2, where additional polarizers are used in both the foveated imaging path (135) and the wide-FOV imaging path (125) to reduce the crosstalk between two paths.

13. The foveated imaging system according to any one of claims 1-2, where the foveated imaging lens is a group of rotationally symmetric lenses that magnify the foveated image.

14. The foveated imaging system according to any one of claims 1-2, where the foveated imaging lens may contain aspherical reflective or refractive surfaces.

15. The foveated imaging system according to any one of claims 1-2, where the wide field of view imaging lens is a group of rotationally symmetric lenses that magnify the wide field of view image.

16. The foveated imaging system according to any one of claims 1-2, where the wide field of view imaging lens may contain aspherical reflective or refractive surfaces.

17. The foveated imaging system according to any one of claims 1-16, where the system comprises at least two wide-FOV foveated imaging devices clustered together to capture a designated FOV larger than that of a single unit;

18. The system according to the claim 17, where the multiple wide-FOV imaging devices are placed at multiple viewpoints without FOV gap, such as to capture a continuous wide-FOV image.

19. The system according to the claim 17, where the multiple wide-FOV imaging devices are co-located at a common virtual viewpoint through a multi-facet mirror as if the wide-FOV image is captured from a single viewpoint.

20. The foveated imaging system according to any one of claims 1-2, where the scanning mirror is controlled by a scanning mirror controller (151) where the scanning mirror controller has an electronic interface that can communicate with a microprocessor.

21. The foveated imaging system according to any one of claims 1-2, combined with a method of controlling the target region of the foveated image, within the wide field of view image, where the method comprises the steps of:

a. receiving images from the wide field of view image sensor;

b. detecting a target region of interest;

c. generating commands to the fast scanning mirror control, corresponding to pointing the scanning mirror at the target of interest;

d. receiving images from the foveated image sensor;

e. characterizing the object in the foveated image.