METHODS AND APPARATUS FOR ENVIRONMENTAL MEASUREMENTS AND/OR STEREOSCOPIC IMAGE CAPTURE

• US 20160212409A1
• Filed: 11/25/2015
• Published: 07/21/2016
• Est. Priority Date: 01/21/2015
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Medina, Hector M., Cole, David, Moss, Alan McKay
• Granted Patent: US 10,531,071 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/232 using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details
• CPC Class Codes: H04N 23/50 Constructional details

Abstract

A camera rig including one or more stereoscopic camera pairs and/or one or more light field cameras are described. Images are captured by the light field cameras and stereoscopic camera pairs are captured at the same time. The light field images are used to generate an environmental depth map which accurately reflects the environment in which the stereoscopic images are captured at the time of image capture. In addition to providing depth information, images captured by the light field camera or cameras is combined with or used in place of stereoscopic image data to allow viewing and/or display of portions of a scene not captured by a stereoscopic camera pair.

Claims

1. An image capture system, comprising:

a first light field camera facing in a first direction; and

a first stereoscopic camera pair including at least a first camera and a second camera for capturing left and right views, respectively.

2. The system of claim 1,

wherein said first light field camera includes a fish eye lens;

wherein said first camera includes a fisheye lens; and

wherein said second camera includes a fish eye lens.

3. The system of claim 1, further comprising:

a second light field camera facing in a second direction; and

a second stereoscopic camera pair including at least a third camera and a fourth camera.

4. The system of claim 3,

wherein said second light field camera includes a fish eye lens;

wherein said third camera includes a fisheye lens; and

wherein said fourth camera includes a fish eye lens.

5. The system of claim 3, further comprising:

a third light field camera facing in a third direction; and

a third stereoscopic camera pair including at least a fifth camera and a sixth camera.

6. The system of claim 5, further comprising:

a fourth light field camera facing in a fourth direction; and

a fourth stereoscopic camera pair including at least a seventh camera and an eighth camera.

7. The system of claim 6, further comprising:

a ninth camera facing in an upward vertical direction; and

wherein the first, second third and fourth directions are generally horizontal directions.

8. The system of claim 7, further comprising:

a tenth camera facing in a downward vertical direction opposite the direction in which the ninth camera is pointed; and

wherein the first, second, third, fourth, sixth, seventh, ninth and tenth cameras are equidistant from one another.

9. The system of claim 1, further comprising:

a second light field camera facing in a second direction; and

a third light field camera facing in a third direction, said first, second and third directions being different.

10. The system of claim 9, further comprising:

a fourth light field camera facing in a fourth direction.

11. The system of claim 6, wherein said first, second, third and fourth light field cameras each include a processor for generating a depth map from an image captured by the light field camera in which the processor is included.

12. The system of claim 11, further comprising:

a processor configured to receive the depth maps corresponding to portions of the environment and combine the depth maps to generate an environmental depth map.

13. The system of claim 11 further comprising:

memory for storing images captured by said first, second and third, light field cameras, said environmental depth map and images captured from said first, second and third stereoscopic camera pairs, at least some of the images captured by the light field camera having been captured while a stereoscopic camera was capturing some of said stored images captured by the stereoscopic camera pairs.

14. An image capture and processing method, comprising:

capturing images using a first light field camera facing in a first direction; and

capturing left and right eye images using a first stereoscopic camera pair including at least a first camera and a second camera for capturing left and right views, respectively.

15. The method of claim 14, further comprising:

capturing images using a second light field camera facing in a second direction;

capturing left and right eye images using a second stereoscopic camera pair including at least a third camera and a fourth camera for capturing left and right views, respectively.

16. The method of claim 15, further comprising:

capturing images using a third light field camera facing in a third direction; and

capturing left and right eye images using a third stereoscopic camera pair including at least a fifth camera and a sixth camera.

17. The method of claim 16, further comprising:

generating an environmental depth map from images captured by said first, second, and third light field cameras.

18. The method of claim 16, further comprising:

capturing images using a fourth light field camera facing in a fourth direction;

capturing images using a fourth stereoscopic camera pair including at least a seventh camera and an eighth camera; and

generating an environmental depth map from images captured by said first, second, third and fourth light field cameras.

19. The method of claim 16, further comprising:

generating stereoscopic image data from a light field image captured by one of said first, second and third light field cameras, said stereoscopic image data including image data corresponding to a portion of the environment which was not captured by a camera of a stereoscopic camera pair.