Camera array including camera modules

• US 10,665,261 B2
• Filed: 01/03/2019
• Issued: 05/26/2020
• Est. Priority Date: 05/29/2014
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Van Hoff, Arthur, Annau, Thomas M., Christensen, Jens, Gardiner, Koji
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US16/239,525
• Publication Number:
  US 20190180783A1
• Time in Patent Office: 509 Days
• Field of Search: None
• CPC Class Codes: G06Q 30/0263 based upon Internet or website rating
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G11B 19/20 Driving; Starting; Stopping; Control thereof
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The disclosure includes a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens, a status indicator, and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in the other camera modules and the switch configured to instruct each of the camera modules to initiate a stop operation to stop recording, the status indicator configured to indicate a status of at least one of the camera modules. The camera modules of the camera array are configured to provide a 3× field of view overlap.

Claims

1. A camera system for capturing video data for use in generating augmented reality content, the camera system comprising:

a camera array comprising;

a housing; and

a set of camera modules, wherein each camera module in the set of camera modules includes a memory, a sensor, and a lens, the sensor operable to execute a start operation to start recording video data using the lens and the sensor operable to execute a stop operation to stop recording;

wherein at least one pixel that is part of the video data is captured by at least three different camera modules in the set of camera modules thereby resulting in an at least 3×

field of view overlap.

2. The camera system of claim 1, wherein each camera module further includes an external SD card reader, a lens mount, and a camera module housing body.

3. The camera system of claim 2, wherein one or more of the camera modules includes a microphone built into the camera module housing body.

4. The camera system of claim 1, wherein the set of camera modules includes first camera modules positioned around an equator of the camera array, second camera modules positioned above the equator of the camera array, and third camera modules positioned below the equator of the camera array.

5. The camera system of claim 4, wherein the second camera modules positioned above the equator of the camera array capture images having a landscape orientation.

6. The camera system of claim 4, wherein the second camera modules are positioned at a first pitch of positive sixty-five degrees relative to the first camera modules and the third camera modules are positioned at a second pitch of negative sixty-five degrees relative to the first camera modules.

7. The camera system of claim 1, wherein the at least three different camera modules are positioned on an equator of the camera array.

8. The camera system of claim 7, wherein the video data includes image frames and a timestamp for each of the image frames.

9. The camera system of claim 7, wherein the at least three different camera modules are positioned side-by-side.

10. The camera system of claim 1, wherein the housing forms a spherical honeycomb shape with apertures for each of the set of camera modules.

11. The camera system of claim 1, wherein the camera system further comprises an expansion port with hardware or software operable to enable the camera array to be communicatively coupled to at least one of a wired remote, a wireless dongle, and a touch panel display.

12. The camera system of claim 1, wherein one or more camera modules in the set of camera modules includes a heat dissipation element that forms a plurality of fins on a top and a bottom of the one or more camera modules.

13. A computer-implemented method for generating augmented reality content, the method comprising:

receiving video data from a camera array that includes a set of camera modules, wherein the video data describes image frames captured by the set of camera modules;

identifying a device identifier and a relative position of each camera module in the set of camera modules;

stitching the image frames together based on a timestamp associated with each of the image frames, the device identifier, and the relative position of each camera module;

generating a stream of three-dimensional video based on stitched image frames;

generating a stream of three-dimensional audio by synthesizing audio data based on timing associated with three-dimensional audio data; and

generating augmented reality content that combines the three-dimensional video and the three-dimensional audio.

14. The method of claim 13, wherein at least one pixel from the image frames is captured by at least three different camera modules.

15. The method of claim 14, wherein stitching the image frames together further includes selecting the image frames from candidate image frames based on the image frames resulting in a least amount of stitching artifacts, ghosting, or stitching aberrations.

16. The method of claim 14, wherein the at least three different camera modules have a field of view overlap of 2×

to 3×

.

17. The method of claim 13, wherein the image frames are stitched together further based on a frame sync signal in the video data.

18. The method of claim 13, wherein the image frames are stitched together to form two large-format panoramic video streams for left and right eye viewing.

19. The method of claim 13, further comprising:

instructing the camera array to start recording, wherein the set of camera modules in the camera array are configured in a daisy configuration.

20. The method of claim 13, further comprising:

instructing a master camera module of the set of camera modules to instruct slave camera modules of the set of camera modules to begin recording.