Camera array including camera modules

• US 10,210,898 B2
• Filed: 03/02/2018
• Issued: 02/19/2019
• Est. Priority Date: 05/29/2014
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Jaunt Inc. (Verizon Communications Inc.)
• Inventors: van Hoff, Arthur, Annau, Thomas M., Christensen, Jens, Gardiner, Koji
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US15/910,932
• Publication Number:
  US 20180190321A1
• Time in Patent Office: 354 Days
• Field of Search: None
• CPC Class Codes: G06Q 30/0263 based upon Internet or website rating
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G11B 19/20 Driving; Starting; Stopping; Control thereof
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The disclosure includes a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens, a status indicator, and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in the other camera modules and the switch configured to instruct each of the camera modules to initiate a stop operation to stop recording, the status indicator configured to indicate a status of at least one of the camera modules. The camera modules of the camera array are configured to provide a 3X field of view overlap.

Claims

1. A camera system comprising:

a camera array comprising;

a set of camera modules, wherein each camera module in the set of camera modules includes a processor, a memory, a heat dissipation element, a sensor, and a lens, the sensor operable to execute a start operation to start recording raw video data using the lens and the sensor operable to execute a stop operation to stop recording;

wherein at least one pixel that is part of the raw video data is captured by at least three different camera modules in the set of camera modules thereby resulting in an at least 3X field of view overlap.

2. The camera system of claim 1, wherein:

the set of camera modules include equator camera modules; and

the equator camera modules are operable to provide the at least 3X field of overlap.

3. The camera system of claim 2, wherein the at least three different camera modules that provide at least 3X field of overlap are located side-by-side.

4. The camera system of claim 1, wherein the heat dissipation element forms a plurality of fins on a top and a bottom of one or more camera modules in the set of camera modules.

5. The camera system of claim 1, wherein the camera array further comprises a connection hub that is operable to receive and aggregate streams of the raw video data from each camera modules in the set of camera modules in the set of camera modules, wherein the raw video data describe image frames.

6. The camera system of claim 5, further comprising:

an aggregation system that is operable to receive the raw video data from the connection hub, stitch the image frames together based on a relative position of each camera module from the set of camera modules, and generate a stream of three-dimensional video.

7. The camera system of claim 6, wherein:

the aggregation system is further operable to receive audio data from the connection hub, synchronize the audio data, and generate virtual reality content that includes the stream of three-dimensional video and synchronized audio data.

8. The camera system of claim 1, wherein:

each camera module from the set of camera modules includes a device identifier and the raw video data includes the device identifier for a corresponding camera module that captured the raw video data.

9. The camera system of claim 8, further comprising:

an aggregation system that is operable to receive the raw video data from a connection hub, identify camera modules from the set of camera modules that recorded the raw video data based on device identifiers, stitch the image frames together based on the device identifiers and a relative position of each camera module from the set of camera modules, and generate a stream of three-dimensional video.

19. The camera system of claim 1, wherein the camera array includes a fault detection module that prevents the set of camera modules from capturing the image frames if two or more camera modules that are next to each other are identified as malfunctioning.

20. The camera system of claim 1, wherein:

compressed three-dimensional video is generated from the raw video data; and

compressed three-dimensional is generated by synthesizing audio data based on timing associated with three-dimensional audio data.

10. A computer-implemented method comprising:

receiving raw video data from a camera array that includes a set of camera modules, wherein the raw video data describes image frames captured by the set of camera modules;

identifying a device identifier and a relative position of each camera module in the set of camera modules;

stitching the image frames together based on the device identifier and the relative position of each camera module;

generating a stream of compressed three-dimensional video based on stitched image frames;

generating a stream of compressed three-dimensional audio by synthesizing audio data based on timing associated with three-dimensional audio data; and

generating virtual reality content that combines the compressed three-dimensional video and the compressed three-dimensional audio.

11. The method of claim 10, wherein at least some of the image frames include pixels that were captured by at least three camera modules in the set of camera modules.

12. The method of claim 10, wherein the raw video data is received from a connection hub that is associated with the camera array.

13. The method of claim 10, further comprising providing a user interface to a user device that includes options for starting and stopping capturing of the raw video data.

14. The method of claim 10, wherein the compressed three-dimensional audio data changes based on a head position of a user that views the virtual reality content.

15. The method of claim 10, further comprising detecting and correcting exposure or color deficiencies in the stitched image frames.

16. The method of claim 10, further comprising generating a user interface to be displayed on a client device, wherein the user interface includes an option for a user to specify when the set of camera modules will start capturing image frames.

17. The method of claim 10, wherein at least one pixel that is part of the raw video data is captured by at least three different camera modules in the set of camera modules thereby resulting in an at least 3X field of view overlap.

18. The method of claim 17, further comprising:

selecting candidate image frames for stitching based on the image frames that would result in a least amount of stitching artifacts or stitching aberrations.