Camera array including camera modules

• US 9,451,162 B2
• Filed: 07/28/2014
• Issued: 09/20/2016
• Est. Priority Date: 08/21/2013
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Jaunt Inc. (Verizon Communications Inc.)
• Inventors: Van Hoff, Arthur, Annau, Thomas M., Christensen, Jens, Gardiner, Koji
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): SMITH, STEPHEN R
• Application Number:
  US14/444,938
• Publication Number:
  US 20150055929A1
• Time in Patent Office: 785 Days
• Field of Search: 386/201, 386/223, 386/341, 386/358, 348 36- 39, 348 42- 60, 348/218.1, 348/153, 348/159, 348/211.11
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0246 Traffic
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0263 based upon Internet or website rating
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06Q 30/0284 Time or distance, e.g. usage of parking meters or taximeters
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: G11B 31/006 with video camera or receiver
• CPC Class Codes: H04N 13/106 : Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/111 : Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/117 : the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/189 : Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/282 : for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

The disclosure includes a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens, a status indicator, and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in the other camera modules and the switch configured to instruct each of the camera modules to initiate a stop operation to stop recording, the status indicator configured to indicate a status of at least one of the camera modules.

Claims

1. A camera system comprising:

a camera array comprising;

a housing forming one or more slits through which heat from inside the camera array dissipates, a first aperture, and a second aperture;

a first camera module including a first lens and a first sensor, wherein the first camera module is removably coupled to the first aperture of the housing and rotationally symmetrical such that the first camera module is configured to be removed from the camera array housing, rotated substantially 90 degrees, and reinserted into the housing;

a second camera module including a second lens and a second sensor, wherein the second camera module is removably coupled to the second aperture of the housing and rotationally symmetrical such that the second camera module is configured to be removed from the camera array housing, rotated substantially 90 degrees, and reinserted into the housing; and

a master camera that includes a processor, a memory, a status indicator, and a switch, the switch configured to instruct the first camera module and the second camera module to each initiate a start operation to start recording video data using the first lens and the first sensor, and the second lens and the second sensor, respectively, and the switch configured to instruct the first camera module and the second camera module to each initiate a stop operation to stop recording, the status indicator configured to indicate a status of at least one of the first camera module, the second camera module, and the camera array; and

a microphone array mounted to a top of the camera array, the microphone array configured to capture audio for enabling reconstruction of sound from any arbitrary direction.

2. The system of claim 1, wherein the camera array further comprises n camera modules, the n camera modules including the first camera module, the second camera module, and the master camera, and wherein the n camera modules are oriented to capture overlapping video data.

3. The system of claim 1, wherein the housing is formed in a shape of a honeycomb.

4. The system of claim 1, wherein the camera modules form a daisy chain with the master camera being coupled to the first camera module, the first camera module being coupled to the second camera module, and the second camera module being coupled to the master camera.

5. The system of claim 1, wherein the first camera module, the second camera module, and the master camera are each positioned to have at least one overlapping field of view with a corresponding one of the other camera modules.

6. The system of claim 1, wherein:

the status of the at least one of the first camera module, the second camera module, and the camera array includes a faulty status; and

the status indicator is configured to indicate the faulty status responsive to a fault occurring in the at least one of the first camera module, the second camera module, and the camera array.

7. The system of claim 1, wherein the status indicator is an overall status indicator configured to indicate a faulty status of a fault occurring in at least one of the first camera module and the second camera module and wherein the first camera module and the second camera module further include individual status indicators configured to indicate the faulty status of the fault occurring in the first camera module or the second camera module.

8. The system of claim 1, wherein the first sensor includes a depth sensor that is used to determine a depth map.

9. The system of claim 1, further comprising an aggregation system for generating a stream of three-dimensional video and audio data for displaying panoramic images.

10. The system of claim 9, further comprising a viewing system configured to decode and render the three-dimensional video and play the audio data on a virtual reality display and a surround sound system.

11. The system of claim 1, wherein the first camera module, the second camera module, and the master camera are synchronized through a daisy chain to capture corresponding video data in different directions simultaneously.

12. The system of claim 11, wherein the first camera module, the second camera module, and the master camera pass control and status messages to one another via the daisy chain.

13. The system of claim 1, further comprising a connection hub coupled to the camera modules and configured to transfer the video data from each of the first camera module, the second camera module, and the master camera to a client device.

14. The system of claim 13, wherein the connection hub includes a battery for supplying power to the first camera module, the second camera module, and the master camera.

15. The system of claim 1, wherein the first camera module, the second camera module, and the master camera each include an image signal processor for embedding a device identifier associated with a corresponding camera module into the video data.

17. The system of claim 1, wherein the first sensor includes at least one of a gyroscope that measures an orientation of the first camera module and an accelerometer that measures and acceleration of the first camera module.

18. The system of claim 1, wherein the first camera module, the second camera module, and the master camera each include a microphone configured to record audio.

19. The system of claim 1, wherein the camera array further comprises n camera modules, the n camera modules including the first camera module, the second camera module, and the master camera, and wherein the n camera modules are arranged on a horizontal axis and a vertical axis.

20. The system of claim 1, wherein the camera array further comprises n camera modules, the n camera modules including the first camera module, the second camera module, and the master camera, and wherein the n camera modules are configured as a sphere.

21. The system of claim 1, wherein the first camera module, the second camera module, and the master camera each form a substantially square shape.

22. The system of claim 1, wherein the first camera module, the second camera module, and the master camera are positioned along a horizontal axis.

16. A camera system comprising:

a camera array comprising;

a housing forming one or more slits through which heat from inside the camera array dissipates and apertures for camera modules; and

camera modules, the camera modules comprising a master camera that includes a processor, a first memory, a sensor, a lens, a visual status indicator, and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using a corresponding lens and a corresponding sensor in each of the camera modules and the switch configured to instruct the camera modules to initiate a stop operation to stop recording, the visual status indicator communicatively coupled to the master camera so that the visual status indicator is located physically on the master camera, the visual status indicator configured to provide a human-perceptible indication of a status of at least one of the camera modules, wherein the camera modules are rotationally symmetrical such that the camera modules are configured to be removed from the camera array housing, rotated substantially 90 degrees, and reinserted into the housing; and

a client device comprising;

one or more processors; and

a second memory configured to store instructions that are executed by the one or more processors, the instructions including operations configured to;

receive video data from one or more of the camera modules, the video data comprising image frames;

determine overlap between the image frames;

correct at least one of an exposure deficiency and a color deficiency based on the overlap;

stitch the image frames together based on the overlap; and

generate a three-dimensional video from stitched image frames.