Camera array including camera modules

• US 10,694,167 B1
• Filed: 12/12/2018
• Issued: 06/23/2020
• Est. Priority Date: 12/12/2018
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Van Hoff, Arthur, Annau, Thomas M., Christensen, Jens, Gardiner, Koji
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US16/218,376
• Publication Number:
  US 20200195906A1
• Time in Patent Office: 559 Days
• CPC Class Codes: G02B 27/0025 for optical correction, e.g. distorsion, aberration
• CPC Class Codes: G06F 9/30003 Arrangements for executing specific machine instructions
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/65 Control of camera operation in relation to power supply
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

The disclosure includes a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens, a status indicator, and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in the other camera modules and the switch configured to instruct each of the camera modules to initiate a stop operation to stop recording, the status indicator configured to indicate a status of at least one of the camera modules. Lens distortion effects may be removed from the frames described by the video data. The camera modules of the camera array are configured to provide a 3× field of view overlap.

Claims

1. A camera system comprising:

a camera array comprising;

equator camera modules arranged around an equator of the camera array, wherein the equator camera modules capture a set of images;

wherein the equator camera modules are arranged around the equator of the camera array so that each pixel in each image included in the set of images is captured by at least three adjacent camera modules selected from the equator camera modules to provide at least a 3×

field of view overlap; and

an aggregation system comprising one or more processors and a non-transitory memory storing computer code which, when executed by the one or more processors causes the one or more processors to;

receive video data that describes the set of images from the camera array;

stitch the set of images together based on a relative position of each corresponding camera module to generate augmented reality video; and

correct lens distortion that occurs in the 3D video based on an inverse model that is used to map a 3D real-world point from an undistorted 2D image to a 3D image given a particular distance.

2. The system of claim 1, wherein the inverse model is used to determine a vector V and determine a length of the vector V by normalizing V.

3. The system of claim 1, wherein the inverse model is used to normalize vector V by dividing vector V by a Euclidian norm of vector V.

4. The system of claim 2, wherein the set of images are stitched together by identifying, for each image, the equator camera modules that captured the image and a time when the image was captured so that, for a given time frame, images are identified as having been captured at substantially the same time and the images are stitched together in an order corresponding to arrangement of the equator camera modules around the equator of the camera array so that stitched stereoscopic images form the three hundred and sixty degree view of an environment.

5. The system of claim 1, wherein the augmented reality video includes a stitched set of stereoscopic images that are synchronized for each frame based on synchronization of the equator camera modules.

6. The system of claim 1, wherein the equator camera modules form a daisy chain with a master camera being coupled to a first camera module that is coupled to an n camera module.

7. The system of claim 1 further comprising a tray to hold the equator camera modules from a bottom of the camera array.

8. The system of claim 1 further comprising a status indicator configured to indicate a status of at least one of the equator camera modules.

9. The system of claim 8, wherein:

the status of one of the equator camera modules includes a faulty status; and

the status indicator is configured to indicate the faulty status responsive to a fault occurring in one of the equator camera modules.

10. The system of claim 8, wherein the status indicator is an overall status indicator configured to indicate a faulty status of a fault occurring in any of the equator camera modules and wherein the equator camera modules further include individual status indicators configured to indicate the faulty status of the fault occurring in one of the equator camera modules, the below the equator camera modules, and the above the equator camera modules.