Camera array removing lens distortion
First Claim
1. A camera system comprising:
- a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in each of the camera modules and the switch configured to instruct the camera modules to initiate a stop operation to stop recording; and
an aggregation system communicatively coupled to the camera array to receive the video data from the camera array, the aggregation system including code and routines stored on a non-transitory memory and configured to provide lens distortion correction for the video data when executed by the processor based on a forward model to map a three-dimensional real-world point onto an undistorted two-dimensional image by capturing the three-dimensional real-world point (V), determining a length of V by calculating a Euclidean norm, determining a normalized three-dimensional point (VN) by normalizing V, and determining an angle of VN;
wherein the camera modules of the camera array are configured to provide at least a 3×
field of view overlap; and
wherein the camera array comprises equator camera modules that are arranged around an equator of the camera array so that the equator camera modules are adjacent to one another and configured to capture images having a same orientation relative to one another, wherein the same orientation is a portrait orientation or a landscape orientation of an optical sensor.
2 Assignments
0 Petitions
Accused Products
Abstract
The disclosure includes a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens, a status indicator, and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in the other camera modules and the switch configured to instruct each of the camera modules to initiate a stop operation to stop recording, the status indicator configured to indicate a status of at least one of the camera modules. Lens distortion effects may be removed from the frames described by the video data. The camera modules of the camera array are configured to provide a 3× field of view overlap.
-
Citations
21 Claims
-
1. A camera system comprising:
-
a camera array comprising camera modules, the camera modules comprising a master camera that includes a processor, a memory, a sensor, a lens and a switch, the switch configured to instruct each of the camera modules to initiate a start operation to start recording video data using the lens and the sensor in each of the camera modules and the switch configured to instruct the camera modules to initiate a stop operation to stop recording; and an aggregation system communicatively coupled to the camera array to receive the video data from the camera array, the aggregation system including code and routines stored on a non-transitory memory and configured to provide lens distortion correction for the video data when executed by the processor based on a forward model to map a three-dimensional real-world point onto an undistorted two-dimensional image by capturing the three-dimensional real-world point (V), determining a length of V by calculating a Euclidean norm, determining a normalized three-dimensional point (VN) by normalizing V, and determining an angle of VN; wherein the camera modules of the camera array are configured to provide at least a 3×
field of view overlap; andwherein the camera array comprises equator camera modules that are arranged around an equator of the camera array so that the equator camera modules are adjacent to one another and configured to capture images having a same orientation relative to one another, wherein the same orientation is a portrait orientation or a landscape orientation of an optical sensor. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A computer-implemented method comprising:
-
identifying, with one or more processors, a device identifier and a position of each camera module in a camera array, the camera modules including a master camera and the camera modules configured to provide at least a 3×
field of view overlap wherein the camera array comprises at least sixteen camera modules that are arranged around an equator of the camera array so that the at least sixteen camera modules are adjacent to one another and configured to capture images having a same orientation relative to one another, wherein the same orientation is a portrait orientation or a landscape orientation of an optical sensor;confirming an absence of faults in the camera module; initiating a start operation in the master camera, the master camera instructing the other camera modules to start recording; receiving video data comprising image frames from the camera modules; removing one or more lens distortion effects present in the image frames based on a forward model to map a three-dimensional real-world point onto an undistorted two-dimensional image by determining an angle of a normalized three-dimensional point, determining an undistorted distance to an optical center of a lens of one of the camera modules in the camera array, and determining (x, y) coordinates of a two-dimensional point of the undistorted two-dimensional image; stitching the image frames together based on the video data; generating three-dimensional video; synthesizing audio data; and generating a stream of the three-dimensional video and the audio data for displaying panoramic images. - View Dependent Claims (20)
-
-
21. A special purpose computing device comprising a non-transitory computer-usable medium including a computer-readable program, wherein the computer-readable program when executed on the special purpose computing device causes the special purpose computing device to:
-
identify a device identifier and a position of each camera module in a camera array, the camera modules including a master camera and the camera modules configured to provide at least a 3×
field of view overlap of the camera modules wherein the camera array comprises at least sixteen camera modules that are arranged around an equator of the camera array so that the at least sixteen camera modules are adjacent to one another and configured to capture images having a same orientation relative to one another, wherein the same orientation is a portrait orientation or a landscape orientation of an optical sensor;confirm an absence of faults in the camera modules; initiate a start operation in the master camera, the master camera instructing the other camera modules to start recording; receive video data comprising image frames from the camera modules; remove one or more lens distortion effects from the image frames based on a forward model to map a three-dimensional real-world point onto an undistorted two-dimensional image by determining a length of the three-dimensional real-world point, determining a normalized three-dimensional point, determining an angle of the normalized three-dimensional point, determining an undistorted distance to an optical center of a lens of one of the camera modules in the camera array, and determining (x, y) coordinates of a two-dimensional point of the undistorted two-dimensional image; stitch the image frames together based on the video data, wherein the image frames are stitched together based on a relative position of each camera module and each pixel of the image has three candidate sets of video data to select from when stitching the image frames based on the at least 3×
field of view overlap;generate three-dimensional video; synthesize audio data; and generate a stream of the three-dimensional video and the audio data for displaying panoramic images.
-
Specification