Systems and methods for transmitting and receiving array camera image data

• US 10,218,889 B2
• Filed: 01/08/2018
• Issued: 02/26/2019
• Est. Priority Date: 05/11/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: McMahon, Andrew Kenneth John
• Primary Examiner(s): Misleh, Justin P.
• Application Number:
  US15/864,429
• Publication Number:
  US 20180131852A1
• Time in Patent Office: 414 Days
• Field of Search: None
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules

Abstract

Systems and methods for transmitting and receiving image data captured by an imager array including a plurality of focal planes are described. One embodiment of the invention includes capturing image data using a plurality of active focal planes in a camera module, where an image is formed on each active focal plane by a separate lens stack, generating lines of image data by interleaving the image data captured by the plurality of active focal planes, and transmitting the lines of image data and the additional data.

Claims

1. A method of transmitting image data, comprising:

capturing image data using a first set of active cameras in an array of cameras;

generating a first line of image data by multiplexing at least a portion of the image data captured by the first set of active cameras using a predetermined process, wherein the predetermined process is selected from a plurality of predetermined processes for multiplexing captured image data;

generating a first set of additional data containing information identifying the cameras in the array of cameras that form the first set of active cameras and information indicating the predetermined process used to multiplex at least the portion of the image data;

transmitting the first set of additional data and the first line of image data;

capturing image data using a second set of active cameras in the array of cameras, wherein the second set of active cameras is different from the first set of active cameras;

generating a second line of image data by multiplexing at least a portion of the image data captured by the second set of active cameras;

generating a second set of additional data containing information identifying the cameras in the array of cameras that form the second set of active cameras; and

transmitting the second set of additional data and the second line of image data.

2. The method of claim 1, wherein:

each camera in the array of cameras comprises an array of pixels including a plurality of rows of pixels that also form a plurality of columns of pixels; and

the predetermined process selected from the plurality of predetermined processes for multiplexing captured image data comprises multiplexing image data from a row selected from each of the cameras in the first set of active cameras.

3. The method of claim 1, wherein the predetermined process selected from the plurality of predetermined processes for multiplexing captured image data comprises using a modulo process, where the modulo process involves multiplexing image data captured by pixels in a plurality of rows of the cameras in the first set of active cameras by multiplexing a predetermined number of pixels from each of the plurality of rows.

4. The method of claim 1, wherein individual cameras in the array of cameras are independently controlled.

5. The method of claim 1, wherein different cameras in the array of cameras have different imaging characteristics.

6. The method of claim 1, wherein the first set of active cameras comprises all of the cameras in the array of cameras.

7. The method of claim 1, wherein a number of active cameras in the first set of active cameras is different from a number of active cameras in the second set of active cameras.

8. The method of claim 1, wherein different sets of cameras are active at different times.

9. The method of claim 1, further comprising generating additional data that describes said array of cameras.

10. The method of claim 1, wherein the first set of additional data includes data that is selected from a group consisting of additional data describing:

a number of active cameras in the first set of active cameras;

dimensions of the array of cameras;

a number of pixels in each dimension of at least one of the cameras in the first set of active cameras;

an integration time for pixels in at least one of the cameras in the first set of active cameras; and

a gain for pixels in at least one of the active cameras in the first set of active cameras.

11. The method of claim 10, wherein the additional data describing the gain for pixels in the at least one of the cameras in the first set of active cameras includes additional data describing an analog gain for pixels in said at least one of the cameras in the first set of active cameras and additional data describing the digital gain for pixels in said at least one of the cameras in the first set of active cameras.

12. The method of claim 1, wherein the first set of additional data describes at least one of the first and second lines of image data.

13. The method of claim 1, wherein the first set of additional data includes data selected from the group consisting of additional data describing:

a time at which each of the cameras in the first set of active cameras started capturing the image data; and

row numbers of rows in each of the cameras in the first set of active cameras used to capture image data used to generate the first line of image data.

14. The method of claim 1, further comprising:

packetizing the first set of additional data and the first line of image data to produce at least one packet of data; and

wherein transmitting the first set of additional data and the first line of image data further comprises transmitting the at least one packet of data.

15. The method of claim 14, wherein:

the at least one packet of data comprises a packet header including at least a portion of the first set of additional data and at least a portion of the first line of image data.

16. The method of claim 15, wherein the at least one packet of data further comprises a packet footer indicating the end of the transmission of the first line of image data.

17. The method of claim 15, wherein the packet header further comprises additional data describing the array of cameras.

18. The method of claim 1, wherein a plurality of cameras in the array of cameras has a different field of view.

19. An array camera, comprising:

an array of cameras comprising a plurality of cameras that capture images of a scene from different viewpoints;

a processor; and

memory in communications with the processor;

wherein software directs the processor to;

capture image data using a first set of active cameras in the array of cameras;

generate a first line of image data by multiplexing at least a portion of the image data captured by the first set of active cameras using a predetermined process, wherein the predetermined process is selected from a plurality of predetermined processes for multiplexing captured image data;

generate a first set of additional data containing information identifying the cameras in the array of cameras that form the first set of active cameras and information indicating the predetermined process used to multiplex at least the portion of the image data;

transmit the first set of additional data and the first line of image data;

capture image data using a second set of active cameras in the array of cameras, wherein the second set of active cameras is different from the first set of active cameras;

generate a second line of image data by multiplexing at least a portion of the image data captured by the second set of active cameras;

generate a second set of additional data containing information identifying the cameras in the array of cameras that form the second set of active cameras;

transmit the second set of additional data and the second line of image data.