Dense field imaging

• US 10,277,885 B1
• Filed: 10/03/2016
• Issued: 04/30/2019
• Est. Priority Date: 02/15/2013
• Status: Expired due to Fees

• Current Assignee: Red.com, LLC (Nikon Corporation)
• Original Assignee: Red.com, LLC (Nikon Corporation)
• Inventors: Jannard, James H., Mathur, Bimal, Postley, Howard John
• Primary Examiner(s): Beasley, Deirdre L
• Application Number:
  US15/283,731
• Time in Patent Office: 939 Days
• Field of Search: 348 36
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/70 SSIS architectures; Circuits associated therewith

Abstract

Dense field imagers are disclosed which are configured to provide combined, aggregated, fused, and/or stitched light field data for a scene. A dense field imager can include a plurality of imaging elements configured to be joined into image blocks or facets that each provides light field data about a scene. The dense field imager can include a plurality of facets in a fixed or modular fashion such that the dense field imager is configured to combine, aggregate, fuse and/or stitch light field data from the plurality of facets. The facets can be mounted such that one or more facets are non-coplanar with other facets. The facets can be configured to provide a representation of the light field with overlapping fields of view. Accordingly, the dense field imager can provide dense field data over a field of view covered by the plurality of facets.

Claims

1. An imaging system configured to capture digital images, comprising:

a first imaging block configured to obtain and output a first image data set representative of a first portion of a light field, wherein the first image data set represents the first portion of the light field as a function comprising at least four dimensions as inputs to represent luminance or radiance as a function of space and direction;

a second imaging block physically separate from the first imaging block and configured to obtain and output a second image data set representative of a second portion of the light field, wherein the second image data set represents the second portion of the light field as a function comprising at least four dimensions as inputs to represent luminance or radiance as a function of space and direction;

one or more processors operably coupled to the first and second imaging blocks;

one or more memory devices operably coupled to the one or more processors, the one or more processors and one or more memory devices configured to;

combine at least the first image data set and the second image data set to derive a third image data set based on at least a spatial relationship between the first and second portions of the light field; and

store the third image data set in the one or more memory devices.

2. An imaging system as in claim 1, wherein each respective imaging block comprises at least two imaging elements, wherein the first image data set is derived from pixel data acquired by the at least two imaging elements of the first imaging block and the second image data set is derived from pixel data acquired by the at least two imaging elements of the second imaging block, each imaging element comprising a sensor having an array of pixels and a lens.

3. An imaging system as in claim 2, wherein the at least two imaging elements in the first imaging block are coplanar, the at least two imaging elements in the second imaging block are coplanar, and the at least two imaging elements in the first imaging block are non-coplanar with respect to the at least two imaging elements in the second imaging block.

4. An imaging system as in claim 1, wherein the one or more processors and one or more memory devices are further configured to process the first image data set and the second image data set to determine the spatial relationship between the first portion of the light field and the second portion of the light field.

5. An imaging system as in claim 1, wherein the first portion of the light field and the second portion of the light field comprise regions of the light field which at least partially overlap, and wherein the third data set comprises light field information derived from data in both of the first and second data sets that corresponds to the region of the light field lying within the overlap.

6. An imaging system as in claim 1, wherein the first portion of the light field and the second portion of the light field comprise regions of the light field which only partially overlap, and wherein the third data set comprises light field information derived from:

data in the first image data set that corresponds to a first portion of a scene; and

data in the second image data set that corresponds to a second portion of the scene that does not overlap with the first portion.

7. A mobile device comprising the imaging system as in claim 1, wherein

the mobile device is a smartphone, cellular phone, PDA, tablet, or laptop.

21. An imaging system as in claim 1, wherein the one or more processors and one or more memory devices are further configured to combine the first image data set and the second image data set to generate an intermediate image data set that represents the light field as a function comprising a number of dimensions as inputs to represent luminance or radiance as a function of space and direction, wherein the number of dimensions is equal to the sum of the number of dimensions of the first and second image data sets.

22. An imaging system as in claim 21, wherein the one or more processors and one or more memory devices are further configured to derive the third image data set from the intermediate image data set, wherein the third image data set represents the light field as a function comprising a reduced number of dimensions as inputs to represent luminance or radiance as a function of space and direction, wherein the reduced number is at least equal to a number of dimensions of at least one of the first and second image data sets.

23. An imaging system as in claim 6, wherein the data in the first image data set that corresponds to the first portion of the scene is stitched together with the data in the second image data set that corresponds to the second portion of the scene that does not overlap with the first portion based on the spatial relationship between the first and second portions of the light field in the regions of the light field which only partially overlap.

8. A method of compiling an image data set, comprising:

obtaining a first image data set representative of a first portion of a light field with a first imaging block, wherein the first image data set represents the first portion of the light field as a function comprising at least four dimensions as inputs to represent luminance or radiance as a function of space and direction;

obtaining a second image data set representative of a second portion of the light field with a second imaging block that is physically separate from the first image block, wherein the second image data set represents the second portion of the light field as a function comprising at least four dimensions as inputs to represent luminance or radiance as a function of space and direction;

with one or more processors, combining at least the first image data set and the second image data set to derive a third image data set that is based on at least a spatial relationship between the first and second portions of the light field; and

storing the third image data set in one or more memory devices.

9. A method as in claim 8, wherein the first image data set is derived from pixel data acquired by a first group of at least two imaging elements and the second image data set is derived from pixel data acquired by a second group of at least two imaging elements, each imaging element comprising a sensor having an array of pixels and a lens.

10. A method as in claim 9, wherein the at least two imaging elements in the first group are coplanar, the at least two imaging elements in the second group are coplanar, and the at least two imaging elements in the first group are non-coplanar with respect to the at least two imaging elements in the second group.

11. A method as in claim 8, further comprising processing the first image data set and the second image data set to determine the spatial relationship between the first portion of the light field and the second portion of the light field.

12. A method as in claim 8, wherein the first portion of the light field and the second portion of the light field comprise regions of the light field which at least partially overlap, and wherein the third data set comprises light field information derived from data in both of the first and second data sets that corresponds to the region of the light field lying within the overlap.

13. A method as in claim 8, wherein the first portion of the light field and the second portion of the light field comprise regions of the light field which only partially overlap, and wherein the third data set comprises light field information derived from:

data in the first image data set that corresponds to a first portion of a scene; and

data in the second image data set that corresponds to a second portion of the scene that does not overlap with the first portion.

14. A method as in claim 8, wherein the third image data set comprises at least 4D light field information.

15. A method as in claim 8, wherein the first and second image data sets respectively represent the first and second portions of the light field as functions having at least four input parameters.

16. A method as in claim 8, wherein the first and second image data sets respectively represent the first and second portions of the light field as functions that represent luminance as a function of a position in space and a pointing direction.

17. A method as in claim 8, wherein viewable images are extractable from the third image data set, the viewable images comprising 2D images.

18. A method as in claim 8, wherein viewable images are extractable from the third image data set, the viewable images comprising 3D images.

19. A method as in claim 8, wherein viewable motion video is extractable from the third image data set.

20. An imaging system incorporated into a mobile device, the imaging system configured to execute the method as in claim 8, wherein the mobile device is a smartphone, cellular phone, PDA, tablet, or laptop.