Systems and methods for generating a digital image

• US 9,998,721 B2
• Filed: 12/08/2017
• Issued: 06/12/2018
• Est. Priority Date: 05/01/2015
• Status: Active Grant

• Current Assignee: Duelight LLC
• Original Assignee: Duelight LLC
• Inventors: Rivard, William, Feder, Adam, Kindle, Brian
• Primary Examiner(s): Jerabek, Kelly L
• Application Number:
  US15/836,655
• Publication Number:
  US 20180109771A1
• Time in Patent Office: 186 Days
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 9/77 Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

A system, method, and computer program product for generating a digital image is disclosed. In use, a first image set is captured by a first image sensor, the first image set including two or more first source images and a plurality of chrominance values, and a second image set is captured by a second image sensor, the second image set including two or more second source images and a plurality of luminance values. Next, a first image of the first source images and a second image of the first source images are combined to form a first pair of source images, and a first image of the second source images and a second image of the second source images are combined to form a second pair of source images. Additionally, a first resulting image by is generated combining the first pair of source images with the second pair of source images. Additional systems, methods, and computer program products are also presented.

Claims

1. A device, comprising:

a non-transitory memory storing instructions; and

one or more processors in communication with the non-transitory memory, wherein the one or more processors execute the instructions to;

cause a first image sensor to capture a first image set, the first image set including two or more first source images and a plurality of chrominance values;

cause a second image sensor to capture a second image set, the second image set including two or more second source images and a plurality of luminance values, wherein the second image sensor has a higher resolution than the first image sensor;

combine a first image of the first source images and a second image of the first source images to form a first pair of source images;

combine a first image of the second source images and a second image of the second source images to form a second pair of source images; and

generate a first resulting image by combining the first pair of source images with the second pair of source images.

2. The device of claim 1, wherein the device is configured such that the first image sensor is configured to capture chrominance image samples.

3. The device of claim 1, wherein the device is configured such that the second image sensor is configured to capture luminance image samples.

4. The device of claim 1, wherein the one or more processors further execute the instructions to generate a second resulting image by combining a third pair of source images with a fourth pair of source images, the third pair formed by combining a third image of the first source images and a fourth image of the first source images, and the fourth pair formed by combining a third image of the second source images and a fourth image of the second source images.

5. The device of claim 4, wherein the device is configured such that each of the first pair of source images, the second pair of source images, the third pair of source images, and the fourth pair of source images includes a selection parameter.

6. The device of claim 5, wherein the device is configured such that the selection parameter includes a weight assigned to each of the first pair of source images, the second pair of source images, the third pair of source images, and the fourth pair of source images, wherein the weight is used to perform a blend between two of the first pair of source images, the second pair of source images, the third pair of source images, and the fourth pair of source images.

7. The device of claim 1, wherein the device is configured such that the first image set and the second image set are sampled under ambient illumination, strobe illumination, or a combination of ambient illumination and strobe illumination.

8. The device of claim 1, wherein the device is configured such that the first resulting image is generated by:

combining the first pair of source images to generate a high dynamic range (HDR) chrominance image using an HDR image blending technique;

combining the second pair of source images to generate an HDR luminance image using an HDR image blending technique; and

combining the HDR chrominance image and the HDR luminance image.

9. The device of claim 8, wherein the one or more processors further execute the instructions to mutually align the first pair of source images to the second pair of source images.

10. The device of claim 8, wherein the HDR chrominance image includes a hue or a saturation component per pixel.

11. The device of claim 1, wherein a first lens is configured to focus light from a photographic scene onto the first image sensor and a second lens is configured to focus light from the photographic scene onto the second image sensor.

12. The device of claim 1, wherein the device is configured such that the first image sensor and the second image sensor are included in a wireless mobile device or a digital camera.

13. The device of claim 1, further comprising a lens that focuses light from a photographic scene onto a beam splitter that is configured to split the light from the photographic scene onto the first image sensor and the second image sensor.

14. The device of claim 1, wherein the device is configured such that the first image sensor and the second image sensor sample images utilizing a global shutter operation.

15. The device of claim 1, wherein the device is configured such that the first image sensor and the second image sensor sample images utilizing a rolling shutter operation.

16. The device of claim 1, further comprising a strobe unit, and wherein the first image set and the second image set are sampled under strobe illumination.

17. The device of claim 16, wherein the one or more processors further execute the instructions to capture one of the first image set or the second image set under ambient illumination, analyze the first image set to determine a white balance setting, and enable the strobe unit to provide strobe illumination of a particular color based on the white balance setting.

18. A method, comprising:

causing, by a processor, a first image sensor to capture a first image set, the first image set including two or more first source images and a plurality of chrominance values;

causing, by the processor, a second image sensor to capture a second image set, the second image set including two or more second source images and a plurality of luminance values, wherein the second image sensor has a higher resolution than the first image sensor;

combining, using the processor, a first image of the first source images and a second image of the first source images to form a first pair of source images;

combining, using the processor, a first image of the second source images and a second image of the second source images to form a second pair of source images; and

generating, using the processor, a first resulting image by combining the first pair of source images with the second pair of source images.

19. A computer program product comprising computer executable instructions stored on a non-transitory computer readable medium that when executed by a processor instruct the processor to:

cause a first image sensor to capture a first image set, the first image set including two or more first source images and a plurality of chrominance values;

cause a second image sensor to capture a second image set, the second image set including two or more second source images and a plurality of luminance values, wherein the second image sensor has a higher resolution than the first image sensor;

combine a first image of the first source images and a second image of the first source images to form a first pair of source images;

combine a first image of the second source images and a second image of the second source images to form a second pair of source images; and

generate a first resulting image by combining the first pair of source images with the second pair of source images.