Compensating for sensor saturation and microlens modulation during light-field image processing
First Claim
Patent Images
1. A method for adjusting pixel sensitivities in an image capture device, comprising:
- capturing a frame of light-field image data representing a scene, the light-field image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors;
for each color, determining a flat-field response contour for each of at least one region of an image sensor;
for each color, generating a modulation image based on the at least one flat-field response contour;
for each color, generating a demodulation image from the modulation image;
applying the generated demodulation images to the captured light-field image data to generate a demodulated light-field image;
applying a demosaicing operation to the demodulated light-field image;
applying an automatic white-balance correction algorithm to the demodulated light-field image;
estimating a chrominance of the scene illumination;
adjusting sensor gain individually for each color, based on the estimated chrominance;
capturing a subsequent frame of light-field image data, using the adjusted sensor gain for each of the colors; and
storing the captured subsequent frame.
3 Assignments
0 Petitions
Accused Products
Abstract
According to various embodiments, the system and method of the present invention process light-field image data so as to reduce color artifacts, reduce projection artifacts, and/or increase dynamic range. These techniques operate, for example, on image data affected by sensor saturation and/or microlens modulation. Flat-field images are captured and converted to modulation images, and then applied on a per-pixel basis, according to techniques described herein.
-
Citations
16 Claims
-
1. A method for adjusting pixel sensitivities in an image capture device, comprising:
-
capturing a frame of light-field image data representing a scene, the light-field image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors; for each color, determining a flat-field response contour for each of at least one region of an image sensor; for each color, generating a modulation image based on the at least one flat-field response contour; for each color, generating a demodulation image from the modulation image; applying the generated demodulation images to the captured light-field image data to generate a demodulated light-field image; applying a demosaicing operation to the demodulated light-field image; applying an automatic white-balance correction algorithm to the demodulated light-field image; estimating a chrominance of the scene illumination; adjusting sensor gain individually for each color, based on the estimated chrominance; capturing a subsequent frame of light-field image data, using the adjusted sensor gain for each of the colors; and storing the captured subsequent frame. - View Dependent Claims (2)
-
-
3. A method for adjusting pixel sensitivities in an image capture device, comprising:
-
in a processor, receiving light-field image data representing a scene, the light-field image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors; for each color, determining a flat-field response contour for each of at least one region of an image sensor; for each color, generating a modulation image based on the at least one flat-field response contour; and for each color, generating a demodulation image from the modulation image; applying the generated demodulation images to the received light-field image data to generate a demodulated light-field image; applying a demosaicing operation to the demodulated light-field image; applying an automatic white-balance correction algorithm to the demodulated light-field image; in a processor, estimating a chrominance of the scene illumination; in the processor, based on the estimated chrominance, determining maximum sensor values for each of the different colors; in the processor, clamping pixel values to corresponding maximum sensor values, to generate a processed image; and outputting the processed image on a display device. - View Dependent Claims (4)
-
-
5. A method for adjusting pixel sensitivities in an image capture device, comprising:
-
in a processor, receiving image data representing a scene, the image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors; in the processor, estimating a chrominance of the scene illumination; in the processor, based on the estimated chrominance, determining maximum sensor values for each of the different colors; in the processor, clamping pixel values to corresponding maximum sensor values, to generate a processed image; performing white-point adjustment on the processed image; and outputting the processed image on a display device.
-
-
6. A method for adjusting pixel sensitivities in an image capture device, comprising:
-
in a processor, receiving image data representing a scene, the image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors; in the processor, estimating a chrominance of the scene illumination; in the processor, based on the estimated chrominance, determining maximum sensor values for each of the different colors; in the processor, clamping pixel values to corresponding maximum sensor values, to generate a processed image; converting the processed image to a chrominance component and a luminance component; and outputting the converted image on a display device. - View Dependent Claims (7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable medium for adjusting pixel sensitivities in an image capture device, comprising instructions stored thereon, that when executed by a processor, perform the steps of:
-
receiving image data representing a scene, the image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors; for each color, determining a flat-field response contour for each of at least one region of an image sensor; for each color, generating a modulation image based on the at least one flat-field response contour; for each color, generating a demodulation image from the modulation image; applying the generated demodulation images to the received light-field image data to generate a demodulated light-field image; applying a demosaicing operation to the demodulated light-field image; applying an automatic white-balance correction algorithm to the demodulated light-field image; estimating a chrominance of the scene illumination; based on the estimated chrominance, determining maximum sensor values for each of the different colors; clamping pixel values to corresponding maximum sensor values, to generate a processed image; causing a display device to output the processed image.
-
-
16. A system for adjusting pixel sensitivities in an image capture device, comprising:
-
circuitry configured to perform the steps of; receiving image data representing a scene, the image data comprising a plurality of pixels, each pixel having a plurality of values associated with different colors; for each color, determining a flat-field response contour for each of at least one region of an image sensor; for each color, generating a modulation image based on the at least one flat-field response contour; for each color, generating a demodulation image from the modulation image; applying the generated demodulation images to the received light-field image data to generate a demodulated light-field image; applying a demosaicing operation to the demodulated light-field image; applying an automatic white-balance correction algorithm to the demodulated light-field image; estimating a chrominance of the scene illumination; based on the estimated chrominance, determining maximum sensor values for each of the different colors; and clamping pixel values to corresponding maximum sensor values, to generate a processed image; and a display device, communicatively coupled to the circuitry, configured to output the processed image.
-
Specification