METHOD, APPARATUS, AND MANUFACTURE FOR ENHANCED RESOLUTION FOR IMAGES FROM HIGH DYNAMIC RANGE (HDR) INTERLACED SENSORS

• US 20150002689A1
• Filed: 07/01/2013
• Published: 01/01/2015
• Est. Priority Date: 07/01/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm Technologies Incorporated (Qualcomm, Inc.)
• Inventors: Weissman, Assaf, Zobel, Pia
• Granted Patent: US 9,172,890 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A method, apparatus, and manufacture for generating an HDR image is provided. An original image is received from an HDR interlaced sensor that includes at least two fields captured with different exposures. The fields are separated from each other to provide separate images, and each of the separate images is upscaled. Next, blending is performed on each of the upscaled separate images to generate a high-dynamic range image, and ghost identification is performed on the high-dynamic range image. Subsequently, detail identification is performed on the high-dynamic range image. The detail identification includes identifying areas in the non-ghost areas of the high-dynamic range image that have details, and modifying the high-dynamic image by replacing each of the areas identified to have details with the corresponding area from the original image.

Claims

1. A method, comprising:

receiving an original image that includes lines, wherein the original image includes at least two fields, wherein each field of the at least two fields includes a subset of the lines in the original image such that the lines of each of the at least two fields are interlaced with each other, wherein each of the at least two fields was captured at a different exposure than each other field of the at least two fields, and wherein there is at least a partial overlap in capture time of each of the at least two fields with each other field of the at least two fields;

separating each of the at least two fields from each other to provide separate images;

upscaling each of the separate images;

blending the upscaled separate images to generate a high-dynamic range image;

performing ghost identification on the high-dynamic range image; and

performing detail identification on the high-dynamic range image, including identifying areas in the non-ghost areas of the high-dynamic range image that have details, and modifying the high-dynamic image by replacing each of the areas identified to have details with the corresponding area from the original image.

2. The method of claim 1, wherein the at least two fields include a first field that includes each of the odd lines in the original image, and a second field that includes each of the even lines in the original image.

3. The method of claim 1, further including employing a high dynamic range (HDR) sensor to capture the original image such that:

the original image includes lines, the original image includes at least two fields, each field of the at least two fields includes a subset of the lines in the original image such that the lines of each of the at least two fields are interlaced with each other, each of the at least two fields is captured at a different exposure than each other field of the at least two fields, and such that there is at least a partial overlap in capture time of each of the at least two fields with each other field of the at least two fields.

4. The method of claim 1, wherein identifying areas in the non-ghost areas of the high-dynamic range image that have details includes identifying areas in the non-ghost areas of the high-dynamic range image that have edges.

5. The method of claim 1, wherein

upscaling each of the separate images includes, for each of a plurality of pixel values to be determined, determining difference in pixel values from a plurality of different directions on each side of the pixels, and for each of the different directions, determining the direction at which the difference between pixels values in that direction is the least from among of the plurality of directions, and employing the average of the pixels in the determined direction for the pixel value to be determined.

6. The method of claim 5, wherein the plurality of directions include a vertical direction, a first diagonal direction, and a second diagonal direction that is different from the first diagonal direction.

7. The method of claim 1, wherein identifying areas in the non-ghost areas of the high-dynamic range image that have details is accomplished at least in part by identifying the non-ghost areas of the high-dynamic range image in which the difference in average luma values between close groups of pixels exceeds a pre-determined threshold.

8. The method of claim 7, wherein the size of each of the groups of pixels is pre-determined.

9. The method of claim 7, wherein identifying the non-ghost areas of the high-dynamic range image in which the difference in average luma values between close groups of pixels exceeds the pre-determined threshold further includes noise cleaning.

10. The method of claim 7, wherein identifying the non-ghost areas of the high-dynamic range image in which the difference in average luma values between close groups of pixels exceeds the pre-determined threshold is accomplished in at least two different directions.

11. An apparatus, comprising:

a memory that is configured to store code; and

at least one processor that is configured to execute the code to enable actions, including;

receiving an original image that includes lines, wherein the original image includes at least two fields, wherein each field of the at least two fields includes a subset of the lines in the original image such that the lines of each of the at least two fields are interlaced with each other, wherein each of the at least two fields was captured at a different exposure than each other field of the at least two fields, and wherein there is at least a partial overlap in capture time of each of the at least two fields with each other field of the at least two fields;

separating each of the at least two fields from each other to provide separate images;

upscaling each of the separate images;

blending the upscaled separate images to generate a high-dynamic range image;

performing ghost identification on the high-dynamic range image; and

performing detail identification on the high-dynamic range image, including identifying areas in the non-ghost areas of the high-dynamic range image that have details, and modifying the high-dynamic image by replacing each of the areas identified to have details with the corresponding area from the original image.

12. The apparatus of claim 11, further comprising a high-dynamic range (HDR) sensor that is arranged to capture the original image such that:

the original image includes lines, the original image includes at least two fields, each field of the at least two fields includes a subset of the lines in the original image such that the lines of each of the at least two fields are interlaced with each other, each of the at least two fields is captured at a different exposure than each other field of the at least two fields, and such that there is at least a partial overlap in capture time of each of the at least two fields with each other field of the at least two fields.

13. The apparatus of claim 11, wherein the at least one processor is further configured such that identifying areas in the non-ghost areas of the high-dynamic range image that have details includes identifying areas in the non-ghost areas of the high-dynamic range image that have edges.

14. The apparatus of claim 11, wherein the at least one processor is further configured such that identifying areas in the non-ghost areas of the high-dynamic range image that have details is accomplished at least in part by identifying the non-ghost areas of the high-dynamic range image in which the difference in average luma values between close groups of pixels exceeds a pre-determined threshold.

15. The apparatus of claim 14, wherein the at least one processor is further configured such that the size of each of the groups of pixels is pre-determined.

16. A tangible processor-readable storage medium that arranged to encode processor-readable code, which, when executed by one or more processors, enables actions, comprising:

receiving an original image that includes lines, wherein the original image includes at least two fields, wherein each field of the at least two fields includes a subset of the lines in the original image such that the lines of each of the at least two fields are interlaced with each other, wherein each of the at least two fields was captured at a different exposure than each other field of the at least two fields, and wherein there is at least a partial overlap in capture time of each of the at least two fields with each other field of the at least two fields;

separating each of the at least two fields from each other to provide separate images;

upscaling each of the separate images;

blending the upscaled separate images to generate a high-dynamic range image;

performing ghost identification on the high-dynamic range image; and

performing detail identification on the high-dynamic range image, including identifying areas in the non-ghost areas of the high-dynamic range image that have details, and modifying the high-dynamic image by replacing each of the areas identified to have details with the corresponding area from the original image.

17. The tangible processor-readable medium of claim 16, wherein identifying areas in the non-ghost areas of the high-dynamic range image that have details includes identifying areas in the non-ghost areas of the high-dynamic range image that have edges.

18. The tangible processor-readable storage medium of claim 16, wherein

upscaling each of the separate images includes, for each of a plurality of pixel values to be determined, determining difference in pixel values from a plurality of different directions on each side of the pixels, and for each of the different directions, determining the direction at which the difference between pixels values in that direction is the least from among of the plurality of directions, and employing the average of the pixels in the determined direction for the pixel value to be determined.

19. The tangible processor-readable storage medium of claim 16, wherein identifying areas in the non-ghost areas of the high-dynamic range image that have details is accomplished at least in part by identifying the non-ghost areas of the high-dynamic range image in which the difference in average luma values between close groups of pixels exceeds a pre-determined threshold.

20. The tangible processor-readable storage medium of claim 19, wherein the size of each of the groups of pixels is pre-determined.