Enhancing digital photography

• US 8,570,389 B2
• Filed: 07/21/2005
• Issued: 10/29/2013
• Est. Priority Date: 07/22/2004
• Status: Expired due to Fees

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Sorek, Noam, Vitsnudel, Ilya
• Primary Examiner(s): FOSSELMAN, JOEL W
• Application Number:
  US11/187,355
• Publication Number:
  US 20060017837A1
• Time in Patent Office: 3,022 Days
• Field of Search: 348/239, 348/284, 348/234, 348/362, 348/222.1
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A method for enhancing imaging in low light conditions. The method comprises acquiring two or more images, long images relating to long exposure times and a short images relating to short exposure times; processing the images to separate zones with high image details from zones of low image details; and reconstructing a final image where image information is selected from the short image in zones with high image details and from the long image in zones with low image details.

Claims

1. A method for enhancing imaging in low light conditions, the method comprising:

acquiring a sequence of two or more images comprising at least a long image relating to long exposure time and a short image relating to short exposure time;

processing the sequence of two or more images by;

identifying at least one image zone of the sequence of two or more images having corresponding image levels between the long image and the short image upon gray level alignment;

filtering the short image and the long image to identify image edges in the short image and the long image generating a filtered short image and a filtered long image;

based on the identified image edges, masking the filtered short image and the filtered long image to identify high activity regions of the short image and the long image;

based on the masking, identifying high resolution areas of the filtered short image and generating an image combination mask; and

reconstructing a final image by combining the long image and the short image with the image combination mask.

2. The method according to claim 1, wherein the gray level alignment comprises estimating an offset and a ratio between the long image and the short image to align the sequence of two or more images.

3. The method according to claim 2, further comprising using the estimated offset and ratio to correct the short image to have proper gray level alignment with the long image.

4. The method according to claim 1, further comprising:

performing a geometrical registration process to align the short image and the long image; and

when the geometrical registration process indicates excessive motion common to the sequence of two or more images, discarding the long image and producing the final image from the filtered short image.

5. The method according to claim 4, wherein, when the geometrical registration process indicates a presence of global motion between the filtered short image and the filtered long image, masking comprises discarding the long image and producing the final image from the filtered short image at selected image margins of the short image.

6. The method according to claim 4, wherein performing the geometrical registration process comprises estimating a cross-correlation between the filtered short image and the filtered long image in a vertical direction and a horizontal direction.

7. The method according to claim 6, wherein the cross-correlation is performed along a plurality of lines among the filtered short image and the filtered long image.

8. The method according to claim 1, further comprising applying a smoothing filter to the image combination mask.

9. The method according to claim 8, wherein the smoothing filter is an infinite impulse response (IIR) filter.

10. The method according to claim 9, wherein the smoothing filter has a predefined pole.

11. An image processing device comprising:

a sensor;

a memory;

a processor in communication with the memory and the sensor, the processor being operable to;

acquire a sequence of two or more images comprising a plurality of image zones from the sensor, the sequence of two or more images comprising at least a long image relating to a long exposure time and a short image relating to a short exposure time;

identify at least one image zone of the sequence of two or more images having corresponding image levels between the long image and the short image upon gray level alignment;

filter the short image and the long image to identify image edges in the short image and the long image generating a filtered short image and a filtered long image;

based on the identified image edges, mask the filtered short image and the filtered long image to identify frequency content above a predetermined threshold;

based on the masking, identify high resolution areas of the filtered short image and generate an image combination mask;

compare the filtered short image and the filtered long image to identify regions of local motion; and

reconstruct a final image with the image combination mask, wherein image information is selected from the short image in said at least one identified image zones with frequency content above the predetermined threshold and from the long image in other zones.

12. The image processing device according to claim 11, wherein the gray level alignment comprises estimating an offset and ratio between the long image and the short image to align the sequence of two or more images.

13. The image processing device according to claim 12, further comprising using the estimated offset and ratio to correct the short image to have proper gray level alignment with the long image.

14. The image processing device according to claim 11, wherein the processor is further operable to:

perform a geometrical registration process to align the short image and the long image; and

when the geometrical registration process indicates excessive motion common to the sequence of two or more images, discard the long image and producing the final image from the filtered short image.

15. The image processing device according to claim 14, wherein, when the processor determines that the geometrical registration process indicates presence of global motion between the short image and the long image, the processor is further operable to discard the long image and produce the final image from the filtered short image.

16. The image processing device according to claim 14, further comprising a detector that detects motion of the image processing device relative to a subject of an image of the sequence of two or more images to detect the excessive motion common to the sequence of two or more images.

17. The image processing device according to claim 14, wherein the processor is operable to perform the geometrical registration process comprising estimating a cross-correlation between the short image and the long image in a vertical direction and a horizontal direction.

18. The image processing device according to claim 17, wherein the cross-correlation comprises, in the processor, cross-correlating along a plurality of lines among the short image and the long image.

19. The image processing device according to claim 11, wherein the image processing device comprises a camera.

20. The image processing device according to claim 11, further comprising a gyro configured to assess motion of the image processing device.