Enhancing digital photography

• US 20060017837A1
• Filed: 07/21/2005
• Published: 01/26/2006
• Est. Priority Date: 07/22/2004
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Sightic Vista Limited
• Inventors: Sorek, Noam, Vitsnudel, Ilya
• Granted Patent: US 8,570,389 B2
• Time in Patent Office: Days
• US Class Current: 348/362
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A method for enhancing imaging in low light conditions. The method comprises acquiring two or more images, long images relating to long exposure times and a short images relating to short exposure times; processing the images to separate zones with high image details from zones of low image details; and reconstructing a final image where image information is selected from the short image in zones with high image details and from the long image in zones with low image details.

Claims

1. A method for enhancing imaging in low light conditions, the method comprising:

acquiring a sequence of two or more images, at least a long image relating to long exposure time and a short image relating to short exposure time;

processing the sequence of two or more images to separate zones with high image details from zones of low image details; and

reconstructing a final image where image information is selected from the short image in zones with high image details and from the long image in zones with low image details.

2. The method of claim 1, comprising:

aligning the sequence of two or more images;

creating a combination mask that determines weights attributed locally to the long and short image information for reconstruction of the final image;

reconstructing the final image using the combination mask.

3. The method of claim 2, wherein the aligning comprises aligning the image levels of the sequence of two or more images and/or aligning geometrically the sequence of two or more images. And/or aligning the PSF of the images.

4. The method of claim 2, wherein the aligning is done based on identifying edges and aligning the images using partial edge information.

5. The method of claim 2, wherein the aligning is done using motion vectors extracted from an MPEG compression engine.

6. The method of claim 2, wherein the combination mask is created by finding edge zones and smoothing the edge zones.

7. The method of claim 6, wherein the combination mask further comprises information relating to smoothed local motion, and/or information on inconsistency between the images as from changing light sources, reflections, indication how much short or long image information ought to be added to zones of low image details, and information relating to misalignment between the images.

8. The method of claim 1, wherein the long image is simulated from the short image.

9. The method of claim 1, further comprising:

evaluating the images before processing to determine if any of them can be used solely as a final image, and if so providing that image as a final image.

10. The method of claim 1, wherein the images are acquired by taking a first image and sampling partial field information from the imaging sensor followed by taking the reset of the sequence of two or more images and sampling partial or full field information from the imaging sensor.

11. The method of claim 10, wherein the first image is the long image.

12. The method of claim 10, wherein the first image is the short image.

13. The method of claim 1, wherein processing of the images is carried out on raw image data.

14. The method of claim 1, wherein processing of the images is carried out on YUV image data.

15. The method of claim 1, wherein processing of the images is carried out in JPEG domain.

16. The method of claim 15, wherein color information for the final image is taken only from the long image.

17. The method of claim 1, wherein the final image is compressed in JPEG domain, and wherein the JPEG compression is adapted to comply with the processing of the images.

18. The method of claim 1, wherein during the acquisition of the images a strobe light is used with different or similar light energy for each image.

19. The method of claim 18, further comprising applying red eye reduction.

20. The method of claim 19, wherein the red eye reduction comprises identifying red eye pixels by local analysis of the consistency of the images taken with different strobe durations, and utilizing the mask to deselect red eye areas from the images where they appear in.

21. The method of claim 1, wherein in the processing of the images a point spread function (PSF) is evaluated for each image.

22. The method of claim 21, wherein the PSF is evaluated using a gyro device.

23. The method of claim 21 wherein the PSF of each image is used for reconstructing each image, before the processing step of selecting regions from each image.

24. The method of claim 21 wherein the PSF of all the images together with all the image data is used for a simultaneously reconstruction of the result image.

25. A camera comprising:

a lens;

imaging sensor on which the sensor may be focuses;

memory for storing image information;

controller for initiating acquiring of a set of two or more images, at least a long image relating to long exposure time and a short image relating to short exposure time;

processor for processing the sequence of two or more images to separate zones with high image details from zones of low image details; and

reconstructing a final image where image information is selected from the short image in zones with high image details and from the long image in zones with low image details.

26. The camera of claim 25, wherein the processor is adapted to perform:

aligning the sequence of two or more images;

creating a combination mask that determines weights attributed locally to the long and short image information for reconstruction of the final image;

reconstructing the final image using the combination mask.

27. The camera of claim 26, wherein the aligning comprises aligning the image levels of the sequence of two or more images and/or aligning geometrically the sequence of two or more images and/or aligning the PSF of the images.

28. The camera of claim 26, wherein the aligning is done based on identifying edges and aligning the images using partial edge information.

29. The camera of claim 26, wherein the aligning is done using motion vectors extracted from an MPEG compression engine.

30. The camera of claim 26, wherein the combination mask is created by finding edge zones and smoothing the edge zones.

31. The camera of claim 30, wherein the combination mask further comprises information relating to smoothed local motion, and/or information on inconsistency between the images as from changing light sources, reflections, indication how much short or long image information ought to be added to zones of low image details, and information relating to misalignment between the images.

32. The camera of claim 25, wherein the long image is simulated from the short image.

33. The camera of claim 25, wherein the processor further performs:

evaluating the images before processing to determine if any of them can be used solely as a final image, and if so providing that image as a final image.

34. The camera of claim 25, wherein the images are acquired by taking a first image and sampling partial field information from the imaging sensor followed by taking the reset of the sequence of two or more images and sampling partial or full field information from the imaging sensor.

35. The camera of claim 25, wherein processing of the images is carried out on raw image data.

36. The camera of claim 25, wherein processing of the images is carried out on YUV image data.

37. The camera of claim 25, wherein processing of the images is carried out in JPEG domain.

38. The camera of claim 37, wherein color information for the final image is taken only from the long image.

39. The camera of claim 25, wherein the final image is compressed in JPEG domain, and wherein the JPEG compression is adapted to comply with the processing of the images.

40. The camera of claim 25, further comprising a strobe light, wherein during the acquisition of the images a strobe light is used with different or similar light energy for each image.

41. The camera of claim 40, wherein the processor further applies red eye reduction.

42. The camera of claim 41, wherein the red eye reduction comprises identifying red eye pixels by local analysis of the consistency of the images taken with different strobe durations, and utilizing the mask to deselect red eye areas from the images where they appear in.

43. The camera of claim 25, wherein in the processing of the images a point spread function (PSF) is evaluated for each image.

44. The camera of claim 43, wherein the PSF is evaluated using a gyro device.

45. The camera of claim 43 wherein the PSF of each image is used for reconstructing each image, before the processing step of selecting regions from each image.

46. The camera of claim 43 wherein the PSF of all the images together with all the image data is used for a simultaneously reconstruction of the result image.