Method and apparatus for initiating subsequent exposures based on determination of motion blurring artifacts

• US 8,244,053 B2
• Filed: 12/16/2009
• Issued: 08/14/2012
• Est. Priority Date: 11/10/2004
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter, Prilutsky, Yury
• Primary Examiner(s): Bhatnagar, Anand
• Assistant Examiner(s): Bloom, Nathan
• Application Number:
  US12/639,727
• Publication Number:
  US 20100201827A1
• Time in Patent Office: 972 Days
• Field of Search: 382/107, 382/164, 382/176, 382/181, 382/190, 382/197, 382/276, 382282-284, 348/208.99, 348/208.1, 348/208.4, 348/208.14, 348/208.16
• US Class Current: 382/255
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction

Abstract

A digital image acquisition system includes a portable apparatus for capturing digital images and a digital processing component for detecting, analyzing, invoking subsequent image captures and informing the photographer regarding motion blur, and for reducing camera motion blur in an image captured by the apparatus. The digital processing component operates by comparing the image with at least one other image, for example a preview image, of nominally the same scene taken outside the exposure period of the main image. In one embodiment the digital processing component identifies at least one feature in a single preview image which is relatively less blurred than the corresponding feature in the main image, calculates a point spread function (PSF) in respect of such feature, and initiates a subsequent capture if determined that the motion blur exceeds a certain threshold. In another embodiment the digital processing determines the degree of blur by analyzing the motion blur in the captured image itself, and initiates a subsequent capture if determined that the motion blur exceeds a certain threshold. Such real time analysis may use the auto focusing mechanism to qualitatively determine the PSF.

Claims

1. A digital image acquisition system, comprising:

an apparatus for capturing digital images; and

a digital processing component for determining a camera motion blur function in a captured digital image based on a comparison of at least two sets of image data, including at least one relatively low resolution reference image and a captured digital image, each acquired with a same sensor, such that at least each pixel of a same portion of said same sensor captures at least a pixel of each of said at least two sets of image data,

wherein said apparatus is configured to capture each said at least one relatively low resolution reference image within a temporal range that includes times proximately before or after said exposure period, and such that the captured digital image and each said at least one relatively low resolution reference image comprise approximately the same scene, and

wherein said digital processing component is arranged to;

align at least portions of the at least two sets of image data; and

compare the at least two sets of image data for determining said camera motion blur function in said captured digital image including calculating a point spread function (PSF) to points within an object captured within the captured digital image, and

determine to de-blur said captured digital image when the camera motion blur function indicates that sufficient blur exists within the captured digital image, and

deblur the captured digital image by de-convolving the captured digital image using the point spread function (PSF).

2. A system according to claim 1, wherein said digital processing component uses the captured image and a single reference image as the image data sets that are compared.

3. A system according to claim 1, wherein said digital processing component identifies at least one characteristic of the reference image which is relatively less blurred than the corresponding characteristic of the captured image, and calculates said blur function in respect of said characteristic.

4. A system according to claim 1, wherein said digital processing component uses the captured image and multiple reference images taken at successive points in time.

5. A system according to claim 1, wherein said digital processing component is configured to inform the photographer if it determines that the captured image is blurred.

6. A system of according to claim 1, wherein the digital image capturing apparatus is configured to capture another image if the digital processing component determines that the captured image is blurred.

7. A system accordingly to claim 1, wherein said digital processing component is located in said portable digital camera.

8. A system according to claim 1, wherein said digital image acquisition system comprises a combination of a portable digital camera and an external processing device.

9. A system accordingly to claim 8, wherein said digital processing component is located in said external processing device.

10. A system according to claim 1, wherein said digital image capturing apparatus is configured to capture a reference image before said exposure period of said captured image.

11. A system according to claim 10, wherein the reference image comprises a preview image.

12. A system according to claim 1, wherein the reference image comprises a lower pixel resolution than the captured image, and wherein said digital processing component is arranged to match the pixel resolutions of the captured image and the reference image.

13. A system according to claim 12, wherein said digital processing component is arranged to match the pixel resolutions of the captured image and the reference mage by at least one of up-sampling the reference image and sub-sampling the captured image.

14. A system according to claim 1, wherein said motion blur function determining occurs in real-time relative to the image capture.

15. A system according to claim 1, wherein said apparatus further comprises an optical system and said at least two sets of image data are acquired along a same optical path.

16. In a digital image acquisition system including an apparatus for capturing digital images, including a lens, an image sensor and a processor, a method comprising:

programming said processor with program code embedded within a computer readable medium, and

using said processor,

determining a camera motion blur function in a captured digital image based on a comparison of at least two images each acquired with a same sensor, including at least one relatively low resolution reference image, such that at least each pixel of a same portion of said same sensor captures at least a pixel of each of said at least two sets of image data, within a temporal range that includes times proximately before or after said exposure period, and such that the captured digital image and said at least one relatively low resolution reference image comprise approximately the same scene, and

wherein the determining of said camera motion blur function comprises

aligning at least portions of said at least two sets of image data,

calculating a point spread function (PSF) to points within an object captured within the captured digital image, and

comparing the at least two sets of image data for determining said camera motion blur function in said captured digital image, and

wherein the method further comprises using the processor in determining to de-blur said captured digital image when the camera motion blur function indicates that sufficient blur exists within the captured digital image, and

deblurring the captured digital image including de-convolving the captured digital image using the point spread function (PSF).

17. A method according to claim 16, wherein said reference image is taken before said exposure period of said captured image.

18. A method according to claim 16, wherein the determining comprises matching pixel resolutions of the reference and captured images by at least one of up-sampling the reference image and sub-sampling the captured image.

19. A method according to claim 16, further comprising identifying at least one characteristic of the reference image which is relatively less blurred than the corresponding characteristic of the captured image, and calculating said blur function in respect of said characteristic.

20. A method according to claim 16, further comprising calculating a trajectory of at least one characteristic in the reference image, extrapolating such characteristic onto the captured image, and calculating said blur function in respect of said characteristic.

21. A method according to claim 16, further including informing the photographer when the captured image is determined to be blurred.

22. A method according to claim 16, further including transferring the at least two images from the digital image capturing apparatus to an external processing device for determining the motion blur function.

23. A method according to claim 16, wherein the motion blur function determining is performed in real time relative to the capturing of the digital image.

24. A method according to claim 16, wherein said capturing of said reference and captured digital images occurs along a same optical path.

25. One or more processor-readable storage devices having digital code embedded therein for programming a processor to perform a camera motion blur function determining method that comprises:

programming said processor with the embedded digital code,

using said processor;

determining a camera motion blur function in a captured digital image based on a comparison of at least two images each acquired with a same sensor, including at least one relatively low resolution reference image, such that at least each pixel of a same portion of said same sensor captures at least a pixel of each of said at least two sets of image data, within a temporal range that includes times proximately before or after said exposure period, and such that the captured digital image and said at least one relatively low resolution reference image comprise approximately the same scene, and

wherein the determining of said camera motion blur function comprises

aligning at least portions of said at least two sets of image data;

calculating a point spread function (PSF) to points within an object captured within the captured digital image, and

comparing the at least two sets of image data for determining said camera motion blur function in said captured digital image, and

further using the processor in determining to de-blur said captured digital image when the camera motion blur function indicates that sufficient blur exists within the captured digital image, and

deblurring the captured digital image including de-convolving the captured digital image using the point spread function (PSF).

26. One or more processor-readable storage devices according to claim 25, wherein said reference image is taken before said exposure period of said captured image.

27. One or more processor-readable storage devices according to claim 25, wherein the determining comprises matching pixel resolutions of the reference and captured images by at least one of up-sampling the reference image and sub-sampling the captured image.

28. One or more processor-readable storage devices according to claim 25, wherein the method further comprises identifying at least one characteristic of the reference image which is relatively less blurred than the corresponding characteristic of the captured image, and calculating said blur function in respect of said characteristic.

29. One or more processor-readable storage devices according to claim 25, wherein the method further comprises calculating a trajectory of at least one characteristic in the reference image, extrapolating such characteristic onto the captured image, and calculating said blur function in respect of said characteristic.

30. One or more processor-readable storage devices according to claim 25, wherein the method further comprises informing the photographer when the captured image is determined to be blurred.

31. One or more processor-readable storage devices according to claim 25, wherein the method further comprises transferring the at least two images from the digital image capturing apparatus to an external processing device for determining the motion blur function.

32. One or more processor-readable storage devices according to claim 25, wherein the motion blur function determining is performed in real time relative to the capturing of the digital image.

33. One or more processor-readable storage devices according to claim 25, wherein said capturing of said reference and captured digital images occurs along a same optical path.