Method and apparatus to correct blur in all or part of a digital image by combining plurality of images

• US 10,171,740 B2
• Filed: 09/25/2017
• Issued: 01/01/2019
• Est. Priority Date: 03/25/2004
• Status: Active Grant

• Current Assignee: Clear Imaging Research, LLC
• Original Assignee: Clear Imaging Research, LLC
• Inventors: Ozluturk, Fatih M.
• Primary Examiner(s): Monk, Mark T
• Application Number:
  US15/714,558
• Publication Number:
  US 20180027184A1
• Time in Patent Office: 463 Days
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A method and apparatus for use in a digital imaging device for correcting image blur in digital images by combining plurality of images. The plurality of images that are combined include a main subject that can be selected by user input or automatically by the digital imaging device. Blur correction can be performed to make the main subject blur-free while the rest of the image is blurred. All of the image may be made blur-free or the main subject can be made blur-free at the expense of the rest of the image. Result is a blur corrected image that is recorded in a memory.

Claims

1. A method for use in an imaging device, the method comprising:

displaying an image in a user interface of the device, wherein the image is a preview of a field of view of the device, and wherein the image includes a first subject and a second subject;

designating by a processor of the imaging device the first subject in the image to be kept blur free;

capturing a plurality of images by the imaging device, wherein the plurality of images include the first subject and the second subject;

combining the plurality of images by the processor to obtain a combined image, such that;

the combined image includes the first subject and the second subject, the first subject in the combined image is blur free, and the second subject in the combined image is blurred compared to the first subject;

displaying the combined image in the display of the device; and

storing the combined image in a memory of the device.

3. The method of claim 1, wherein combining the plurality of images includes calculating by the processor a pixel value for each pixel in the first subject of the combined image based on values of the pixel of the first subject in one or more of the plurality of images.

4. The method of claim 1, wherein the areas of the combined image other than the first subject are blurred compared to the plurality of images being combined.

5. The method of claim 1, wherein the plurality of images is two images.

6. The method of claim 1, wherein the processor indicates the first subject by an indicator in the user interface of the device.

7. The method of claim 1, wherein combining the plurality of images to generate a combined image comprises modifying the plurality of images such that the first subject is aligned in a same location in the plurality of images.

8. The method of claim 1, wherein combining the plurality of images to generate a combined image is performed by combining images captured at different times.

9. The method of claim 1, wherein combining the plurality of images further comprises determining a pixel value, for each pixel not in the first subject of the combined image, based on pixel values in the plurality of images.

2. The method of 1, wherein the method further comprises;

receiving a user input in the user interface;

based on the user input, combining the plurality of images by the processor to obtain a second combined image, such that;

the second combined image includes the first subject and the second subject;

the second subject in the first combined image is blurred compared to the second subject in the second combined image; and

displaying the second combined image in the display of the device.

10. An imaging device for capturing and processing images, comprising:

a user interface configured to display an image, wherein the image is a preview of the field of view of the device, and wherein the image includes a first subject and a second subject;

a processor configured to designate the first subject in the image to be kept blur free;

a first memory configured to store a plurality of images captured by the device, wherein the plurality of images include the first subject and the second subject;

the processor further configured to combine the plurality of images to obtain a combined image, such that;

the combined image includes the first subject and the second subject, the first subject in the combined image is blur free, and the second subject in the combined image is blurred compared to the first subject;

the user interface further configured to display the combined image; and

a second memory configured to store the combined image.

11. The imaging device of claim 10, wherein the first and second memory are the same memory.

12. The imaging device of claim 10, wherein:

the user interface is further configured to receive a user input;

the processor is further configured to combine the plurality of images to obtain a second combined image based on the user input, such that;

the second combined image includes the first subject and the second subject, the second subject in the first combined image is blurry compared to the second subject in the second combined image; and

the display is further configured to display the second combined image.

13. The imaging device of claim 10, wherein the processor is further configured to calculate a pixel value for each pixel in the first subject of the combined image, based on values of the pixel of the first subject in one or more of the plurality of images.

14. The imaging device of claim 10, wherein the processor is configured to combine the plurality of images to obtain the combined image such that the areas of the combined image other than the first subject are blurred compared to the plurality of images being combined.

15. The imaging device of claim 10, wherein the plurality of images is two images.

16. The imaging device of claim 10, wherein the processor is further configured to indicate the first subject by an indicator in the user interface.

17. The imaging device, of claim 10, wherein the processor is further configured to modify the plurality of images such that the first subject is aligned in a same location in the plurality of images before combining the plurality of images to obtain the combined image.

18. The imaging device of claim 10, wherein the processor is further configured to combine the plurality of images to generate the combined image by combining images captured at different times sequentially.

19. The imaging device of claim 10, wherein the processor is further configured to calculate a pixel value, for each pixel not in the first subject of the combined image, based on pixel values in the plurality of images.

20. An imaging device for capturing and processing images, comprising:

a user interface configured to display an image, wherein the image is a preview of the field of view of the device, and wherein the image includes a first subject and a second subject;

a processor configured to receive user input designating the first subject in the image to be kept blur free;

a memory configured to store a plurality of images captured by the device, wherein the plurality of images include the first subject and the second subject;

the processor further configured to combine the plurality of images to obtain a combined image, such that;

the combined image includes the first subject and the second subject, the first subject in the combined image is blur free, and the second subject in the combined image is blurred compared to the first subject;

the user interface further configured to display the combined image; and

a memory configured to store the combined image.

21. The imaging device of claim 20, wherein:

the user interface is further configured to receive a second user input;

the processor is further configured to combine the plurality of images to obtain a second combined image based on the second user input, such that;

the second combined image includes the first subject and the second subject, the second subject in the first combined image is blurry compared to the second subject in the second combined image; and

the display is further configured to display the second combined image.

22. The imaging device of claim 20, wherein:

the user interface is further configured to receive a user input;

the processor is further configured to combine the plurality of images to obtain a second combined image based on the user input, such that;

the second combined image includes the first subject and the second subject, the second subject in the first combined image is blurry compared to the second subject in the second combined image; and

the display is further configured to display the second combined image.

23. The imaging device of claim 20, wherein the processor is further configured to calculate a pixel value for each pixel in the first subject of the combined image, based on values of the pixel of the first subject in one or more of the plurality of images.

24. The imaging device of claim 20, wherein the processor is configured to combine the plurality of images to obtain the combined image such that the areas of the combined image other than the first subject are blurred compared to the plurality of images being combined.

25. The imaging device of claim 20, wherein the plurality of images is two images.

26. The imaging device of claim 20, wherein the processor is further configured to indicate the first subject by an indicator in the user interface.

27. The imaging device, of claim 20, wherein the processor is further configured to modify the plurality of images such that the first subject is aligned in a same location in the plurality of images before combining the plurality of images to obtain the combined image.

28. The imaging device of claim 20, wherein the processor is further configured to combine the plurality of images to generate the combined image by combining images captured at different times sequentially.

29. The imaging device of claim 20, wherein the processor is further configured to calculate a pixel value, for each pixel not in the first subject of the combined image, based on pixel values in the plurality of images.