Method and apparatus to correct blur in all or part of an image

• US 10,389,944 B2
• Filed: 12/29/2017
• Issued: 08/20/2019
• Est. Priority Date: 03/25/2004
• Status: Active Grant

• Current Assignee: Clear Imaging Research, LLC
• Original Assignee: Clear Imaging Research, LLC
• Inventors: Ozluturk, Fatih M.
• Primary Examiner(s): Peterson, Christopher K
• Application Number:
  US15/858,339
• Publication Number:
  US 20180124322A1
• Time in Patent Office: 599 Days
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
• CPC Class Codes: H04N 5/265 Mixing

Abstract

The effect of blur in digital images of an imaging device is corrected by displaying a preview image of a scene to be captured in a user interface of a device. A user input designates a first subject in the preview image and a plurality of images that include the first subject and a second subject are captured. The plurality of images are processed to obtain a combined image, taking into account at least one of a focal length of a lens of the imaging device and a zoom level of a lens of the imaging device, and the combined image includes the first subject and the second subject, the first subject in the combined image is substantially blur free, and the second subject in the combined image is blurred compared to the first image. The combined image is stored in a memory of the device.

Claims

1. A method for use in an imaging device for capturing digital images, the method comprising:

displaying a preview image of the scene to be captured in a user interface of the device;

receiving a user input in the user interface, wherein the user input designates a first subject in the preview image;

capturing a plurality of images, wherein the plurality of images include the first subject and a second subject;

processing the plurality of images by a processor of the device to obtain a combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and a zoom level of a lens of the imaging device, and

the combined image includes the first subject and the second subject, the first subject in the combined image is substantially blur free, and the second subject in the combined image is blurred compared to the first image; and

storing the combined image in a memory of the device.

2. The method of claim 1, wherein the method further comprises;

displaying the combined image in the user interface;

receiving a second user input based on selecting of an indicator in the user interface, wherein the second user input designates the second subject in the combined image;

processing the plurality of images by the processor to obtain a second combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and the zoom level of a lens of the imaging device, and

the second combined image includes the first subject and the second subject, the second subject in the second combined image is substantially blur free, and the first subject in the second combined image is blurred compared to the second subject; and

storing the combined image in the memory.

3. The method of claim 1, wherein the processing of the images includes calculating by the processor a pixel value for each pixel representing an image point in the first subject of the combined image based on values of the pixels representing the image point in the first subject in the one or more of the plurality of images.

4. The method of claim 1, wherein the areas of the combined image other than the first subject are blurred compared to the areas other than the first subject in the plurality of images being processed.

5. The method of claim 1, wherein the processing the plurality of images comprises modifying one or more of the plurality of images such that the first subject in the plurality of images is aligned in a same location.

6. An imaging device for capturing digital images, comprising:

a user interface configured to display an image and to receive a user input, wherein the displayed image is a preview of the scene to be captured, and wherein the user input designates a first subject in the displayed image;

an image sensor, configured to capture a plurality of images, wherein the plurality of images include the first subject and the second subject;

a processor configured to process the plurality of images to obtain a combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and the zoom level of a lens of the imaging device, and

the combined image includes the first subject and the second subject, the first subject in the combined image is substantially blur free, and the second subject in the combined image is blurred compared to the first subject; and

a memory configured to store the combined image.

7. The imaging device of claim 6, wherein:

the user interface is further configured to display the combined image and to receive a second user input, wherein the second user input designates the second subject in the combined image;

the processor is further configured to process the plurality of images to obtain a second combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and the zoom level of a lens of the imaging device, and

the second combined image includes the first subject and the second subject, the second subject in the second combined image is substantially blur free, and the first subject in the second combined image is blurred compared to the second subject; and

the memory is further configured to store the second combined image.

8. The imaging device of claim 6, wherein the processor is further configured to calculate pixel values for each pixel representing an image point in the first subject of the combined image based on values of the pixels representing the image point in the first subject in the one or more of the plurality of images.

9. The imaging device of claim 6, wherein the processor is further configured to process the plurality of images to obtain the combined image such that the areas of the combined image other than the first subject are blurred compared to the areas other than the first subject in the plurality of images being processed.

10. The imaging device of claim 6, wherein the processor is further configured to modify the plurality of images such that the first subject in the plurality of images is aligned in a same location.

11. A method for use in an imaging device for capturing digital images, the method comprising:

displaying a preview image of the scene to be captured in a user interface of the device;

designating by a processor of the imaging device a first subject in the preview image;

capturing a plurality of images, wherein the plurality of images include the first subject and a second subject;

processing the plurality of images by a processor of the device to obtain a combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and the zoom level of a lens of the imaging device, and

the combined image includes the first subject and the second subject, the first subject in the combined image is substantially blur free, and the second subject in the combined image is blurred compared to the first image; and

storing the combined image in a memory of the device.

12. The method of claim 11 wherein the method further comprises;

displaying the combined image in the user interface;

receiving a user input in the user interface, wherein the second user input designates the second subject in the combined image;

processing the plurality of images by the processor to obtain a second combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and the zoom level of a lens of the imaging device, and

the second combined image includes the first subject and the second subject, the second subject in the second combined image is substantially blur free, and the first subject in the second combined image is blurred compared to the second subject; and

storing the combined image in the memory.

13. The method of claim 11, wherein the processing of the images includes calculating by the processor a pixel value for each pixel representing an image point in the first subject of the combined image based on values of the pixels representing the image point in the first subject in the one or more of the plurality of images.

14. The method of claim 11, wherein the areas of the combined image other than the first subject are blurred compared to the areas other than the first subject in the plurality of images being processed.

15. The method of claim 11, wherein the processing the plurality of images comprises modifying one or more of the plurality of images such that the first subject in the plurality of images is aligned in a same location.

16. An imaging device for capturing digital images, comprising:

a user interface configured to display an image, wherein the image is a preview of the scene to be captured;

a processor configured to designate a first subject in the image;

an image sensor, configured to capture a plurality of images, wherein the plurality of images include the first subject and the second subject;

the processor further configured to process the plurality of images to obtain a combined image, such that;

the processing of the images takes into account at least one of a focal length of a lens of the imaging device and the zoom level of a lens of the imaging device, and

the combined image includes the first subject and the second subject, the first subject in the combined image is substantially blur free, and the second subject in the combined image is blurred compared to the first subject;

the user interface further configured to display the combined image; and

a memory configured to store the combined image.

17. The imaging device of claim 16, wherein:

the user interface is further configured to display the combined image and the to receive a user input, wherein the user input designates the second subject in the combined image;

the processor is further configured to combine the plurality of images to obtain a second combined image, such that;

the second combined image includes the first subject and the second subject, the second subject in the second combined image is substantially blur free, and the first subject in the second combined image is blurred compared to the second subject; and

the memory is further configured to store the second combined image.

18. The imaging device of claim 16, wherein the processor is further configured to calculate pixel values for each pixel representing an image point in the first subject of the combined image based on values of the pixels representing the image point in the first subject in the one or more of the plurality of images.

19. The imaging device of claim 16, wherein the processor is further configured to process the plurality of images to obtain the combined image such that the areas of the combined image other than the first subject are blurred compared to the areas other than the first subject in the plurality of images being processed.

20. The imaging device of claim 16, wherein the processor is further configured to modify the plurality of images such that the first subject in the plurality of images is aligned in a same location.