Algorithm description on non-motion blur image generation project

• US 8,018,999 B2
• Filed: 12/05/2005
• Issued: 09/13/2011
• Est. Priority Date: 12/05/2005
• Status: Active Grant

• Current Assignee: ArcSoft, Inc.
• Original Assignee: ArcSoft, Inc.
• Inventors: Yao, Tianxiang, Jin, Yiqing, Wu, Donghui
• Primary Examiner(s): Pathak; Sudhanshu C
• Assistant Examiner(s): GETANEH, MESFIN S
• Application Number:
  US11/295,236
• Publication Number:
  US 20070127574A1
• Time in Patent Office: 2,108 Days
• Field of Search: 348/208.4, 348/208.6, 348/208.13, 348/208.14, 375/240.16, 375/240.24, 375/240.26, 382/255, 382/291, 382/294, 382/299
• US Class Current: 375/240.16
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A method for simulating an image captured at a long exposure time (“simulated image”), includes (1) capturing each of first, second, and third images at a short exposure time, (2) determining a first relative motion between the first and the second images, (3) transforming the first image to remove the first relative motion, (4) determining a second relative motion between the third and the second images, (5) transforming the third image to remove the second relative motion, and (6) combining the first, the second, and the third images to form the simulated image. Relative motions between images are determined by matching blocks at multiple resolutions to determine corresponding points between the images. Transformation to remove relative motion is determined by fitting corresponding points between the images using a minimum square error (MSE) algorithm in a random sample consensus (RANSAC) framework.

Claims

1. A method for simulating an image captured at a long exposure time (“

simulated image”

), comprising;

capturing each of a first image, a second image, and a third image at a short exposure time;

matching blocks in the first image with blocks in the second image to determine a first plurality of motion vectors between the first and the second images, said matching comprising;

down-sampling both the first and the second images from an original resolution to a first resolution and a second resolution, wherein the second resolution is lower than the first resolution;

matching blocks between the first and the second images at the second resolution;

propagating locations of center pixels of the matching blocks at the second resolution to both the first and the second images at the first resolution;

using the propagated locations of the center pixels of the matching blocks at the second resolution as starting points, matching blocks between the first and the second images at the first resolution; and

propagating locations of center pixels of the matching blocks at the first resolution to both the first and the second images at the original resolution to determine corresponding points between the first and the second images;

determining, using a processor, a first global motion between the first and the second images from the first plurality of motion vectors, said determining comprising;

fitting the corresponding points between the first and the second images using a random sample consensus (RANSAC) algorithm to determine the motion parameters of the first global motion between the first and the second images;

transforming the first image to compensate for the first global motion;

matching blocks in the third image with the blocks in the second image to determine a second plurality of motion vectors between the third and the second images;

determining, using the processor, a second global motion between the third and the second images from the second plurality of motion vectors;

transforming the third image to compensate for the second global motion; and

combining the first, the second, and the third images linearly to form the simulated image.

2. The method of claim 1, wherein the short exposure time is at most 1/25 second.

3. The method of claim 1, wherein the long exposure time is at least ⅕

second.

4. The method of claim 1, wherein said transforming the first image comprises determining a transformation comprising:

5. The method of claim 1, further comprising scaling the simulated image by a factor.

6. The method of claim 5, wherein said combining the first, the second, and the third images linearly comprises:

7. A non-transitory computer-readable storage medium encoded with executable instructions for execution by a processor to simulate an image captured at a long exposure time (“

simulated image”

), the instructions comprising;

capturing each of a first image, a second image, and a third image at a short exposure time;

matching blocks in the first image with blocks in the second image to determine a first plurality of motion vectors between the first and the second images, said matching comprising;

down-sampling both the first and the second images from an original resolution to a first resolution and a second resolution, wherein the second resolution is lower than the first resolution;

matching blocks between the first and the second images at the second resolution;

propagating locations of center pixels of the matching blocks at the second resolution to both the first and the second images at the first resolution;

using the propagated locations of the center pixels of the matching blocks at the second resolution as starting points, matching blocks between the first and the second images at the first resolution; and

propagating locations of center pixels of the matching blocks at the first resolution to both the first and the second images at the original resolution to determine corresponding points between the first and the second images;

determining, using a processor, a first global motion between the first and the second images from the first plurality of motion vectors, said determining comprising;

fitting the corresponding points between the first and the second images using a random sample consensus (RANSAC) algorithm to determine the motion parameters of the first global motion between the first and the second images;

transforming the first image to compensate for the first global motion;

matching blocks in the third image with the blocks in the second image to determine a second plurality of motion vectors between the third and the second images;

determining, using the processor, a second global motion between the third and the second images from the second plurality of motion vectors;

transforming the third image to compensate for the second global motion; and

combining the first, the second, and the third images linearly to form the simulated image.

8. The non-transitory computer-readable storage medium of claim 7, wherein the short exposure time is at most 1/25 second.

9. The non-transitory computer-readable storage medium of claim 7, wherein the long exposure time is at least ⅕

second.

10. The non-transitory computer-readable storage medium of claim 7, wherein said transforming the first image comprises determining a transformation comprising:

11. The non-transitory computer-readable storage medium of claim 7, wherein the instructions further comprise scaling the simulated image by a factor.

12. The non-transitory computer-readable storage medium of claim 11, wherein said combining the first, the second, and the third images linearly comprises: