Image enhancement based on multiple frames and motion estimation

• US 8,482,620 B2
• Filed: 03/04/2009
• Issued: 07/09/2013
• Est. Priority Date: 03/11/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Inventors: Tzur, Meir, Baxansky, Artemy
• Primary Examiner(s): Vu, Ngoc-Yen
• Assistant Examiner(s): Chon, Peter
• Application Number:
  US12/397,727
• Publication Number:
  US 20090231449A1
• Time in Patent Office: 1,588 Days
• Field of Search: 348/207.99, 348/208.99, 348/208.1, 348/208.2, 348/208.3, 348/208.4, 348/208.5, 348/208.6, 348/208.7, 348/208.8, 348/208.11, 348/208.12, 348/208.13, 348/208.14, 348/208.15, 348/208.16, 348/220.1, 348/221.1, 348/222.1, 348/229.1, 348/230.1, 348/241, 348/248, 348/362, 348/363, 348/364
• US Class Current: 348/208.4
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A system and method for capturing images is provided. In the system and method, preview images are acquired and global local and local motion are estimated based on at least a portion of the preview images. If the local motion is less than or equal to the global motion, a final image is captured based at least on an exposure time based on the global motion. If the local motion is greater than the global motion, a first image is captured based on at least a first exposure time and at least a second image is captured based on at least one second exposure time less than the first exposure time. After capturing the first and second images, global motion regions are separated from local motion regions in the first and second images, and the final image is reconstructed at least based on the local motion regions.

Claims

1. A method for capturing images, comprising:

acquiring a plurality of preview images;

estimating a global motion and a local motion based on at least a portion of the plurality of preview images; and

if the estimated local motion is less than or equal to the estimated global motion;

capturing a final image based at least on an exposure time based on the estimated global motion;

else

capturing a first image based on at least a first exposure time,

capturing at least a second image based on at least one second exposure time less than the first exposure time,

separating a global motion region from a local motion region in the first image,

and

reconstructing the final image based at least on combining just the separated local motion region of the second image with at least the global motion region of the first image.

2. The method of claim 1, wherein capturing the first image comprises computing the first exposure time based on the global motion.

3. The method of claim 1, wherein capturing the second image comprises computing the second exposure time based on the local motion.

4. The method of claim 1, wherein separating each global motion region from each local motion region further comprises identifying each global motion region and each local motion region in the first and second images based on calculating motion vectors for the first and second images.

5. The method of claim 1, wherein reconstructing the final image further comprises:

obtaining a registration between the first and second images; and

combining image information from at least one global motion region of the first image with image information from at least one local motion region of the second image.

6. The method of claim 1, wherein reconstructing the final image further comprises:

obtaining a registration between the first and second images;

identifying areas of local motion in the first and second images; and

replacing image information in the first image for at least one area of local motion with image information in the second image for at least one area of local motion.

7. The method of claim 1, wherein capturing the second image further comprises capturing a plurality of, second images.

8. An electronic imaging device, comprising:

a processing element communicatively coupled to an electronic optical system and that is arranged to;

receive a plurality of preview images from the optical system;

estimating a global motion and a local motion based on at least a portion of the plurality of preview images; and

if the estimated local motion is less than or equal to the estimated global motion,

configuring the optical system to provide a final image based at least on an exposure time based on the estimated global motion;

else

configuring the optical system to provide a first image based on at least a first exposure time and at least a second image based at least on a second exposure time less than the first exposure time,

separating a global motion region from a local motion region in the first image,

separating a global motion region from a local motion region in the second image,

reconstructing the final image based at least on combining just the separated local motion region of the second image with at least the global motion region of the first image.

9. The electronic imaging device of claim 8, wherein the processing element configures the optical system to capture the first image based at least on computing the first exposure time based on the global motion.

10. The electronic imaging device of claim 8, wherein the processing element configures the optical system to capture the first image based at least on computing the second exposure time based on the local motion.

11. The electronic imaging device of claim 8, wherein the processing element separates each global motion region from each local motion region based on identifying at least one global motion region and at least one local motion region in the first and second images based on calculating motion vectors for the first and second images.

12. The electronic imaging device of claim 8, wherein the processing element reconstructs the final image based at least on:

obtaining a registration between the first and second images; and

combining image information from at least one global motion region of the first image with image information from at least one local motion region of the second image.

13. The electronic imaging device of claim 8, wherein reconstructing the final image further comprises:

obtaining a registration between the first and second images;

identifying at least one area of local motion in the first and second images; and

replacing image information in the first image for at least one area of local motion with image information in the second image for at least one area of local motion.

14. The electronic imaging device of claim 8, wherein configuring the optical system to provide the first and the second image further comprises configuring the optical system to provide a plurality of second images based on a plurality of second exposure times less than the first exposure time.

15. A non-transitory processor readable storage medium that stores one or more components that, when executed, enable actions for capturing images, comprising:

acquiring a plurality of preview images;

estimating a global motion and a local motion based on at least a portion of the plurality of preview images; and

if the estimated local motion is less than or equal to the estimated global motion,

capturing a final image based at least on an exposure time based on the estimated global motion;

else;

capturing a first image based on at least a first exposure time,

capturing at least a second image based on at least one second exposure time less than the first exposure time,

separating a global motion region from a local motion region in the first image,

separating a global motion region from a local motion region in the second image, and

reconstructing the final image based on combining just the separated local motion region of the second image with at least the global motion region of the first image.

16. The non-transitory processor readable storage medium of claim 15, wherein capturing the first image comprises computing the first exposure time based on the global motion.

17. The non-transitory processor readable storage medium of claim 15, wherein capturing the second image comprises computing the second exposure time based on the local motion.

18. The non-transitory processor readable storage medium of claim 15, wherein separating each global motion region from each local motion region further comprises identifying at least one global motion region and at least one local motion region in the first and second images based on calculating motion vectors for the first and second images.

19. The non-transitory processor readable storage medium of claim 15, wherein reconstructing the final image further comprises:

obtaining a registration between the first and second images; and

combining image information from at least one global motion region of the first image with image information from at least one local motion region of the second image.

20. The non-transitory processor readable storage medium of claim 15, wherein reconstructing the final image further comprises:

obtaining a registration between the first and second images;

identifying at least one area of local motion in the first and second images; and

replacing image information in the first image for the at least one area of local motion with image information in the second image for at least one area of local motion.

21. The non-transitory processor readable storage medium of claim 15, wherein capturing the second image further comprises capturing a plurality of second images.