IMAGE ENHANCEMENT BASED ON MULTIPLE FRAMES AND MOTION ESTIMATION

• US 20130258174A1
• Filed: 05/28/2013
• Published: 10/03/2013
• Est. Priority Date: 03/11/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Inventors: TZUR, MEIR, Baxansky, Artemy
• Granted Patent: US 8,711,234 B2
• Time in Patent Office: Days
• US Class Current: 348/362
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A system and method for capturing images is provided. In the system and method, preview images are acquired and global local and local motion are estimated based on at least a portion of the preview images. If the local motion is less than or equal to the global motion, a final image is captured based at least on an exposure time based on the global motion. If the local motion is greater than the global motion, a first image is captured based on at least a first exposure time and at least a second image is captured based on at least one second exposure time less than the first exposure time. After capturing the first and second images, global motion regions are separated from local motion regions in the first and second images, and the final image is reconstructed at least based on the local motion regions.

Claims

1. An imaging device, comprising:

an optical system used to receive images; and

a processing element communicatively coupled to the optical system and that is programmed to perform actions, including;

receiving a plurality of preview images from the optical system;

estimating a global motion and a local motion based on at least a portion of the plurality of preview images; and

when the estimated local motion is less than or equal to the estimated global motion, configuring the optical system to provide a final image based at least on an exposure time based on the estimated global motion; and

otherwise;

configuring the optical system to provide a first image based on at least a first exposure parameter set and a plurality of other images based at least on one or more other exposure parameter sets,

separating a global motion region from a local motion region in the first image,

separating a plurality of local motion regions from global motion regions in the plurality of other images, and

generating a final composite image from a fusion of the separated global motion region from the first image with the separated plurality of local motion regions in the plurality of other images.

2. The imaging device of claim 1, wherein at least one exposure parameter includes an exposure time, and wherein at least one exposure time within the one or more other exposure parameter sets is less than an exposure time in the first exposure parameter set.

3. The imaging device of claim 1, wherein configuring the optical system to provide the first image comprises computing a first exposure time based on the estimated global motion, the first exposure time being included within the first exposure parameter set.

4. The imaging device of claim 1, wherein at least one exposure parameter within the one or more other exposure parameter sets is computed based on the estimated local motion.

5. The imaging device of claim 1, wherein generating a final composite image further comprises replacing image information in the first image for at least one area of location motion with image information in one or more of the plurality of other images for at least one are of local motion.

6. The imaging device of claim 1, wherein at least one exposure parameter in the first and one or more other exposure parameter sets include an exposure time, a size of an aperture opening, or a gain.

7. A controller device, comprising:

a memory management unit configured to receive and manage images; and

a processor unit that is programmed to receive images and perform actions, including;

receiving a plurality of preview images from the memory management unit;

estimating a global motion and a local motion based on at least a portion of the plurality of preview images; and

when the estimated local motion is less than or equal to the estimated global motion, configuring an optical system to provide a final image based at least on an exposure time based on the estimated global motion; and

otherwise;

configuring the optical system to provide a first image based on at least a first exposure parameter set and a plurality of other images based at least on one or more other exposure parameter sets,

separating a global motion region from a local motion region in the first image,

separating a plurality of local motion regions from global motion regions in the plurality of other images, and

generating a final composite image from a fusion of the separated global motion region from the first image with the separated plurality of local motion regions in the plurality of other images.

8. The controller device of claim 7, wherein at least one exposure parameter within the one or more other exposure parameter sets is computed based on at least one of an average, a median, or a maximum amount of motion detected within a scene.

9. The controller device of claim 7, wherein at least one exposure parameter within the one or more other exposure parameter sets is computed based on a determined of a number of portions of a scene having a magnitude of estimated local motion that is greater than a magnitude of estimated global motion in the scene.

10. The controller device of claim 7, wherein generating a final composite image further comprises replacing image information in the first image for at least one area of location motion with image information in one or more of the plurality of other images for at least one are of local motion.

11. The controller device of claim 7, wherein at least one exposure parameter in first and one or more other exposure parameter sets include an exposure time, a size of an aperture opening, or a gain.

12. The controller device of claim 7, wherein a time to capture the first image is determined based on estimating a magnitude of motion by extrapolating the magnitude of motion from the plurality of preview images, and identifying a point of minimal motion within a set period.

13. The controller device of claim 7, wherein at least one exposure parameter includes an exposure time, and wherein at least one exposure time with the one or more other exposure parameter sets is less than an exposure time in the first exposure parameter set.

14. The controller device of claim 7, wherein configuring the optical system to provide the first image comprises computing a first exposure time based on the estimated global motion, the first exposure time being included within the first exposure parameter set.

15. A non-transitory processor-readable storage device, having processor executing instructions that perform actions, comprising:

receiving a plurality of preview images from an optical system;

estimating a global motion and a local motion based on at least a portion of the plurality of preview images; and

when the estimated local motion is less than or equal to the estimated global motion, configuring the optical system to provide a final image based at least on an exposure time based on the estimated global motion; and

otherwise;

configuring the optical system to provide a first image based on at least a first exposure parameter set and a plurality of other images based at least on one or more other exposure parameter sets,

separating a global motion region from a local motion region in the first image,

separating a plurality of local motion regions from global motion regions in the plurality of other images, and

generating a final image from a composite of just the separated global motion region from the first image with the separated plurality of local motion regions in the plurality of other images.

16. The storage device of claim 15, wherein at least one exposure parameter within the one or more other exposure parameter sets is computed based on at least one of an average, a median, or a maximum amount of motion detected within a scene.

17. The storage device of claim 15, wherein at least one exposure parameter within the one or more other exposure parameter sets is computed based on a determined of a number of portions of a scene having a magnitude of estimated local motion that is greater than a magnitude of estimated global motion in the scene.

18. The storage device of claim 15, wherein generating a final image further comprises replacing image information in the first image for at least one area of location motion with image information in one or more of the plurality of other images for at least one are of local motion.

19. The storage device of claim 15, wherein at least one exposure parameter in first and one or more other exposure parameter sets include an exposure time, a size of an aperture opening, or a gain.

20. The storage device of claim 15, wherein a time to capture the first image is determined based on estimating a magnitude of motion by extrapolating the magnitude of motion from the plurality of preview images, and identifying a point of minimal motion within a set period.