System and Method for Constructing a Motion-Compensated Composite Image

• US 20110142369A1
• Filed: 12/16/2009
• Published: 06/16/2011
• Est. Priority Date: 12/16/2009
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Laine, Samuli, AILA, Timo
• Granted Patent: US 8,478,071 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A method for constructing a motion-compensated composite image of a scene includes acquiring a plurality of images of a scene over time, the plurality of images including an earlier-acquired image of the scene and a later-acquired image scene. The relative motion between the earlier and later acquired images are estimated, and an exposure parameter is computed based upon the estimated relative motion occurring between the earlier and later acquired images. A new image of the scene is acquired using the computed exposure parameter, and the earlier, later, and newly acquired images are combined to produce a motion-compensated composite image of the scene.

Claims

1. A method for constructing a motion-compensated composite image of a scene, the method comprising:

(i) acquiring a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) estimating relative motion between the earlier and later images;

(iii) computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) acquiring, using the computed exposure parameter value, a new image of the scene; and

(v) compositing the earlier, later and new images of the scene to produce a motion-compensated composite image of the scene.

2. The method of claim 1, wherein the exposure parameter value comprises an exposure duration.

3. The method of claim 1, wherein (ii) comprises computing an Euclidean distance between a location of a predefined feature within the earlier acquired image and a location of said predefined feature within the later acquired image.

4. The method of claim 1,

wherein the exposure parameter value comprises an exposure duration,

wherein the estimated motion does not exceed a predefined value, and

wherein (iii) comprises increasing an exposure duration of the later image to compute the exposure duration of the new image.

5. The method of claim 1,

wherein the exposure parameter value comprises an exposure duration,

wherein the estimated motion does exceeds a predefined value, and

wherein (iii) comprises decreasing an exposure duration of the later image to compute the exposure duration of the new image.

6. The method of claim 1, wherein (v) comprises:

compositing the earlier and later images to produce an intermediate composite image; and

compositing the new image with the intermediate composite image to produce the motion-compensated composite image of the scene.

7. The method of claim 6, wherein after producing the intermediate composite image and before producing the motion-compensated composite image of the scene, the method comprises:

re-assigning the later image as an earlier image of the scene;

re-assigning the new image as a later image of the scene; and

repeating (ii) for the re-assigned earlier and later images, wherein motion is estimated between the re-assigned earlier and later images.

8. The method of claim 7, wherein the estimated motion between the re-assigned earlier and later images does not exceed a predefined threshold, the method further comprising repeating (ii)-(iv) for the re-assigned earlier and later images,

wherein (ii) comprises estimating relative motion between the re-assigned earlier and later images;

wherein (iii) comprises computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

wherein (iv) comprises acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises compositing the re-assigned earlier and later images to produce the motion-compensated composite image of the scene.

9. The method of claim 7, wherein the estimated motion between the re-assigned earlier and later images exceeds a predefined threshold, the method further comprising:

excluding the re-assigned later image from composition with the reference image; and

re-assigning the excluded later image as an earlier image of the scene;

re-assigning the new image as a later image of the scene; and

repeating (ii) for the re-assigned earlier and the updated later images, wherein motion is estimated between the re-assigned earlier and updated later images.

10. The method of claim 9, further comprising repeating (iii) to compute a new exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images, wherein the new exposure parameter comprises a reduced exposure duration compared to the exposure parameter corresponding to the re-assigned earlier image which is excluded from composition with the reference image.

11. The method of claim 10, wherein the estimated motion between the re-assigned earlier and updated later images does not exceed a predefined threshold, the method further comprising repeating (ii)-(iv) for the re-assigned earlier and updated later images,

wherein (ii) comprises estimating relative motion between the re-assigned earlier and updated later images;

wherein (iii) comprises computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and updated later images;

wherein (iv) comprises acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises compositing the re-assigned earlier and updated later images to produce the motion-compensated composite image of the scene.

12. The method of claim 1,

wherein (i) comprises obtaining a reference image of the scene, and

wherein (ii) comprises:

(a) computing a first offset between the position of a predefined feature located within the earlier acquired image and said predefined feature located within the reference image;

(b) computing a second offset between said position of said predefined feature located within the later acquired image and a position of said predefined feature located within the reference image;

(c) computing a motion vector length comprising the difference between the first and second offsets; and

(d) using the computed motion vector length to estimate the relative motion between the earlier and later acquired images.

13. A computer program product, resident of a computer readable medium, operable to store instructions for constructing a motion-compensated composite image of a scene, the computer program product comprising:

(i) instruction code for acquiring a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) instruction code for estimating relative motion between the earlier and later images;

(iii) instruction code for computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) instruction code for acquiring, using the computed exposure parameter value, a new image of the scene; and

(v) instruction code for compositing the earlier, later and new images of the scene to produce a motion-compensated composite image of the scene.

14. The computer program product of claim 13, wherein (ii) comprises instruction code for computing an Euclidean distance between a location of a predefined feature with the earlier acquired image and a location of said predefined feature within the later acquired image.

15. The computer program product of claim 13, wherein (v) comprises:

instruction code for compositing the earlier and later images to produce an intermediate composite image; and

instruction code for compositing the new image with the intermediate composite image to produce the motion-compensated composite image of the scene.

16. The computer program product of claim 15, wherein after producing the intermediate composite image and before producing the motion-compensated composite image of the scene, the method comprises:

instruction code for re-assigning the later image as an earlier image of the scene;

instruction code for re-assigning the new image as a later image of the scene; and

instruction code for repeating (ii) for the re-assigned earlier and later images, wherein motion is estimated between the re-assigned earlier and later images.

17. The computer program product of claim 16, wherein the estimated motion between the re-assigned earlier and later images does not exceed a predefined threshold, the method further comprising instruction code for repeating (ii)-(iv) for the re-assigned earlier and later images,

wherein (ii) comprises instruction code for estimating relative motion between the re-assigned earlier and later images;

wherein (iii) comprises instruction code for computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

wherein (iv) comprises instruction code for acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises instruction code for compositing the re-assigned earlier and later images to produce the motion-compensated composite image of the scene.

18. A system operable to construct a motion-compensated composite image of a scene, the processor comprising:

(i) imaging means for acquire a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) means for estimating relative motion between the earlier and later images;

(iii) means for computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) imaging means for acquiring, using the computed exposure parameter value, a new image of the scene; and

(v) means for compositing the earlier, later and new images of the scene to produce a motion-compensated composite image of the scene.

19. The system of claim 18, wherein (ii) comprises means for computing an Euclidean distance between a location of a predefined pixel with the earlier acquired image and a location of said predefined pixel within the later acquired image.

20. The system of claim 18, wherein (v) means for compositing comprises:

means for compositing the earlier and later images to produce an intermediate composite image; and

means for compositing the new image with the intermediate composite image to produce the motion-compensated composite image of the scene.

21. The system of claim 18, wherein the system comprises:

means for re-assigning the later image as an earlier image of the scene after producing the intermediate composite image and before producing the motion-compensated composite image of the scene;

means for re-assigning the new image as a later image of the scene after producing the intermediate composite image and before producing the motion-compensated composite image of the scene; and

wherein (ii) comprises means for estimating relative motion between the re-assigned earlier and later images.

22. The system of claim 21,

wherein the estimated motion between the re-assigned earlier and later images does not exceed a predefined threshold,

wherein (ii) comprises means for estimating relative motion between the re-assigned earlier and later images;

wherein (iii) comprises means for computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

wherein (iv) comprises imaging means for acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises means for compositing the re-assigned earlier and later images to produce the motion-compensated composite image of the scene.