System and method for constructing a motion-compensated composite image

• US 8,478,071 B2
• Filed: 12/16/2009
• Issued: 07/02/2013
• Est. Priority Date: 12/16/2009
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Aila, Timo, Laine, Samuli
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US12/639,382
• Publication Number:
  US 20110142369A1
• Time in Patent Office: 1,294 Days
• Field of Search: None
• US Class Current: 382/284
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A method for constructing a motion-compensated composite image of a scene includes acquiring a plurality of images of a scene over time, the plurality of images including an earlier-acquired image of the scene and a later-acquired image scene. The relative motion between the earlier and later acquired images are estimated, and an exposure parameter is computed based upon the estimated relative motion occurring between the earlier and later acquired images. A new image of the scene is acquired using the computed exposure parameter, and the earlier, later, and newly acquired images are combined to produce a motion-compensated composite image of the scene.

Claims

1. A method for constructing a motion-compensated composite image of a scene, the method comprising:

(i) acquiring a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) estimating relative motion between the earlier and later images;

(iii) computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) acquiring a number of new images of the scene, wherein the number is determined based on the computed exposure parameter value; and

(v) compositing the earlier, later and new images of the scene to produce a motion-compensated composite image of the scene.

2. The method of claim 1, wherein the exposure parameter value comprises an exposure duration.

3. The method of claim 1, wherein (ii) comprises computing an Euclidean distance between a location of a predefined feature within the earlier acquired image and a location of said predefined feature within the later acquired image.

4. The method of claim 1,

wherein the exposure parameter value comprises an exposure duration,

wherein the estimated motion does not exceed a predefined value, and

wherein (iii) comprises increasing an exposure duration of the later image to compute the exposure duration of at least one of the new images.

5. The method of claim 1,

wherein the exposure parameter value comprises an exposure duration,

wherein the estimated motion does exceed a predefined value, and

wherein (iii) comprises decreasing an exposure duration of the later image to compute the exposure duration of at least one of the new images.

6. The method of claim 1, wherein (v) comprises:

compositing the earlier and later images to produce an intermediate composite image; and

compositing the new images the intermediate composite image to produce the motion-compensated composite image of the scene.

12. The method of claim 1,

wherein (i) comprises obtaining a reference image of the scene, and

wherein (ii) comprises:

(a) computing a first offset between the position of a predefined feature located within the earlier acquired image and said predefined feature ;

located within the reference image;

(b) computing a second offset between said position of said predefined feature located within the later acquired image and a position of said predefined feature located within the reference image;

(c) computing a motion vector length comprising the difference between the first and second offsets; and

(d) using the computed motion vector length to estimate the relative motion between the earlier and later acquired images.

7. A method for constructing a motion-compensated composite image of a scene, the method comprising:

(i) acquiring a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) estimating relative motion between the earlier and later images;

(iii) computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) acquiring, using the computed exposure parameter value, a new image of the scene; and

(v) compositing the earlier and later images to produce an intermediate composite image;

(vi) re-assigning the later image as an earlier image of the scene;

(vii) re-assigning the new image as a later image of the scene;

(viii) repeating (ii), (iii) and (iv) for the re-assigned earlier and later images, wherein motion is estimated between the re-assigned earlier and later images; and

(ix) compositing the new image with the intermediate composite image to produce a motion-compensated composite image of the scene.

8. The method of claim 7, wherein the estimated motion between the re-assigned earlier and later images does not exceed a predefined threshold, the method further comprising repeating (ii)-(iv) for the re-assigned earlier and later images,

wherein (ii) comprises estimating relative motion between the re-assigned earlier and later images;

wherein (iii) comprises computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

wherein (iv) comprises acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises compositing the re-assigned earlier and later images to produce the motion-compensated composite image of the scene.

9. The method of claim 7, wherein the estimated motion between the re-assigned earlier and later images exceeds a predefined threshold, the method further comprising:

excluding the re-assigned later image from composition with the reference image;

re-assigning the excluded later image as an earlier image of the scene;

re-assigning the new image as a later image of the scene; and

repeating (ii) for the re-assigned earlier and the updated later images, wherein motion is estimated between the re-assigned earlier and updated later images.

10. The method of claim 9, further comprising repeating (iii) to compute a new exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images, wherein the new exposure parameter comprises a reduced exposure duration compared to the exposure parameter corresponding to the re-assigned earlier image which is excluded from composition with the reference image.

11. The method of claim 10, wherein the estimated motion between the re-assigned earlier and updated later images does not exceed a predefined threshold, the method further comprising repeating (ii)-(iv) for the re-assigned earlier and updated later images,

wherein (ii) comprises estimating relative motion between the re-assigned earlier and updated later images;

wherein (iii) comprises computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and updated later images;

wherein (iv) comprises acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises compositing the re-assigned earlier and updated later images to produce the motion-compensated composite image of the scene.

13. A computer program product, resident of a non-transitory computer readable medium, operable to store instructions for constructing a motion-compensated composite image of a scene, the computer program product comprising:

(i) instruction code for acquiring a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) instruction code for estimating relative motion between the earlier and later images;

(iii) instruction code for computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) instruction code for acquiring a number of new images of the scene, wherein the number is determined based on the computed exposure parameter value; and

(v) instruction code for compositing the earlier, later and new images of the scene to produce a motion-compensated composite image of the scene.

14. The computer program product of claim 13, wherein (ii) comprises instruction code for computing an Euclidean distance between a location of a predefined feature with the earlier acquired image and a location of said predefined feature within the later acquired image.

15. The computer program product of claim 13, wherein (v) comprises:

instruction code for compositing the earlier and later images to produce an intermediate composite image; and

instruction code for compositing at least one of the new images with the intermediate composite image to produce the motion-compensated composite image of the scene.

16. A computer program product, resident of a non-transitory computer readable medium, operable to store instructions for constructing a motion-compensated composite image of a scene, the computer program product comprising:

(i) instruction code for acquiring a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) instruction code for estimating relative motion between the earlier and later images;

(iii) instruction code for computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) instruction code for acquiring, using the computed exposure parameter value, a new image of the scene;

(v) instruction code for compositing the earlier and later images to produce an intermediate composite image;

(vi) instruction code for re-assigning the later image as an earlier image of the scene;

(vii) instruction code for re-assigning the new image as a later image of the scene;

(viii) instruction code for repeating (ii), (iii) and (iv) for the re-assigned earlier and later images, wherein motion is estimated between the re-assigned earlier and later images; and

(ix) instruction code for compositing the image with the intermediate composite image to produce the motion-compensated composite image of the scene.

17. The computer program product of claim 16, wherein the estimated motion between the re-assigned earlier and later images does not exceed a predefined threshold, the method further comprising instruction code for repeating (ii)-(iv) for the re-assigned earlier and later images,

wherein (ii) comprises instruction code for estimating relative motion between the re-assigned earlier and later images;

wherein (iii) comprises instruction code for computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

wherein (iv) comprises instruction code for acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises instruction code for compositing the re-assigned earlier and later images to produce the motion-compensated composite image of the scene.

18. A system operable to construct a motion-compensated composite image of a scene, the processor comprising:

(i) imaging means for acquire a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) means for estimating relative motion between the earlier and later images;

(iii) means for computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) imaging means for acquiring a number of new images of the scene, wherein the number is determined based on the computed exposure parameter value; and

(v) means for compositing the earlier, later and new images of the scene to produce a motion-compensated composite image of the scene.

19. The system of claim 18, wherein (ii) comprises means for computing an Euclidean distance between a location of a predefined pixel with the earlier acquired image and a location of said predefined pixel within the later acquired image.

20. The system of claim 18, wherein (v) means for compositing comprises:

means for compositing the earlier and later images to produce an intermediate composite image; and

means for compositing at least one of the new images with the intermediate composite image to produce the motion-compensated composite image of the scene.

21. A system operable to construct a motion-compensated composite image of a scene, the processor comprising:

(i) imaging means for acquire a plurality of images of a scene over time, the plurality of images including an earlier image of the scene and a later image of the scene;

(ii) means for estimating relative motion between the earlier and later images;

(iii) means for computing an exposure parameter value based upon the relative motion estimated between the earlier and later images;

(iv) imaging means for acquiring, using the computed exposure parameter value, a new image of the scene; and

(v) means for compositing the earlier and later images to produce an intermediate composite image;

(vi) means for re-assigning the later image as an earlier image of the scene;

(vii) means for re-assigning the new image as a later image of the scene

(viii) wherein (ii) comprises means for estimating relative motion between the re-assigned earlier and later images;

(ix) wherein (iii) comprises means for computing an additional exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

(x) wherein (iv) comprises means for acquiring, using the computed additional exposure parameter value, an additional new image of the scene; and

(xi) means for compositing the additional new image and the new image with the intermediate composite image to produce the motion-compensated composite image of the scene.

22. The system of claim 21,

wherein the estimated motion between the re-assigned earlier and later images does not exceed a predefined threshold,

wherein (ii) comprises means for estimating relative motion between the re-assigned earlier and later images;

wherein (iii) comprises means for computing an exposure parameter value based upon the relative motion estimated between the re-assigned earlier and later images;

wherein (iv) comprises imaging means for acquiring, using the computed exposure parameter value, a new image of the scene; and

wherein (v) comprises means for composting there-assigned earlier and later images to produce the motion-compensated composite image of the scene.