Image Blending

• US 20090129700A1
• Filed: 07/28/2006
• Published: 05/21/2009
• Est. Priority Date: 07/28/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rother, Carsten, Kolmogorov, Vladimir, Blake, Andrew
• Granted Patent: US 8,019,177 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

Previously, Poisson blending has been used for image blending including cloning an object onto a target background and blending pairs of source images together. Such Poisson blending works well in many situations. However, whilst this method is always workable, we have found that discolorations sometimes occur. We realized that these discolorations occur when the gradient of the source image is preserved too insistently, at the expense of preserving object and background color. In some situations object outlines become smeared or blurred. We develop a color preservation term and a fragility measure to address these problems. This gives a user additional control to obtain smooth compositions and reduce discoloration artifacts.

Claims

1. A method of blending a first image and a second image to form a constructed image said method comprising the steps of:

(i) combining at least part of the first image and at least part of the second image to form the constructed image;

(ii) identifying a boundary in the constructed image being between a region in the constructed image formed from the first image and a region in the constructed image formed from the second image;

(iii) modifying the constructed image on the basis of the gradient of at least part of either of the first and second images; and

(iv) wherein said modification is also carried out on the basis of one or more of;

a color preservation term and an indication of a degree of contrast in at least one of the first and second images.

2. A method as claimed in claim 1 wherein at least part of the first image is cloned onto the second image to form the constructed image and wherein the step of modifying the constructed image comprises modifying only the part of the constructed image cloned from the first image.

3. A method as claimed in claim 1 wherein said first image is a source image and said second image is a source image and those source images are to be blended together to form a composite image.

4. A method as claimed in claim 1 wherein said modification is substantially equivalent to a minimization of a functional E(u)=∫

_S(u−

I_S)²+λ

₂∥

∇

u−

∇

I_S∥

², subject to boundary conditions and where u represents the constructed image, I_S represents a source image and X is a weighting parameter.

5. A method as claimed in claim 1 wherein said color preservation term has the effect in use of controlling the degree of blending between the first and second images.

6. A method as claimed in claim 1 wherein a weighting parameter is used to control a trade off between modification on the basis of the color preservation term and the modification on the basis of the gradient.

7. A method as claimed in claim 6 which further comprises receiving user input specifying a value for said weighting parameter.

8. A method as claimed in claim 7 which further comprises receiving user input specifying an ordered sequence of values for said weighting parameter.

9. A method as claimed in claim 8 which further comprises:

(i) recomputing said constructed image on the basis of the ordered sequence of values for the weighting parameter; and

(ii) outputting said recomputed constructed images to a display in real time.

10. A method as claimed in claim 8 wherein said user input is received via a graphical user interface comprising a sliding graphical input tool.

11. A method as claimed in claim 1 wherein said indication of a degree of contrast in the source image acts to detect the absence of edges such that in use, the modification is arranged to preserve absence of edges in the constructed image.

12. A method as claimed in claim 1 wherein said modification on the basis of an indication of the degree of contrast comprises using a fragility term.

13. A method as claimed in claim 1 wherein said boundary is identified as a result of user input.

14. A method as claimed in claim 1 wherein said boundary is identified as a result of automatic segmentation computation.

15. A method as claimed in claim 14 wherein said automatic segmentation computation comprises a graph cut method.

16. A method as claimed in claim 3 wherein said source images are arranged to partially overlap and wherein said boundary is within the region of overlap between those images.

17. A method as claimed in claim 4 wherein said minimization is carried out using a numerical method based on finite difference approximation of gradients.

18. A computer program stored on a computer readable medium and arranged to carry out the method of claim 1.

19. An apparatus for blending a first image and a second image to form a constructed image, said apparatus comprising:

—

(i) an image combiner arranged to combine at least part of the first image and at least part of the second image to form the constructed image;

(ii) a processor arranged to identify a boundary in the constructed image being between a region in the constructed image formed from the first image and a region in the constructed image formed from the second image;

(iii) an image processor arranged to modify the constructed on the basis of the gradient of at least part of either of the first and second images; and

wherein said image processor is further arranged to modify the constructed image on the basis of one or more of;

a color preservation term and an indication of a degree of contrast in at least one of the first and second images.

20. An apparatus as claimed in claim 19 wherein said processor is arranged to receive user input identifying said boundary.