System and method for image editing using an image stack

• US 20050030315A1
• Filed: 08/04/2003
• Published: 02/10/2005
• Est. Priority Date: 08/04/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Drucker, Steven M., Cohen, Michael, Colburn, R. Alex
• Granted Patent: US 7,519,907 B2
• Time in Patent Office: Days
• US Class Current: 345/538
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing

Abstract

A system and method for editing images. A simple but powerful image stack is employed in creating an enhanced image from a stack of registered images. This paradigm combines pixels using multi-image operations on the image stack. Image Stacks can help create group photographs, create high dynamic range images, combine images captured under different lighting conditions, remove unwanted objects from images, and combine images captured at different times and with different focal lengths.

Claims

1. A computer-implemented process for creating a composite image, comprising using a computer to perform the following process actions:

inputting an image stack comprising a stack of original images, wherein the pixel position of each original image in the image stack is defined in a three dimensional coordinate system, and wherein two dimensions describe the dimensions of each image in the image stack, and the third dimension describes the time an image was captured;

applying one or more filters to the image stack to create one or more new intermediate images;

selecting one of the original images in the image stack or an intermediate image as a source image; and

selecting pixels from the source image to be added to a composite image to create a final composite image.

2. The process of claim 1 wherein the process action of inputting an image stack comprises inputting an image stack wherein said original images are defined in a Cartesian coordinate system.

3. The process of claim 1 wherein said process action of applying a filter comprises applying a slice filter wherein said filter returns an image in said image stack.

4. The process of claim 1 wherein said process action of applying a filter comprises applying a median filter that returns the median pixel luminance along a span of the image stack, wherein a span is a set of image pixels at the same location in all images of the image stack.

5. The process of claim 1 wherein said process action of applying a filter comprises applying a maximum histogram filter that returns the pixel with the minimum sum of squared distances in red, green, blue color space to all other pixels along a span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

6. The process of claim 1 wherein said process action of applying a filter comprises applying a minimum histogram filter that returns the pixel with the maximum sum of squared distances in red, green, blue color space to all other pixels along a span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

7. The process of claim 1 wherein said process action of applying a filter comprising applying a maximum luminance filter that returns the pixel with the maximum luminance along a span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

8. The process of claim 1 wherein said process action of applying a filter comprises applying a maximum contrast filter that returns the pixel that has the highest contrast in a small neighborhood around it along a span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

9. The process of claim 8 wherein said small neighborhood is 5 by 5 pixels.

10. The process of claim 1 wherein said process action of applying a filter comprises applying a temporal smoothing filter that returns a weighted blend of a current image and the images before and after it, for a given span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

11. The process of claim 1 wherein said process action of applying a filter comprises applying a temporal sharpening filter that returns a pixel in the current image modified by the difference of the pixels in the images before and after the current image for a given span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

12. The process of claim 1 wherein said process action of applying a filter comprises applying a high dynamic range filter that combines different exposures over a span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

13. The process of claim 12 wherein the high dynamic range filter extracts exposure information associated with the original images that comprise the image stack.

14. The process of claim 1 wherein said process action of applying a filter comprises applying a surface filter that operates on a given surface through the image stack.

15. The process of claim 1 wherein said process action of applying a filter comprises applying a mat filter that produces a mat of a given portion of the image stack, wherein the mat is an image of transparency values that will modify the source image when it is used for creating said composite image.

16. A system for compositing digital images, the system comprising:

a general purpose computing device; and

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

input an image stack comprising a stack of original images, wherein the pixel position of each original image in the image stack is defined in an x, y, z coordinate system, where x and y define the width and height of each image in the image stack and z defines an image taken in time;

apply at least one filter to the image stack to create at least one intermediate images;

select at least one original image or at least one intermediate image to serve as a source image; and

select portions from the source image to be added to a composite image to create a final composite image.

17. The system of claim 16 wherein said module to apply a filter applies a slice (x,y) filter wherein for each (x, y) said filter returns a pixel at depth z from said image stack.

18. The system of claim 16 wherein said module to apply a filter applies a high dynamic range luminance filter that comprises sub-modules to:

compute a radiance value for each pixel in said image stack;

map the radiance value for each pixel to its luminance value by mapping red, green and blue channels to a display to match the luminance.

19. The system of claim 16 wherein said module to apply a filter applies a median filter that returns an image I(x,y,z) where z is the depth with the median pixel luminance along a span of the image stack, wherein a span is the set of image pixels at the same location in all images of the image stack.

20. The system of claim 16 wherein said module to apply a filter applies a maximum luminance filter that returns the pixel in a span at (x, y) with the maximum luminance, wherein a span is the set of image pixels at the same location in all images of the image stack.

21. The system of claim 16 wherein said module to apply a filter applies a maximum contrast filter that returns the pixel in the span at (x,y) that has highest contrast in a small neighborhood around it, wherein a span is the set of image pixels at the same location in all images of the image stack.

22. The system of claim 16 wherein said module to apply a filter applies a high dynamic range filter that comprises sub-modules to:

compute a radiance value for each pixel in said image stack;

map the radiance values for each pixel back to a set of display values via a tone-map.

23. The system of claim 22 wherein said tone-map is user-defined.

24. The system of claim 16 wherein said module to apply a filter applies a mat filter that produces a mat of a given portion of the image stack, wherein the mat is an image of transparency values that will modify the source image when it is used for painting pixels to said composite image.

25. The system of claim 24 wherein said module that applies a mat filter comprises sub-modules for:

inputting a first and a second image;

inputting a matting function; and

producing a mat of the first image with its transparency modified.

26. The system of claim 16 wherein said module to apply a filter applies a surface filter that extracts pixels laying on a surface embedded in the image stack.

27. The system of claim 26 wherein said surface embedded in the image stack is user-defined.

28. A computer-readable medium having computer-executable instructions for editing an image, said computer executable instructions comprising:

inputting an image stack comprising a stack of images, wherein the pixel position of each in the image stack is defined in a three dimensional coordinate system;

applying a filter to the image stack to create an intermediate images;

selecting one of the images in the image stack or an intermediate image to serve as a source image for creating a new composite image; and

selecting pixels from the source image to create a final composite image.

29. A graphical user interface for creating a composite image, comprising:

a source image window for displaying a source image derived from an image stack comprised of a stack of original images, wherein the pixel position of each original image in the image stack is defined in a three dimensional coordinate system, and wherein two dimensions of the three dimensional coordinate system describe the dimensions of each image in the image stack, and the third dimension describes a time a different image was captured; and

a composite image window wherein a composite image is displayed that is composed of parts of said source image that are transferred from the source image to the composite image by a user.

30. The graphical user interface of claim 29 wherein said user creates said intermediate image by applying at least one filter to the image stack and uses said intermediate image as a source image.

31. The graphical user interface of claim 29 wherein parts of said source image are transferred to said composite image by transferring pixels from the source image to the composite image.

32. The graphical user interface of claim 29 wherein said transfer of pixels from said source image to said composite image is based on a one-to-one correspondence regardless of whether the user initiates pixel transfer from the source image or the composite image.

33. The graphical user interface of claim 29 further comprising a paint brush function that transfers some pixels from said source image to said composite image.

34. The graphical user interface of claim 33 wherein a radius of pixel transfer is user-defined.

35. The graphical user interface of claim 29 further comprising a paint brush function that transfers all pixels from said source image to said composite image.

36. The graphical user interface of claim 33 wherein scaling the source image or the composite image scales paint brush function.

37. The graphical user interface of claim 36 wherein a highest resolution image available is used when transferring pixels using the paint brush function even when the source image or composite image are scaled.

38. The graphical user interface of claim 29 further comprising a paint brush function that transfers all pixels associated with a face from said source image to said composite image when said paint brush function is used to select a portion of said face.