Method for generating soft-edge mattes for visual elements of images

• US 5,422,986 A
• Filed: 05/12/1993
• Issued: 06/06/1995
• Est. Priority Date: 05/12/1993
• Status: Expired due to Term

• Current Assignee: Pacific Data Images LLC (Comcast Corporation)
• Original Assignee: Pacific Data Images, Inc. (Comcast Corporation)
• Inventors: Neely, Shawn R.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US08/061,492
• Time in Patent Office: 755 Days
• Field of Search: 395/141, 395/142, 395/143, 395/126, 395/155, 395/161, 345/121, 345/133, 345/136
• US Class Current: 345/426
• CPC Class Codes: G06T 15/60 Shadow generation

Abstract

Method and apparatus are disclosed for generating mattes of film, video, or other visual image elements, in which the operator may specify soft-edge regions for the extended visible fall-off of certain difficult areas such as hair, clouds, shadows, and blurred objects. The elements may be either still or in motion, and automatic interpolation of keyframe regions may be used to simplify the generation of in-between frames to create traveling soft-edge mattes.

Claims

1. In an image comprising a plurality of regions, including a first region including a visual element, a second region adjacent to the first region, and a third region adjacent to the second region, the second region disposed between the first and third regions, selected portions of the regions having a continuous luminance transition therebetween, a method for generating a soft-edge matte in the second region for the visual element in the first region, comprising the steps of:

(a) specifying an inner outline to define an inner region of the image having a plurality of pixels, the inner region corresponding to the first region;

(b) specifying a soft-edge outline to define a soft-edge region of the image having a plurality of pixels, the soft-edge region including an area of the image between the inner outline and the soft-edge outline, the soft-edge region corresponding to the second region;

(c) establishing representative values for the pixels of the inner region; and

(d) establishing representative values for the pixels of the soft-edge region.

2. A method for generating soft-edge mattes for a visual element of an image according to claim 1, further comprising the step of storing representations of the specified inner and soft-edge outlines in a database.

3. The method of claim 1, further comprising the step of displaying the inner outline and the soft-edge outline using distinguishing visual attributes.

4. The method of claim of claim 3, wherein the displaying step comprises the substeps of:

displaying the inner outline using a first color; and

displaying the soft-edge outline using a second color.

5. The method of claim 1, wherein step (d) includes establishing a succession of varying representative values forming a gradient for the pixels of the soft-edge region.

6. The method of claim 5, wherein step (d) comprises, for each pixel within the soft-edge region:

(d.1) determining the relative location of the pixel with respect to the inner outline and the soft-edge outline; and

(d.2) responsive to the results of step (d.1), generating a representative value for the pixel.

7. The method of claim 6, wherein step (d.2) comprises:

(d.2.1) selecting a first point on the inner outline;

(d.2.2) selecting a second point on the inner outline, the second point being located at a distance of approximately one pixel width away from the first point;

(d.2.3) determining a midpoint between the first and second points;

(d.2.4) determining a third point and a fourth point, the third and fourth points lying along the soft-edge outline, the third point having a parametric distance from a reference point on the soft-edge outline that is substantially equal to the parametric distance of the first point from a corresponding reference point on the inner outline, and the fourth point having a parametric distance from the reference point that is substantially equal to the parametric distance of the second point from the corresponding reference point;

(d.2.5) determining a midpoint between the third and fourth points;

(d.2.6) determining a plurality of points, the points being spaced at a distance of approximately one pixel width and lying along a line from the midpoint between the first and second points to the midpoint between the third and fourth points; and

(d.2.7) determining a representative value for all pixels fully or partially contained in a circular area within a small distance of each of said points.

8. The method of claim 5, wherein step (d) comprises:

(d.1) selecting a first point on the inner outline;

(d.2) selecting a second point on the inner outline, the second point being located at a distance of approximately one pixel width from the first point;

(d.3) determining a third point and a fourth point, the third and fourth points each lying on the soft-edge outline, the third point having a parametric distance from a reference point on the soft-edge outline that is substantially equal to the parametric distance of the first point from a corresponding reference point on the inner outline, and the fourth point having a parametric distance from the reference point that is substantially equal to the parametric distance of the second point from the corresponding reference point;

(d.4) establishing representative values for the pixels lying within a quadrilateral formed by the first, second, third and fourth points in accordance with gradient polygon fill logic; and

(d.5) repeating steps (d.1) to (d.4) until all representative values for the pixels in the soft-edge region are established.

9. The method of claim 1, wherein the specifying of the inner outline and the soft-edge outline of steps (a) and (b) are manually performed.

10. The method of claim 1, wherein step (a) comprises:

(a.1) specifying a first set of control points around the visual element; and

(a.2) constructing the inner outline based on the first set of control points;

and wherein step (b) comprises;

(b.1) specifying a second set of control points around the visual element; and

(b.2) constructing the soft-edge outline based on the second set of control points.

11. The method of claim 10, wherein:

step (a.2) comprises determining the inner outline based on the first set of control points using cubic piece-wise polynomial splines; and

step (b.2) comprises determining the soft-edge outline based on the second set of control points using cubic piece-wise polynomial splines.

12. The method of claim 1, wherein step (a) comprises:

(a.1) specifying a set of control points around the visual element; and

(a.2) constructing the inner outline based on the set of control points;

(b.1) constructing the soft-edge outline based on the set of control points;

(b.2) selecting at least one of the control points; and

(b.3) moving the selected control point to alter the soft-edge outline without changing the inner outline.

13. The method of claim 1, further comprising the step, before performing step (d), of adjusting the characteristics of the soft-edge region.

14. The method of claim 13, wherein the adjusting step comprises specifying falloff contours within the soft-edge region.

15. A method for generating a soft-edge matte for a visual element of a moving image, comprising the steps of:

(a) defining at least two keyframes separated by a plurality of in-between frames;

(b) for each keyframe;

(b.1) specifying a keyframe inner outline to define a keyframe inner region having a plurality of pixels;

(b.2) specifying a keyframe soft-edge outline to define a keyframe soft-edge region having a plurality of pixels, the keyframe soft-edge region including an area between the keyframe inner outline and the keyframe soft-edge outline;

(b.3) establishing representative values for the pixels of the keyframe inner region; and

(b.4) establishing representative values for the pixels of the keyframe soft-edge region;

(c) for each in-between frame;

(c.1) interpolating an inner outline for the in-between frame from the keyframe inner outlines to define an inner region for the in-between frame, the inner region for the in-between frame having a plurality of pixels;

(c.2) interpolating a soft-edge outline for the in-between frame from the keyframe soft-edge outlines to define a soft-edge region for the in-between frame, the soft-edge region for the in-between frame having a plurality of pixels and including an area between the inner outline for the in-between frame and the soft-edge outline for the in-between frame;

(c.3) establishing representative values for the pixels of the inner region for the in-between frame; and

(c.4) establishing representative values for the pixels of the soft-edge region for the in-between frame.

16. The method of claim 15, wherein each of steps (c.1) and (c.2) is performed using cubic piece-wise polynomial spline interpolation.

17. The method of claim 15, further comprising:

(d) storing representations of the specified keyframe inner and soft-edge outlines in a database; and

(e) storing representations of the interpolated inner and soft-edge outlines for the in-between frames in a database.

18. In an image comprising a plurality of regions, including a first region including a visual element, a second region adjacent to the first region, and a third region adjacent to the second region, the second region disposed between the first and third regions, selected portions of the regions having a continuous luminance transition therebetween, a method for generating a soft-edge boundary between the second and third regions for an image-processing operation, comprising the steps of:

(a) specifying an inner outline to define an inner region having a plurality of pixels, the inner region corresponding to the first region;

(b) specifying a soft-edge outline to define a soft-edge region having a plurality of pixels, the soft-edge region including an area between the inner outline and the soft-edge outline, the soft-edge region corresponding to the second region;

(c) establishing representative values for the pixels of the inner region using the image processing operation; and

(d) establishing representative values for the pixels of the soft-edge region using the image processing operation.

19. In an image comprising a plurality of regions, including a first region including a visual element, a second region adjacent to the first region, and a third region adjacent to the second region, the second region disposed between the first and third regions, selected portions of the regions having a continuous luminance transition therebetween, an apparatus for generating a soft-edge matte for the visual element in the first region, comprising:

an input device, for accepting input specifying an inner outline to define an inner region of the image having a plurality of pixels, the inner region corresponding to the first region, and for accepting input specifying a soft-edge outline to define a soft-edge region of the image having a plurality of pixels, the soft-edge region including an area of the image between the inner outline and the soft-edge outline, and corresponding to the second region;

means coupled to the input device for establishing representative values for the pixels of the inner region;

means coupled to the input device for establishing representative values for the pixels of the soft-edge region; and

a display device coupled to the input device for displaying the inner outline, the soft-edge outline, and the soft-edge region.

20. The apparatus of claim 19, further comprising a database storage device, coupled to the input device, for storing representations of the specified inner and soft-edge outlines in a database.

21. An apparatus for generating a soft-edge matte for a visual element of a moving image, comprising:

keyframe definition means for defining at least two keyframes separated by a plurality of in-between frames;

an input device, coupled to the keyframe-definition means, for specifying for each keyframe a keyframe inner outline to define a keyframe inner region having a plurality of pixels, and for specifying a keyframe soft-edge outline to define a keyframe soft-edge region having a plurality of pixels, the keyframe soft-edge region including an area between the keyframe inner outline and the keyframe soft-edge outline;

means coupled to the input device for establishing representative values for the pixels of the inner region for each keyframe;

means coupled to the input device for establishing representative values for the pixels of the soft-edge region for each keyframe;

inner outline interpolating means, coupled to the input device, for interpolating for each in-between frame an inner outline for the in-between frame from the keyframe inner outlines to define an inner region for the in-between frame, the inner region for the in-between frame having a plurality of pixels;

soft-edge outline interpolating means, coupled to the input device, for interpolating for each in-between frame a soft-edge outline for the in-between frame from the keyframe soft-edge outlines to define a soft-edge region for the in-between frame, the soft-edge region for the in-between frame having a plurality of pixels and including an area between the inner outline for the in-between frame and the soft-edge outline for the in-between frame;

means coupled to the input device for establishing representative values for the pixels of the inner region for each in-between frame; and

means coupled to the input device, for establishing representative values for the pixels of the soft-edge region for each in-between frame.

22. The apparatus of claim 21, further comprising a database storage device, coupled to the input device and to the interpolating means, for storing representations of the specified inner and soft-edge outlines for each keyframe and each in-between frame.