Method and system for enhancing portrait images
First Claim
1. A method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
- (a) acquiring a digital image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, eyebrows, nose, mouth, and hair;
(c) using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, eyebrows, nose, mouth, neck and hair regions;
(d) determining one or more facially relevant characteristics of the different regions;
(e) based on the facially relevant characteristics of the different regions, selecting (1) two or more enhancement filters each customized especially for a particular region and (2) the default parameters for the enhancement filters; and
(f) executing the enhancement filters on the particular regions, thereby producing an enhanced digital image from the digital image.
6 Assignments
0 Petitions
Accused Products
Abstract
A retouching method for enhancing an appearance of a face located in a digital image involves acquiring a digital image containing one or more faces and detecting a location of facial feature points in the faces, where the facial feature points include points identifying salient features such as skin, eyes, eyebrows, nose, mouth, and hair. The location of the facial feature points are used to segment the face into different regions such as skin, eyes, eyebrows, nose, mouth, neck and hair regions. Facially relevant characteristics of the different regions are determined and, based on these facially relevant characteristics, an ensemble of enhancement filters are selected, each customized especially for a particular region, and the default parameters for the enhancement filters are selected. The enhancement filters are then executed on the particular regions, thereby producing an enhanced digital image from the digital image.
-
Citations
78 Claims
-
1. A method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
-
(a) acquiring a digital image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, eyebrows, nose, mouth, and hair;
(c) using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, eyebrows, nose, mouth, neck and hair regions;
(d) determining one or more facially relevant characteristics of the different regions;
(e) based on the facially relevant characteristics of the different regions, selecting (1) two or more enhancement filters each customized especially for a particular region and (2) the default parameters for the enhancement filters; and
(f) executing the enhancement filters on the particular regions, thereby producing an enhanced digital image from the digital image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A user-interactive retouching method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
(a) acquiring and displaying a digital image containing one or more faces;
(b) detecting facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, nose, mouth, and hair;
(c) displaying the location of the facial feature points by overlaying the facial feature points on the displayed digital image;
(d) providing an optional user-initiated adjustment of the location of the facial feature points;
(e) using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, nose, mouth and hair regions;
(f) determining one or more facially relevant characteristics of the different regions;
(g) based on the facially relevant characteristics of the different regions, selecting (1) two or more enhancement filters customized especially for each region and (2) the default parameters for the enhancement;
(h) executing the enhancement filters on the regions, thereby producing an enhanced digital image from the digital image;
(i) displaying the enhanced digital image;
(j) providing an optional user-initiated approval of the displayed enhanced digital image, whereby if the image is not approved at least one of steps (d), (e), (f) and (g) are repeated with adjusted regions or adjustments to the default parameters of the filters and step (h) is re-executed on the segmented regions; and
(k) applying an image utilization process to the enhanced digital image. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22)
-
23. A retouching method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
(a) acquiring a digital image containing one or more faces;
(b) detecting facial feature points in the one or more faces, said facial feature points including points identifying salient facial features including eyes, nose, mouth, or hair but not including skin;
(c) using the location of the salient facial feature points to identify a skin region of the face, wherein said skin region excludes the salient facial features;
(d) based on the identified skin region, selecting an enhancement filter and one or more default parameters for the filter for correcting a predefined skin type feature; and
(e) executing the enhancement filter on the skin region, thereby producing an enhanced digital image from the source digital image. - View Dependent Claims (24, 25, 26)
-
27. A system for enhancing an appearance of one or more faces appearing in a digital image, said system comprising:
-
means for detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, eyebrows, nose, mouth, and hair;
means for using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, eyebrows, nose, mouth, neck and hair regions;
means for determining one or more facially relevant characteristics of the different regions;
means for, based on the facially relevant characteristics of the different regions, selecting (1) two or more enhancement filters each customized especially for a particular region and (2) the default parameters for the enhancement filters; and
means for executing the enhancement filters on the particular regions, thereby producing an enhanced digital image from the digital image. - View Dependent Claims (28, 29, 30, 31)
-
-
32. A user-interactive retouching system for enhancing an appearance of one or more faces appearing in a digital image, said system comprising:
-
means for detecting facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, nose, mouth, and hair;
a display for displaying the location of the facial feature points by overlaying the facial feature points on the displayed digital image;
a user interface for providing an optional user-initiated adjustment of the location of the facial feature points;
means for using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, nose, mouth and hair regions;
means for determining one or more facially relevant characteristics of the different regions;
means, based on the facially relevant characteristics of the different regions, for selecting (1) two or more enhancement filters customized especially for each region and (2) the default parameters for the enhancement;
means for executing the enhancement filters on the regions, thereby producing an enhanced digital image from the source digital image, and displaying the enhanced digital image, whereby the user interface provides an optional user-initiated approval of the displayed enhanced digital image and, if the image is not approved, consequent adjustment of the location of feature points or default parameters.
-
-
33. An automatic retouching method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
-
(a) acquiring a digital image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including the skin;
(c) providing a generic probability map for a neck region;
(d) scaling the generic probability map for the neck region to fit the feature points generated in step (b), thereby providing a scaled probability map for the neck region; and
(e) based on the scaled probability map for the neck region, using an enhancement filter to enhance at least the texture or the skin tone of the neck region, thereby producing an enhanced digital image from the digital image. - View Dependent Claims (34, 35, 36)
-
-
37. An automatic retouching method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
-
(a) acquiring a digital image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, including feature points identifying a chin line;
(c) segmenting the digital image into pixels within a skin region, thereby producing a skin probability distribution;
(d) using the skin probability distribution and pixels below the chin line to generate a skin color based probability map for the neck region; and
(e) based on the skin color based probability map for the neck region, using an enhancement filter to enhance at least the texture or the skin tone of the neck region, thereby producing an enhanced digital image from the digital image. - View Dependent Claims (38, 39)
-
-
40. A method for enhancing the skin texture of a face appearing in a digital image, said method comprising the steps of:
-
(a) generating a luminance image from the digital image;
(b) using a valley/peak detector to detect skin features in the luminance image;
(c) classifying the skin features according to their feature-based characteristics;
(d) selecting relevant skin features for modification;
(e) modifying the relevant skin features using an adaptive interpolation procedure, thereby producing a modified image; and
(f) blending the digital image and the modified image to produce an enhanced image. - View Dependent Claims (41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56)
-
-
57. A method for enhancing the skin tone of a face appearing in a digital image, said method comprising the steps of:
-
(a) generating a blurred version of the digital image;
(b) generating a blending coefficient; and
(c) applying a blending operation to the digital image and its blurred version to generate an enhanced image with improved skin tone, wherein the blending operation uses the blending coefficient to combine the digital image and its blurred version. - View Dependent Claims (58, 59, 60, 61, 62)
-
-
63. A method for enhancing the luminance and chrominance of facial regions in a face located in a digital image, said method comprising the steps of:
-
(a) acquiring a digital color image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including at least one of eye and teeth regions;
(c) using the facial feature points to generate one or more masks identifying pixels corresponding to at least one of the eye and teeth regions;
(d) using a valley/peak detector to generate a valley/peak map from the digital image;
(e) using the valley/peak map from the digital image to calculate one or more scaling factors for luminance and chrominance values of the pixels within the masks; and
(f) applying the scaling factors to pixels within the masks to modify the luminance and chrominance values of at least one of the eye and teeth regions of the face. - View Dependent Claims (64, 65)
-
-
66. An automatic retouching method for enhancing an appearance of a face located in a digital image, said method comprising the steps of:
(a) acquiring a digital image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying salient features including one or more of skin, eyes, eyebrows, nose, mouth, and hair;
(c) using the location of the facial feature points to segment the face into different regions, said different regions including one or more of skin, eyes, eyebrows, nose, mouth, neck and hair regions;
(d) determining one or more facially relevant characteristics of the different regions;
(e) based on the determined facially relevant characteristics of the different regions of the face, selecting a plurality of enhancement filters each customized especially for a particular region, said filters including texture and skin tone filters, and at least one of eye, teeth, and shape filters; and
(f) executing the enhancement filters on the particular regions in a predetermined order wherein the texture enhancement filter is always executed before the skin tone filter, thereby producing an enhanced digital image from the digital image. - View Dependent Claims (67)
-
68. A graphical user interface method for use in an automatic retouching method for enhancing the appearance of a face, said method comprising the steps of:
-
(a) acquiring and displaying a digital image containing one or more faces which show skin features in an original state;
(b) using one or more enhancement filters to perform an enhancement of a skin feature, thereby producing a displayed digital image showing the skin feature in an enhanced state;
(c) providing a tool for undoing and redoing the enhancement performed on the skin feature; and
(d) using the tool to toggle the skin feature between its enhanced and original state. - View Dependent Claims (69)
-
-
70. A graphical user interface method for use in a retouching method for enhancing the appearance of a face, said method comprising the steps of:
-
(a) acquiring and displaying a digital image containing one or more faces;
(b) generating a skin feature map representative of the skin features on the one or more faces;
(c) using one or more enhancement filters to perform an enhancement of a skin feature;
(d) providing a pointing tool for pointing at a skin feature; and
(e) in response to using the pointing tool, referencing the skin feature map and initiating the enhancement of the skin feature being pointed at.
-
-
71. A computer storage medium having instructions stored therein for causing a computer to perform the method of claim 71.
-
72. A graphical user interface method for use in a retouching method for enhancing the appearance of a face, said method comprising the steps of:
-
(a) acquiring and displaying a digital image containing one or more faces in an original state;
(b) using a plurality of enhancement filters incorporating respective parameters for performing a plurality of enhancements of the digital image, thereby producing a displayed digital image as an enhanced image;
(c) providing a plurality of slider tools for varying the respective parameters of the enhancement filters; and
(d) providing a combination tool for varying the effect of all of the respective parameters at once. - View Dependent Claims (73, 74, 75)
-
-
76. A method for enhancing the shape of features appearing in a face located in a digital image, said method comprising the steps of:
-
(a) acquiring a digital image containing one or more faces;
(b) detecting a location of facial feature points in the one or more faces, said facial feature points including points identifying one or more salient features amenable to shape adjustment;
(c) using the facial feature points to determine two or more source points that are positioned relative to the salient feature;
(d) using the facial feature points to determine two or more destination points which determine the desired change of the salient feature; and
(e) warping the shape of the salient feature by using the source points and the destination points. - View Dependent Claims (77, 78)
-
Specification