METHOD AND APPARATUS FOR SIMULATION OF FACIAL SKIN AGING AND DE-AGING

• US 20080212894A1
• Filed: 03/02/2007
• Published: 09/04/2008
• Est. Priority Date: 03/02/2007
• Status: Active Grant

• Current Assignee: The Procter & Gamble Company
• Original Assignee: The Procter & Gamble Company
• Inventors: Demirli, Ramazan, Hillebrand, Greg George
• Granted Patent: US 8,290,257 B2
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/442 Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

A novel method and system is disclosed to realistically simulate the progress or worsening of facial skin features that contribute to the overall look and condition of the skin. The method utilizes two close-up photographs of the face, one is captured with a digital camera in standard white light, and the other is captured with the same camera in UV light. Then, the method processes these images to simulate the progress or worsening of the major skin features: hyperpigmented spots, wrinkles and small texture features. The worsening of these features simulates facial skin aging due to prolonged exposures to sunlight, biological aging or degradation of the skin health. The progress of these features simulates the improvement of facial skin in terms of overall look and healthiness as though the patient has gone through a treatment. Therefore, the present invention discloses a series of methods that are useful in dermatology, cosmetics and computer animations.

Claims

1. A method of manipulating a facial image so as to simulate spot aging, the method comprising:

transforming the facial image to a color space format having intensity and color components;

obtaining a subsurface version of the facial image;

detecting subsurface spots based on the subsurface version of the facial image;

selecting a set of the subsurface spots;

generating a contrast image of the set of subsurface spots; and

modifying at least one of the intensity and color components of the facial image based on the contrast image, thereby generating a spot aged facial image.

2. The method of claim 1, comprising:

applying a mask to the subsurface version of the facial image.

3. The method of claim 1, comprising:

registering the subsurface and standard versions of the facial image.

4. The method of claim 1, wherein the subsurface and standard versions of the facial image are substantially registered with respect to each other.

5. The method of claim 1, comprising:

dilating the contrast image so as to enlarge each of the set of subsurface spots.

6. The method of claim 1, wherein detecting subsurface spots includes:

noise filtering the subsurface version of the facial image;

estimating a background illumination of the noise filtered subsurface version of the facial image;

removing the background illumination estimate from the noise filtered subsurface version of the facial image, thereby generating a contrast image;

segmenting the contrast image based on a contrast threshold, thereby generating a set of segmented objects; and

eliminating segmented objects smaller than a minimum area threshold.

7. The method of claim 6, wherein detecting subsurface spots includes:

smoothing the set of segmented objects.

8. The method of claim 1, wherein selecting a subset of UV spots includes decimating the detected UV spots so that the subset of UV spots is sparse.

9. The method of claim 1, wherein dilating the contrast image includes:

filtering the contrast image with a Gaussian filter.

10. The method of claim 9, wherein the contrast image has a working resolution of approximately 220 ppi and the Gaussian filter has a variance of approximately 5.

11. The method of claim 1, wherein modifying the intensity and color components of the facial image includes:

weighting the contrast image; and

adding the weighted contrast image to each of the intensity and color components of the facial image.

12. The method of claim 1, wherein the color space format having intensity and color components is an LAB format having a luminosity component L and color components A and B.

13. A method of manipulating a facial image so as to simulate spot de-aging, the method comprising:

transforming the facial image to a color space format having intensity and color components;

detecting spots in the facial image;

generating a contrast image of the spots; and

modifying the intensity component of the facial image based on the contrast image, thereby generating a spot de-aged facial image.

14. The method of claim 13, comprising:

applying a mask to the facial image before detecting spots.

15. The method of claim 13, wherein detecting spots includes:

noise filtering the facial image;

estimating a background illumination of the noise filtered facial image;

removing the background illumination estimate from the noise filtered facial image, thereby generating a contrast image;

generating a color difference image;

segmenting the color difference image based on a threshold, thereby generating a set of segmented objects; and

eliminating false spots.

16. The method of claim 15, wherein detecting spots includes:

smoothing the set of segmented objects.

17. The method of claim 15, wherein eliminating false spots includes:

removing segmented objects smaller than a minimum area threshold.

18. The method of claim 15, wherein eliminating false spots includes:

removing segmented objects based on at least one of an area, aspect ratio, solidity, major-axis-length, minor-axis-length, eccentricity, and extent.

19. The method of claim 13, wherein modifying the intensity component of the facial image includes:

weighting the contrast image; and

adding the weighted contrast image to the intensity component of the facial image.

20. The method of claim 13, wherein the color space format having intensity and color components is an LAB format having a luminosity component L and color components A and B.

21. A method of manipulating a facial image so as to simulate time-related wrinkle changes, the method comprising:

transforming the facial image to a color space format having intensity and color components;

detecting wrinkles in the facial image;

generating a contrast image of the wrinkles; and

modifying the intensity component of the facial image based on the contrast image, thereby generating a wrinkle changed facial image.

22. The method of claim 21, comprising:

applying a mask to the facial image before detecting spots.

23. The method of claim 21, wherein detecting wrinkles includes:

noise filtering the facial image;

estimating a background illumination of the noise filtered facial image;

removing the background illumination estimate from the noise filtered facial image, thereby generating a contrast image;

selecting dark regions of the contrast image;

performing ridge detection on the dark regions of the contrast image;

generating a ridge map based on the ridge detection; and

applying thresholding to the ridge map to determine wrinkle candidates.

24. The method of claim 23, wherein performing ridge detection includes:

convoluting the contrast image with directional basis filters having vertical, horizontal and diagonal orientations; and

analyzing the responses of the directional basis filters to generate a ridge strength and a ridge orientation.

25. The method of claim 21, comprising:

eliminating false wrinkles.

26. The method of claim 25, wherein eliminating false wrinkles includes:

removing segmented objects based on at least one of an area, aspect ratio, solidity, major-axis-length, minor-axis-length, eccentricity, and extent.

27. The method of claim 21, wherein modifying the intensity component of the facial image includes:

weighting the contrast image; and

adding the weighted contrast image to the intensity component of the facial image.

28. The method of claim 27, comprising:

at least one of dilating and extending the contrast image of the wrinkles before weighting, wherein the weighting has a positive value and the wrinkle changed facial image is a wrinkle aged facial image.

29. The method of claim 27, wherein the weighting has a negative value and the wrinkle changed facial image is a wrinkle de-aged facial image.

30. The method of claim 21, wherein the color space format having intensity and color components is an LAB format having a luminosity component L and color components A and B.

31. A method of manipulating a facial image so as to simulate time-related texture changes, the method comprising:

transforming the facial image to a color space format having intensity and color components;

removing a background intensity from the facial image, thereby generating a contrast image;

segmenting the contrast image into low-texture and high-texture regions;

generating a low-texture contrast and a high-texture contrast; and

modifying the intensity component of the facial image based on the low-texture and high-texture contrasts, thereby generating a texture changed facial image.

32. The method of claim 31, comprising:

applying a mask to the facial image before removing the background intensity.

33. The method of claim 31, wherein modifying the intensity component of the facial image includes:

weighting the low-texture contrast and the high-texture contrast; and

adding the weighted low-texture and high-texture contrasts to the intensity component of the facial image.

34. The method of claim 33, comprising:

dilating the low-texture contrast before weighting,

wherein the texture changed facial image is a texture aged facial image.

35. The method of claim 33, comprising:

shrinking the low-texture contrast before weighting; and

shrinking the high-texture contrast before weighting,

wherein the texture changed facial image is a texture de-aged facial image.

36. The method of claim 31, wherein the color space format having intensity and color components is an LAB format having a luminosity component L and color components A and B.

37. A method of manipulating a facial image so as to simulate time-related changes, the method comprising:

transforming the facial image to a color space format having intensity and color components;

manipulating the facial image to generate a spot change intensity component;

manipulating the facial image to generate a wrinkle change intensity component;

manipulating the facial image to generate a texture change intensity component; and

modifying the intensity component of the facial image based on a combination of the spot change intensity component, wrinkle change intensity component, and texture change intensity component, thereby generating a spot, wrinkle and texture changed facial image.

38. The method of claim 37, wherein the spot, wrinkle and texture changed facial image is a de-aged facial image.

39. The method of claim 37, comprising:

manipulating the facial image to generate a first spot change color component, and a second spot change color component;

modifying a first color component of the facial image based on the first spot change color component; and

modifying a second color component of the facial image based on the second spot change color component,

wherein the spot, wrinkle and texture changed facial image is an aged facial image.

40. The method of claim 37, comprising:

applying a mask to the facial image before manipulating the facial image.

41. The method of claim 40, wherein the mask includes a first mask for spot and wrinkle simulation and a second mask for texture simulation.

42. The method of claim 37, wherein the method generates a first spot, wrinkle and texture changed facial image which is an aged facial image, and a second spot, wrinkle and texture changed facial image which is a de-aged facial image, the method comprising:

selectively blending the aged or de-aged facial image with the facial image to generate a composite image,

wherein displaying the spot, wrinkle and texture changed facial image comprises displaying the composite image.

43. The method of claim 37, wherein the color space format having intensity and color components is an LAB format having a luminosity component L and color components A and B.

44. A method of detecting skin in a facial image, the method comprising:

determining a face region of the facial image;

transforming the facial image to a color space format having intensity and color components;

determining a pigmentation measure for each of a plurality of pixels of the facial image within the face region;

determining a threshold value; and

comparing the pigmentation measure for each pixel against the threshold value, wherein those pixels with a pigmentation measure below the threshold value are designated as skin.

45. The method of claim 44, wherein the color space format is an LAB format having a luminosity component L and color components A and B, and wherein the pigmentation measure is an Individual Typology Angle (ITA) measure.

46. The method of claim 45, comprising:

smoothing the L and B components of the facial image within the face region.

47. The method of claim 44, wherein determining a face region of the facial image includes:

transforming the facial image to LUX space; and

segmenting out the face region using a thresholding procedure.

48. The method of claim 47, wherein the thresholding procedure includes Otsu thresholding.

49. The method of claim 44, wherein the threshold value will segment the facial image into two classes having a minimum inter-class variance.

50. The method of claim 44, wherein the facial image is in an RGB color format.