Skin diagnostic and image processing methods

• US 9,101,320 B2
• Filed: 04/09/2013
• Issued: 08/11/2015
• Est. Priority Date: 04/09/2013
• Status: Active Grant

• Current Assignee: ELC Management LLC (The Estee Lauder Companies Inc.)
• Original Assignee: ELC Management LLC (The Estee Lauder Companies Inc.)
• Inventors: Cummins, Phillip, Vanderover, Garrett W., Jorgensen, Lise W., Adams, David M.
• Primary Examiner(s): Jung, Unsu
• Assistant Examiner(s): HUYNH, PHONG KY
• Application Number:
  US13/859,359
• Publication Number:
  US 20140323873A1
• Time in Patent Office: 854 Days
• Field of Search: 600/407, 600/473, 600/476
• US Class Current: 1/1
• CPC Class Codes: A45D 2044/007 Devices for determining the condition of hair or skin or for selecting the appropriate cosmetic or hair treatment
• CPC Class Codes: A45D 44/005 for selecting or displaying personal cosmetic colours or hairstyle
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/442 Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

Skin diagnostic techniques employed in conjunction with image processing techniques. For example, a method includes performing one or more diagnostic operations on at least one portion of a user skin image to generate user skin image data, wherein the diagnostic operations are associated with an identified skin-related application. The user skin image data is processed in accordance with the identified skin-related application. The processing includes identifying one or more sets of skin image data in a database that correspond to the user skin image data based on parameters specified by the skin-related application, and determining at least one image processing filter based on the sets of identified skin image data. The image processing filter is applied to the user skin image to generate a simulated user skin image.

Claims

1. A method comprising steps of:

performing one or more diagnostic operations on at least one portion of a user skin image to generate user skin image data, wherein the one or more diagnostic operations are associated with an identified skin-related application;

processing the user skin image data in accordance with the identified skin-related application, wherein said processing comprises;

identifying one or more sets of skin image data in a database that correspond to the user skin image data based on one or more parameters specified by the skin-related application; and

determining at least one image processing filter based on the one or more sets of identified skin image data from the database; and

applying the at least one image processing filter to the at least one portion of the user skin image to generate a simulated user skin image;

wherein the user skin image comprises a parallel light image component and a perpendicular light image component, and the step of applying the at least one image processing filter to the at least one portion of the user skin image further comprises determining component contribution amounts for one or more of the parallel image light component and the perpendicular image light component based on at least a portion of the identified skin image data;

wherein one or more of the above steps are carried out by at least one computing device comprising a processor and a memory configured to implement the at least one image processing filter; and

wherein the parallel light image component comprises a specular component and first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component.

2. The method of claim 1, further comprising capturing a user skin image.

3. The method of claim 2, wherein capturing a user skin image further comprises capturing at least one visible image of a portion of skin of the user.

4. The method of claim 2, wherein capturing a user skin image further comprises capturing at least one infrared image of a portion of skin of the user.

5. The method of claim 2, wherein capturing a user skin image further comprises capturing at least one ultraviolet image of a portion of skin of the user.

6. The method of claim 2, further comprising providing the captured user skin image as a non-polarized image component, a parallel light image component and a perpendicular light image component.

7. The method of claim 1, further comprising capturing user information via a text capture device.

8. The method of claim 1, further comprising enabling selection of the skin-related application from a plurality of skin-related applications.

9. The method of claim 1, wherein performing one or more diagnostic operations further comprises determining user red, green, blue (RGB) color space values for one or more areas of the at least one portion of the user skin image.

10. The method of claim 9, further comprising calculating average RGB color space values of the user RGB color space values for the one or more areas of the at least one portion of the user skin image.

11. The method of claim 10, further comprising converting the average RGB color space values to user L, a, b color space values.

12. The method of claim 11, wherein identifying one or more sets of skin image data in the database that correspond to the user skin image data comprises identifying one or more L, a, b color space values in the database that approximately match the user L, a, b color space values.

13. The method of claim 12, wherein determining at least one image processing filter comprises setting the at least one image processing filter based on the one or more identified L, a, b color space values from the database.

14. The method of claim 12, further comprising accessing a look-up table for identifying one or more L, a, b color space values from one or more spectral feature values.

15. The method of claim 1, wherein the user skin image further comprises a non-polarized image component.

16. The method of claim 15, wherein determining at least one image processing filter based on the one or more sets of identified skin image data from the database comprises determining a non-polarized image filter, a parallel light image filter and a perpendicular light image filter.

17. The method of claim 16, wherein applying the at least one image processing filter to the at least one portion of the user skin image to generate a simulated user skin image comprises:

applying the non-polarized image filter to the non-polarized image component to generate a proscenium image component;

applying the parallel light image filter to the parallel light image component to generate a simulated parallel light image component; and

applying the perpendicular light image filter to the perpendicular light image component to generate a simulated perpendicular light image component.

18. The method of claim 17, further comprising combining the simulated parallel light image component and the simulated perpendicular light image component to generate a base simulated user image for the skin-related application.

19. The method of claim 18, further comprising combining the base simulated user image with the proscenium image component to generate the simulated user skin image.

20. The method of claim 1, further comprising outputting the simulated user skin image to a display.

21. The method of claim 1, further comprising enabling selection of the at least one portion of a user skin image via manipulation of a graphical user interface.

22. The method of claim 21, wherein manipulation of the graphical user interface comprises executing a before/after swiper tool across the user skin image.

23. The method of claim 21, wherein manipulation of the graphical user interface comprises executing a contrast operation on the user skin image.

24. The method of claim 21, wherein manipulation of the graphical user interface comprises executing a lighting simulation operation on the user skin image.

25. The method of claim 21, wherein manipulation of the graphical user interface comprises shrinking or enlarging the at least one portion of the user skin image via a sizing feature on the graphical user interface.

26. The method of claim 1, wherein the skin-related application comprises one of a foundation matching application, a lines and wrinkles application, a skin lightening application, a skin de-yellowing application, and a de-aging application.

27. The method of claim 1, wherein the skin-related application is identified via specification of at least one of a skin product and a skin product category.

28. The method of claim 1, wherein the one or more parameters specified by the skin-related application comprises a user demographic parameter.

29. The method of claim 1, wherein the one or more parameters specified by the skin-related application comprises a severity-time parameter.

30. A method comprising steps of:

obtaining a user skin image; and

applying a set of image processing filters to polarized components associated with at least a portion of the user skin image to modify the user skin image, wherein the set of image processing filters is controlled by data identified based on a diagnosis of a skin condition from at least a portion of the user skin image, and wherein the identified data represents a previously-determined effect over time of a skincare product usage such that the modification to the user skin image visually simulates the subsequent effect over time of the skincare product usage;

wherein the polarized components associated with the user skin image comprise a parallel light image component and a perpendicular light image component, and modifying the user skin image to visually simulate the subsequent effect over time of the skincare product usage further comprises determining component contribution amounts for one or more of the parallel image light component and the perpendicular image light component based on at least a portion of the identified data;

wherein one or more of the above steps are carried out by at least one computing device comprising a processor and a memory configured to implement the set of image processing filters; and

wherein the parallel light image component comprises a specular component and first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component.

31. The method of claim 30, further comprising presenting the modified user skin image to a user from whom the user skin image is obtained.

32. The method of claim 30, further comprising generating a sequence of images representing the modification of the user skin image over a plurality of sequential timepoints, and presenting the sequence of images to a user from whom the user skin image is obtained.

33. The method of claim 30, wherein the first portion of the undertone component and the second portion of the undertone component each represent half of the undertone component.

34. The method of claim 30, wherein modification of the user skin image further comprises adjusting over time, based on at least a portion of the identified data, one or more of the parallel light image component and the perpendicular light image component associated with the user skin image.

35. The method of claim 34, wherein one or more of the parallel light image component and the perpendicular light image component associated with the user skin image are adjusted as a function of an amount of change at one or more timepoints given by the identified data.

36. The method of claim 35, wherein the perpendicular light image component is adjusted by the change given in the identified data, and the parallel light image component is adjusted by a fraction of the change given in the identified data.

37. The method of claim 36, wherein the change given by the identified data represents a percentage change in melanin over time.

38. The method of claim 30, wherein the diagnosis of the skin condition is associated with a skin-related application comprising one of a foundation matching application, a lines and wrinkles application, a skin lightening application, a skin de-yellowing application, and a de-aging application.