Skin diagnostic and image processing systems, apparatus and articles

• US 9,256,963 B2
• Filed: 04/09/2013
• Issued: 02/09/2016
• Est. Priority Date: 04/09/2013
• Status: Active Grant

• Current Assignee: ELC Management LLC (The Estee Lauder Companies Inc.)
• Original Assignee: ELC Management LLC (The Estee Lauder Companies Inc.)
• Inventors: Cummins, Phillip, Vanderover, Garrett W., Jorgensen, Lise W., Adams, David M.
• Primary Examiner(s): Kim, Matt
• Assistant Examiner(s): Farhadian, FJ
• Application Number:
  US13/859,384
• Publication Number:
  US 20140304629A1
• Time in Patent Office: 1,036 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/004 adapted for image acquisition of a particular organ or body part A61B5/0082 takes precedence; arrangements for optical scanning A61B5/0062
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/742 using visual displays A61B5/339 takes precedence
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders

Abstract

Skin diagnostic techniques employed in conjunction with image processing techniques. For example, a user information module captures a user skin image. A graphical user interface enables selection of a skin-related application from a plurality of skin-related applications. A processor determines user skin image data from the user skin image, and identifies one or more sets of skin image data in the skin image database that correspond to the user skin image data based on one or more parameters specified by the skin-related application. The processor applies at least one image processing filter that corresponds to the one or more identified sets of skin image data from the skin image database to the user skin image to generate a simulated user skin image. An output display displays the simulated user skin image.

Claims

1. A system comprising:

a user information module, wherein the user information module captures a user skin image;

a graphical user interface, wherein the graphical user interface enables selection of a skin-related application from a plurality of skin-related applications;

a skin image database;

a processor coupled to the user information module, the graphical user interface, and the skin image database, wherein the processor;

determines user skin image data from the user skin image;

identifies one or more sets of skin image data in the skin image database that correspond to the user skin image data based on one or more parameters specified by the skin-related application; and

applies at least one image processing filter that corresponds to the one or more identified sets of skin image data from the skin image database to the user skin image to generate a simulated user skin image;

wherein the user skin image comprises a parallel light image component and a perpendicular light image component, and applying the at least one image processing filter comprises determining component contribution amounts for one or more of the parallel light image component and the perpendicular light image component based on at least a portion of the identified sets of skin image data;

wherein the parallel light image component comprises a specular component and a first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component; and

wherein the specular component represents light that reflects off the surface of the skin, and the undertone component comprising the first and second portions represents light that traverses the surface of the skin and reflects off of a subsurface; and

an output display, wherein the output display displays the simulated user skin image.

2. The system of claim 1, wherein the user information module comprises an apparatus configured to capture a visible image.

3. The system of claim 1, wherein the user information module comprises an apparatus configured to capture an infrared image.

4. The system of claim 1, wherein the user information module comprises an apparatus configured to capture an ultraviolet image.

5. The system of claim 1, wherein the user information module comprises a text input module.

6. The system of claim 1, wherein the graphical user interface enables manipulation of the user skin image via at least one of a before/after swiper feature, a zoom-in and/or zoom-out feature, a contrast feature, a lighting simulation feature, and a target sizing feature.

7. The system of claim 1, wherein the skin image database comprises imaging data.

8. The system of claim 7, wherein the imaging data comprises polarization filter data and/or red, green, blue (RGB) color model/L, a, b color space conversion data.

9. The system of claim 1, wherein the skin image database comprises skincare product data.

10. The system of claim 9, wherein the skincare product data comprises at least one of age-related skin image data and skincare product efficacy data.

11. The system of claim 1, wherein the skin image database comprises skin data.

12. The system of claim 11, wherein the skin data comprises at least one of skin color data and skin texture data.

13. The system of claim 1, wherein the processor comprises a diagnostic module, an application selection module, an application execution module, and an image processing module.

14. The system of claim 13, wherein the diagnostic module:

determines user red, green, blue (RGB) color space values for the user skin image;

calculates average RGB color space values for the user RGB color space values for the one or more areas of the user skin image; and

converts the average RGB color space values to a user L, a, b color space values.

15. The system of claim 13, wherein the application selection module enables selection of the skin-related application comprising one of a foundation matching application, a lines and wrinkles application, a skin lightening application, a skin de-yellowing application, and a de-aging application.

16. The system of claim 13, wherein the image processing module:

obtains a non-polarized image component, the parallel light image component and the perpendicular light image component from the user skin image;

determines a non-polarized image filter, a parallel light image filter and a perpendicular light image filter based on the one or more sets of identified skin image data from the database;

applies the non-polarized image filter to the non-polarized image component to generate a proscenium image component;

applies the parallel light image filter to the parallel light image component to generate a simulated parallel light image component;

applies the perpendicular light image filter to the perpendicular light image component to generate a simulated perpendicular light image component;

combines the simulated parallel light image component and the simulated perpendicular light image component to generate a base simulated user image for the skin-related application;

combines the base simulated user image with the proscenium image component to generate the simulated user skin image; and

outputs the simulated user skin image to the output display.

17. The system of claim 1, wherein at least one of the user information module, the graphical user interface, the skin image database, the processor, and the output display are located geographically-remote from the other ones of the user information module, the graphical user interface, the skin image database, the processor, and the output display.

18. The system of claim 1, wherein one or more of the user information module, the graphical user interface, the skin image database, the processor, and the output display are distributed in an Internet-based environment.

19. The system of claim 1, wherein one or more of the user information module, the graphical user interface, the skin image database, the processor, and the output display are distributed in a mobile telephony-based environment.

20. The system of claim 1, wherein one or more of the user information module, the graphical user interface, the skin image database, the processor, and the output display are distributed in a kiosk-based environment.

21. The system of claim 1, wherein one or more of the user information module, the graphical user interface, the skin image database, the processor, and the output display are distributed in a local area network environment.

22. An apparatus comprising:

a memory; and

at least one processor device coupled to the memory, wherein the processor;

performs one or more diagnostic operations on at least one portion of a user skin image to generate user skin image data, wherein the one or more diagnostic operations are associated with an identified skin-related application;

processes the user skin image data in accordance with the identified skin-related application, wherein said processing comprises;

identifying one or more sets of skin image data in a database that correspond to the user skin image data based on one or more parameters specified by the skin-related application; and

determining at least one image processing filter based on the one or more sets of identified skin image data from the database; and

applies the at least one image processing filter to the at least one portion of the user skin image to generate a simulated user skin image;

wherein the user skin image comprises a parallel light image component and a perpendicular light image component, and the step of applying the at least one image processing filter to the at least one portion of the user skin image further comprises determining component contribution amounts for one or more of the parallel light image component and the perpendicular light image component based on at least a portion of the identified skin image data;

wherein the parallel light image component comprises a specular component and first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component; and

wherein the specular component represents light that reflects off the surface of the skin, and the undertone component comprising the first and second portions represents light that traverses the surface of the skin and reflects off of a subsurface.

23. An article of manufacture comprising a non-transitory computer readable storage medium having computer readable instructions tangibly embodied thereon which, when implemented, cause at least one computing device to carry out steps comprising:

performing one or more diagnostic operations on at least one portion of a user skin image to generate user skin image data, wherein the one or more diagnostic operations are associated with an identified skin-related application;

processing the user skin image data in accordance with the identified skin-related application, wherein said processing comprises;

identifying one or more sets of skin image data in a database that correspond to the user skin image data based on one or more parameters specified by the skin-related application; and

determining at least one image processing filter based on the one or more sets of identified skin image data from the database; and

applying the at least one image processing filter to the at least one portion of the user skin image to generate a simulated user skin image;

wherein the user skin image comprises a parallel light image component and a perpendicular light image component, and the step of applying the at least one image processing filter to the at least one portion of the user skin image further comprises determining component contribution amounts for one or more of the parallel light image component and the perpendicular light image component based on at least a portion of the identified skin image data;

wherein the parallel light image component comprises a specular component and first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component; and

wherein the specular component represents light that reflects off the surface of the skin, and the undertone component comprising the first and second portions represents light that traverses the surface of the skin and reflects off of a subsurface.

24. An apparatus comprising:

a memory; and

at least one processor device coupled to the memory, wherein the processor;

obtains a user skin image; and

applies a set of image processing filters to polarized components associated with at least a portion of the user skin image to modify the user skin image, wherein the set of image processing filters is controlled by data identified based on a diagnosis of a skin condition from at least a portion of the user skin image, and wherein the identified data represents a previously-determined effect over time of a skincare product usage such that the modification to the user skin image visually simulates the subsequent effect over time of the skincare product usage;

wherein the polarized components associated with the user skin image comprise a parallel light image component and a perpendicular light image component, and modifying the user skin image to visually simulate the subsequent effect over time of the skincare product usage further comprises determining component contribution amounts for one or more of the parallel light image component and the perpendicular light image component based on at least a portion of the identified data;

wherein the parallel light image component comprises a specular component and first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component; and

wherein the specular component represents light that reflects off the surface of the skin, and the undertone component comprising the first and second portions represents light that traverses the surface of the skin and reflects off of a subsurface.

25. The apparatus of claim 24, wherein the at least one processor device is further configured for presenting the modified user skin image to a user from whom the user skin image is obtained.

26. The apparatus of claim 24, wherein the at least one processor device is further configured for generating a sequence of images representing the modification of the user skin image over a plurality of sequential timepoints, and presenting the sequence of images to a user from whom the user skin image is obtained.

27. The apparatus of claim 24, wherein the first portion of the undertone component and the second portion of the undertone component each represent half of the undertone component.

28. The apparatus of claim 24, wherein modification of the user skin image further comprises adjusting over time, based on at least a portion of the identified data, one or more of the parallel light image component and the perpendicular light image component associated with the user skin image.

29. The apparatus of claim 28, wherein one or more of the parallel light image component and the perpendicular light image component associated with the user skin image are adjusted as a function of an amount of change at one or more timepoints given by the identified data.

30. The apparatus of claim 29, wherein the perpendicular light image component is adjusted by the change given in the identified data, and the parallel light image component is adjusted by a fraction of the change given in the identified data.

31. The apparatus of claim 30, wherein the change given by the identified data represents a percentage change in melanin over time.

32. The apparatus of claim 24, wherein the diagnosis of the skin condition is associated with a skin-related application comprising one of a foundation matching application, a lines and wrinkles application, a skin lightening application, a skin de-yellowing application, and a de-aging application.

33. An article of manufacture comprising a non-transitory computer readable storage medium having computer readable instructions tangibly embodied thereon which, when implemented, cause at least one computing device to carry out steps comprising:

obtaining a user skin image; and

applying a set of image processing filters to polarized components associated with at least a portion of the user skin image to modify the user skin image, wherein the set of image processing filters is controlled by data identified based on a diagnosis of a skin condition from at least a portion of the user skin image, and wherein the identified data represents a previously-determined effect over time of a skincare product usage such that the modification to the user skin image visually simulates the subsequent effect over time of the skincare product usage;

wherein the polarized components associated with the user skin image comprise a parallel light image component and a perpendicular light image component, and modifying the user skin image to visually simulate the subsequent effect over time of the skincare product usage further comprises determining component contribution amounts for one or more of the parallel light image component and the perpendicular light image component based on at least a portion of the identified data;

wherein the parallel light image component comprises a specular component and first portion of an undertone component, and the perpendicular light image component comprises a second portion of the undertone component; and

wherein the specular component represents light that reflects off the surface of the skin, and the undertone component comprising the first and second portions represents light that traverses the surface of the skin and reflects off of a subsurface.