Personalized skin diagnosis and skincare

• US 10,614,921 B2
• Filed: 01/03/2017
• Issued: 04/07/2020
• Est. Priority Date: 05/24/2016
• Status: Active Grant

• Current Assignee: XYZprinting, Inc.
• Original Assignee: Cal-Comp Big Data, Inc.
• Inventors: Shen, Shyh-Yong, Chi, Min-Chang
• Primary Examiner(s): Beg, Samah A
• Application Number:
  US15/397,700
• Publication Number:
  US 20170340267A1
• Time in Patent Office: 1,190 Days
• Field of Search: None
• CPC Class Codes: A61B 2090/309 using white LEDs
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 2560/0431 Portable apparatus, e.g. comprising a handle or case
• CPC Class Codes: A61B 2562/185 Optical shielding, e.g. baffles
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/0079 using mirrors, i.e. for self-examination
• CPC Class Codes: A61B 5/442 Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
• CPC Class Codes: A61B 5/444 Evaluating skin marks, e.g. mole, nevi, tumour, scar
• CPC Class Codes: A61B 5/445 Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
• CPC Class Codes: A61B 5/486 Bio-feedback A61B5/375 takes precedence
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7445 Display arrangements, e.g. multiple display units
• CPC Class Codes: A61B 5/749 Voice-controlled interfaces
• CPC Class Codes: A61B 90/96 using barcodes
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G16H 10/60 : for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Methods, systems, and devices are disclosed herein for conducting skin analysis for a user. In one aspect, an apparatus for conducting skin analysis is disclosed, comprising: a reflective display operative to reflect an image of a user and to render a graphical user interface; an input interface operative to receive a user input for operating the apparatus, wherein the input interface includes a motion sensor module operative to detect gesture user input; an image capturing module, wherein the image capturing module is operative to capture an image of the user; a processing system configured to receive the captured image from the image capturing module, to receive the user input from the input interface, and to generate a skin profile corresponding to the user'"'"'s skin condition based on the captured image; and a wireless communication module operative to transmit the skin profile outbound to a computing device, and operative to receive skincare feedback that is generated based on the skin profile from the computing device.

Claims

1. An apparatus for conducting personalized skin analysis, comprising:

a base;

a reflective display coupled to the base, wherein the reflective display is operative to reflect an image of a user and to display a graphical user interface for the user to receive information, the reflective display further operative to display a template for a part of a human body;

two arrays of LED lights arranged along two sides of the reflective display operable to simulate a plurality of lighting scenarios in connection with an appearance of a product on a skin of the user, wherein each of the plurality of lighting scenarios corresponds to a light temperature of the array of LED lights;

a motion sensor coupled to the base to detect gesture user input and to detect an application of the product;

a camera coupled to the base, wherein the camera is operative to capture an image of the user that is adjusted according to the template;

a processor in communication with the camera, the reflective display and the motion sensor, wherein the processor is configured to receive the captured image from the camera, to receive the user input from the motion sensor, and to generate a user skin profile corresponding to a skin condition of the user based on the captured image, the processor further configured to cause the graphical user interface to display a mapping between the plurality of lighting scenarios and corresponding light temperatures to allow the user to select a desired lighting scenario prior to the application of the product; and

a wireless communication transmitter and a wireless communication receiver in communication with the processor, wherein the wireless communication transmitter is operative to transmit the user skin profile outbound to a computing device, and the wireless communication receiver is operative to receive skincare feedback that is generated based on the user skin profile from the computing device.

2. The apparatus of claim 1, wherein the user skin profile includes data generated based on at least one of the following skin factors, including:

dark circles, pores, red spots, dark spots, fine lines, wrinkles and complexion.

3. The apparatus of claim 1, wherein the user skin profile includes data corresponding to at least one skin index of a plurality of skin indices, the plurality of skin indices being clarity, texture, firmness, brightness, and healthiness;

and wherein each skin index of the plurality of skin indices is derived from data corresponding to at least one of the following skin factors, including;

dark circles, pores, red spots, dark spots, fine lines, wrinkles and complexion.

4. The apparatus of claim 3, further comprising a memory that is operative to store the captured image and the user skin profile, wherein the user skin profile further includes a user selection of one skin index from the plurality of skin indices as a goal, and wherein the processor is configured to periodically capture an image of the user that is adjusted according to the template over a predetermined time period and track the selected skin index for each of the captured images.

5. The apparatus of claim 4, wherein the processor is operative to render a graphical user interface on the reflective display presenting at least a portion of two of the captured images at one time for comparison purposes.

6. The apparatus of claim 1, wherein the skincare feedback includes a recommendation of at least one skincare product based on the user skin profile.

7. The apparatus of claim 6, wherein the skincare feedback further includes a recommended skincare routine based on the user skin profile, including the recommended skincare product and steps to conduct skincare, wherein the user skin profile includes gender, age, and lifestyle information corresponding to the user.

8. The apparatus of claim 1, further comprising:

a microphone that detects voice input.

9. The apparatus of claim 1, wherein the camera is operative to capture an image of bar code and transmit the bar code image to the processor;

and wherein the processor is operative to recognize and categorize skincare products corresponding to the bar code image.

10. The apparatus of claim 1, further comprising:

a printed circuit board under one or more touch keys that are operative to navigate the graphical user interface when actuated.

11. The apparatus of claim 1, further comprising:

an infrared sensor that is operative to detect heat radiation; and

wherein the processor is operative to enter a stand-by mode and wake up from the stand-by mode in response to the infrared sensor detecting an object with heat radiation.

12. The apparatus of claim 1, wherein the reflective display includes a transflective coating.

13. The apparatus of claim 1, wherein the camera includes a plurality of shielding plates operative to open or shut the camera.

14. The apparatus of claim 1, wherein the camera includes a lens module and an auxiliary light source disposed in a periphery surrounding the lens module.

15. The apparatus of claim 1, wherein the camera includes a hinge detachably connected to the base, wherein the hinge is operative to vary the angle of the camera in relation to the base.

16. A computer-implemented method for conducting skin analysis, comprising:

rendering a graphical user interface to display information to a user on a reflective display;

detecting, by a motion sensor, a gesture from the user to interact with the graphical user interface;

displaying, on the reflective display, a template for a part of a human body;

capturing, by a camera, an image of the user that is adjusted according to the template;

processing the captured image to analyze a skin condition of the user;

generating a user skin profile based on the analysis of the skin condition of the user;

wirelessly transmitting the user skin profile outbound to a computing device;

receiving, from the computing device, skincare feedback through the graphical user interface that is generated based on the user skin profile for the user, wherein the skin care feedback comprises historical changes of the skin condition; and

presenting the received skin care feedback to the user via the reflective display, wherein portions of at least two captured images illustrating the historical changes of the skin conditions are presented on the reflective display, and wherein the portions are separated by an adjustable divider line to allow the user to adjust sizes of the portions to review the historical changes happened in the part of the human body.

17. The method of claim 16, further comprising:

displaying the skin profile on the reflective display, wherein the user skin profile includes data corresponding to at least one of the following skin factors, including;

dark circles, pores, red spots, dark spots, fine lines, wrinkles, or complexion.

18. The method of claim 17, further comprising:

selecting one skin index from the plurality of skin indices as a goal;

periodically capturing an image of the user that is adjusted according to the template over a predetermined time period;

tracking the selected skin index for each of the captured images; and

showing the selected skin index for each of the captured images on the reflective display.

19. The method of claim 16, further comprising:

displaying the user skin profile on the reflective display, wherein the user skin profile includes at least one skin index of a plurality of skin indices, the plurality of skin indices being clarity, texture, firmness, brightness and healthiness; and

wherein each skin index of the plurality of skin indices is derived based on at least one of the following skin factors, including;

dark circles, pores, red spots, dark spots, fine lines, wrinkles and complexion.

20. The method of claim 16, further comprising:

rendering a graphical user interface on the reflective display presenting a recommendation of at least one skincare product based on the user skin profile.

21. The method of claim 20, further comprising:

rendering a graphical user interface on the reflective display presenting a recommended skincare routine for the user, including the recommended skincare product and steps to conduct skincare based on the user skin profile, wherein user skin profile includes gender, age, and lifestyle information corresponding to the user.

22. A system of analyzing skin, comprising:

an apparatus for conducting skin analysis, comprising;

a base;

a reflective display coupled to the base, wherein the reflective display is operative to reflect an image of a user and to render a graphical user interface that provides information to the user, the reflective display further operative to display a template for a part of a body;

two arrays of LED lights arranged along two sides of the reflective display operable to simulate a plurality of lighting scenarios in connection with an appearance of a product on a skin of the user, wherein each of the plurality of lighting scenarios corresponds to a light temperature of the array of LED lights;

a motion sensor coupled to the base to detect gesture user input and to detect an application of the product;

a camera coupled to the base, wherein the camera is operative to capture an image of the user that is adjusted according to the template;

a processor in communication with the camera, the reflective display and the motion sensor, wherein the processor is configured to receive the captured image from the camera, to receive the user input from the motion sensor, and to generate a user skin profile corresponding to a skin condition of the user based on the captured image, the processor further configured to cause the graphical user interface to display a mapping between the plurality of lighting scenarios and corresponding light temperatures to allow the user to select a desired lighting scenario prior to the application of the product; and

a wireless communication transmitter and a wireless communication receiver in communication with the processor, wherein the wireless communication transmitter is operative to transmit the user skin profile outbound to a server for analyzing skin, and the wireless communication receiver is operative to receive skincare feedback that is generated based on the user skin profile from the server for analyzing skin; and

the server for analyzing skin, comprising;

a communication transmitter and a communication receiver;

one or more memories holding information that includes instructions; and

a processor coupled to the communication transmitter and the communication receiver and the one or more memories, wherein the processor is operative to read the instructions from the one or more memories to;

receive, via the communication receiver, a user skin profile from the apparatus for conducting skin analysis;

generate skincare feedback based on the user skin profile; and

transmit, via the communication transmitter, the skincare feedback to the apparatus for conducting skin analysis.

23. The system of claim 22, further comprising a mobile device that is operative to receive the user skin profile and the skincare feedback from the server for analyzing skin.

24. The system of claim 22, wherein the processor of the server is operative to read the instructions from the one or more memories to classify the user into a skin condition group based on the user skin profile, where the skin condition group is formed to include one or more user skin profiles from different users.

25. The system of claim 24, wherein the user is classified into the skin condition group based on gender, age, and geographical location information corresponding to the user.