Systems and methods for formulating personalized skincare products

• US 10,546,658 B2
• Filed: 01/28/2019
• Issued: 01/28/2020
• Est. Priority Date: 01/29/2018
• Status: Active Grant

• Current Assignee: Function(x), Inc.
• Original Assignee: Atolla Skin Health, Inc.
• Inventors: Salvi, Sid, Maupin, Meghan, Haghighi, Nava
• Primary Examiner(s): Go, John P
• Application Number:
  US16/259,831
• Publication Number:
  US 20190237194A1
• Time in Patent Office: 365 Days
• CPC Class Codes: A45D 2044/007 Devices for determining the condition of hair or skin or for selecting the appropriate cosmetic or hair treatment
• CPC Class Codes: A45D 44/00 Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment

Abstract

Systems and methods for formulating a personalized skincare product for a user. Data inputs reflecting dermal information of the user (e.g., hydration level measurements, oil level measurements, and a photograph of the user'"'"'s skin reflecting a set of skin concerns) are collected by a computing device and used to determine a set of normalized scores. A skin health data set is generated based on the normalized scores and stored in memory. A skin health metric is determined based on the skin health data set and is stored in memory. The computing device determines, using a machine learning framework, one or more first skincare product formulations based on the user skin health data set. The formulation(s) can be used to manufacture one or more customized skincare products for the user and can be iteratively refined over time, e.g., by collecting additional data from the user over time.

Claims

1. A computerized method of formulating a skin care product for a user using a machine-learning algorithm implemented on a computing device, the method comprising:

receiving, by the computing device, data inputs including one or more hydration level measurements of the user'"'"'s skin taken using a corneocyte test, one or more oil level measurements of the user'"'"'s skin taken using a sebum test, and a photograph of the user'"'"'s skin taken using a camera reflecting a set of skin concerns;

determining, by the computing device, based on the one or more hydration level measurements, a normalized hydration index score using a first computer vision engine trained on a data set of corneocyte test results;

determining, by the computing device, based on the one or more oil level measurements, a normalized oil index score using a second computer vision engine trained on a data set of sebum test results;

determining, by the computing device, based on an adjusted photograph of the user'"'"'s skin, a set of normalized severity scores corresponding to a set of skin concerns of the user using a third computer vision engine trained on an anomaly detection model or a convolutional neural network, wherein the adjusted photograph of the user'"'"'s skin includes one or more illumination adjustments of the photograph and is determined using the anomaly detection model or the convolutional neural network applied to the photograph by the third computer vision engine;

generating, by the computing device, a first skin health data set including the normalized hydration index score, the normalized oil index score, and the set of normalized severity scores;

storing, by the computing device, the first skin health data set in first memory in electronic communication with the computing device;

determining, by the computing device, based on the first skin health data set, a first skin health metric;

storing, by the computing device, the first skin health metric in second memory in electronic communication with the computing device;

determining, by the computing device, using a machine learning framework operating on the computing device, one or more first skin care product formulations based on the first skin health metric and the first skin health data set;

storing, by the computing device, the one or more first skin care product formulations in third memory in electronic communication with the computing device; and

providing, by the computing device, the one or more first skin care product formulations to the user in the form of a recommendation.

2. The method of claim 1 further including:

receiving, by the computing device, one or more additional data inputs reflecting changes in at least one of hydration level measurements, oil level measurements or skin concerns after use of the one or more first skin care product formulations by the user;

generating, by the computing device, based on the one or more additional data inputs, a second skin health data set, by (i) calculating, by the computing device, percentage changes in hydration level measurements, oil level measurements, and the normalized severity scores; and

(ii) calculating, by the computing device, the second skin health data set based on the first skin health data set and the percentage changes; and

determining, by the computing device, using the machine learning framework operating on the computing device, one or more second skin care product formulations based on the second skin health data set.

3. The method of claim 1 wherein the data inputs further include at least one of user age, sex, ethnicity or occupation.

4. The method of claim 1 wherein the photograph is taken with at least one of visible light or ultraviolet light.

5. The method of claim 1 wherein the data inputs include information reflecting at least one of temperature, humidity, or environmental ultraviolet index of the user'"'"'s location.

6. The method of claim 1 wherein the data inputs include information reflecting at least one of user genetics, medical history, diet, water intake, smoking habits, known allergies, alcohol habits, sleep quality, stress levels, time spent in front of electronic screens, or sun exposure.

7. The method of claim 1 wherein the data inputs include information reflecting at least one of a user-reported assessment of skin health, skin care product usage, past skin care product usage, past skin reactions, skin care goals, skin care concerns, skin care absorption or texture preferences.

8. The method of claim 1 wherein the data inputs include at least one of an elasticity measurement of the user'"'"'s skin, a wrinkle measurement of the user'"'"'s skin, or a surface pH level of the user'"'"'s skin.

9. The method of claim 1 wherein the first skin health data set includes information reflecting at least one of the user'"'"'s wrinkles, dark spots, dark circles, texture, acne, sun damage, pore size, redness, or other skin damage.

10. The method of claim 1 wherein the first skin care product formulation includes information reflecting active ingredients, preservatives, and dosage.

11. The method of claim 1 further including generating, by the computing device, based on the first skin health data set, first formulation instructions for manufacturing a first skin care product by a formulation specialist or machine.

12. The method of claim 1 further including generating, by the computing device, based on the first skin health data set, one or more recommendations for adjustments to the user'"'"'s lifestyle, diet, or holistic wellness to achieve a desired skin result.

13. The method of claim 1 further including generating, by the computing device, based on the first skin health data set, a personalized skincare routine including a recommendation for at least one of a cleanser, a serum, a facial oil, a moisturizer, dietary supplements, or a sunscreen.

14. The method of claim 1 further including (i) generating, by the computing device, a user display data set for interpretation and display by a user computing device in electronic communication with the computing device, (ii) and sending the user display data set to the user computing device.

15. The method of claim 1 further including receiving, by the computing device, user certification of the first skin care product formulation prior to storing the first skincare product formulation in memory.

16. A computing system for formulating a skincare product for a user using a machine-learning algorithm implemented on a computing device, the system comprising:

the computing device, configured to;

receive data inputs including one or more hydration level measurements of the user'"'"'s skin taken using a corneocyte test, one or more oil level measurements of the user'"'"'s skin taken using a sebum test, and a photograph of the user'"'"'s skin taken using a camera reflecting a set of skin concerns;

determine, based on the one or more hydration level measurements, a normalized hydration index score using a first computer vision engine trained on a data set of corneocyte test results;

determine, based on the one or more oil level measurements, a normalized oil index score using a second computer vision engine trained on a data set of sebum test results;

determine, based on an adjusted photograph of the user'"'"'s skin, a set of normalized severity scores corresponding to a set of skin concerns of the user using a third computer vision engine trained on an anomaly detection model or a convolutional neural network, wherein the adjusted photograph of the user'"'"'s skin includes one or more illumination adjustments of the photograph and is determined using the anomaly detection model or the convolutional neural network applied to the photograph by the third computer vision engine;

generate a first skin health data set including the normalized hydration index score, the normalized oil index score, and the set of normalized severity scores;

store the first skin health data set in first memory in electronic communication with the computing device;

determine, based on the first skin health data set, a first skin health metric;

store the first skin health metric in second memory in electronic communication with the computing device;

determine, using a machine learning framework operating on the computing device, one or more first skin care product formulations based on the first skin health metric and the first skin health data set;

store, the one or more first skin care product formulations in third memory in electronic communication with the computing device; and

provide the one or more first skin care product formulations to the user in the form of a recommendation.

17. The system of claim 16 further including a user computing device in electronic communication with the computing device, wherein the user computing device is configured to collect and provide the data inputs to the computing device.

18. The system of claim 16 wherein the first storage and the second storage are included in a database in electronic communication with the computing device.

19. A computerized method of formulating first and second skin care products for a user using a machine-learning algorithm implemented on a computing device, the method comprising:

receiving, by the computing device, first data inputs reflecting dermal information of the user, the first data inputs including one or more hydration level measurements of the user'"'"'s skin taken using a corneocyte test, one or more oil level measurements of the user'"'"'s skin taken using a sebum test, and a photograph of the user'"'"'s skin taken using a camera reflecting a set of skin concerns;

determining, by the computing device, based on the one or more hydration level measurements, a normalized hydration index score using a first computer vision engine trained on a data set of corneocyte test results;

determining, by the computing device, based on the one or more oil level measurements, a normalized oil index score using a second computer vision engine trained on a data set of sebum test results;

determining, by the computing device, based on an adjusted photograph of the user'"'"'s skin, a set of normalized severity scores corresponding to a set of skin concerns of the user using a third computer vision engine trained on an anomaly detection model or a convolutional neural network, wherein the adjusted photograph of the user'"'"'s skin includes one or more illumination adjustments of the photograph and is determined using the anomaly detection model or the convolutional neural network applied to the photograph by the third computer vision engine;

generating, by the computing device, based on the first data inputs, a first skin health data set for the user, the first skin health data set including one or more normalized scores reflecting the first data inputs;

storing, by the computing device, the first skin health data set in first storage in electronic communication with the computing device;

determining, by the computing device, a first skincare product formulation based on the first skin health data set;

storing, by the computing device, the first skincare product formulation in second storage in electronic communication with the computing device;

providing, by the computing device, the first skin care product formulation to the user in the form of a first recommendation;

receiving, by the computing device, second data inputs-reflecting changes in the first data inputs after use of a first skincare product based on the first skincare product formulation;

generating, by the computing device, based on the first data inputs and second data inputs, a second skin health data set for the user;

storing, by the computing device, the second skin health data set in third storage in electronic communication with the computing device;

determining, by the computing device, a second skincare product formulation based on the second skin health data set;

storing, by the computing device, the second skincare product formulation in fourth storage in electronic communication with the computing device

providing, by the computing device, the second skin care product formulation to the user in the form of a second recommendation.