Systems, devices, and methods for three-dimensional analysis of eyebags

• US 10,395,099 B2
• Filed: 09/19/2016
• Issued: 08/27/2019
• Est. Priority Date: 09/19/2016
• Status: Active Grant

• Current Assignee: L'Oreal SA
• Original Assignee: L'Oreal SA
• Inventors: Phamduy, Theresa, Norwood, Kelsey
• Primary Examiner(s): Bost, Dwayne D
• Assistant Examiner(s): Brinich, Stephen M
• Application Number:
  US15/269,246
• Publication Number:
  US 20180082108A1
• Time in Patent Office: 1,072 Days
• Field of Search: 382115-118, 382128, 382181, 382170, 382154, 348 47- 48
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/442 Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

In some embodiments of the present disclosure, a system for processing three-dimensional face scan data is provided. A three-dimensional scanner produces an image of a face including an area of interest that includes an eyebag area. A profile of the eyebag area is determined by the system. In some embodiments, the profile is determined based on a vertical slice at the center of the eyebag area. Profiles for multiple sets of scan data may be compared to determine quantitative differences between eyebag profiles. These differences may be used for quantitatively comparing the effects of products applied to the eyebag area between scans. These differences may also be used for predictively generating three-dimensional models to illustrate predicted effects of the use of a product on a face.

Claims

1. A computer-implemented method of processing three-dimensional face scan data, the method comprising:

applying a treatment to an eyebag area of a face;

receiving, by a facial analysis device, first face scan data representing a three-dimensional scan of the face;

determining, by the facial analysis device, a first model of an eyebag area of the first face scan data;

determining, by the facial analysis device, a first score based on the first model;

storing by the facial analysis device, the first score in a scan data store;

comparing, by the facial analysis device, the first model to at least one stored model of previous face scan data representing a previous three-dimensional scan of the face to determine differences between the models;

presenting, by the facial analysis device, the determined differences in order to evaluate the treatment; and

storing, by the facial analysis device, the determined differences in the scan data store.

2. The method of claim 1, further comprising:

receiving, by the facial analysis device, new face scan data representing a three-dimensional scan of a new face;

determining, by the facial analysis device, a new model of an eyebag area of the new face scan data;

determining, by the facial analysis device, predicted face scan data using the new model and the determined differences stored in the scan data store; and

presenting, by the facial analysis device, the predicted face scan data on a display.

3. The method of claim 1, wherein determining the first model of the eyebag area of the first face scan data includes determining a vertical cross-section of the eyebag area at a middle of the eyebag area relative to corners of an associated eye.

4. The method of claim 3, wherein determining the first score based on the first model includes at least one of:

determining an absolute area under curve (AUC) for the first model;

determining a max value or min value for the first model;

determining a slope between curve endpoints for the first model; and

determining an arc length for the first model.

5. The method of claim 1, wherein determining the first model of the eyebag area of the first face scan data includes:

determining a reference plane of the first face scan data;

determining a distance from each voxel within the first face scan data to the reference plane; and

determining a histogram of the distances; and

wherein determining a first score based on the first model includes determining a standard deviation of the histogram.

6. The method of claim 1, further comprising aligning the first face scan data to previous face scan data by:

performing pair-wise coarse adjustment to align the first face scan data with second face scan data; and

performing global fine adjustment to iteratively realign all face scan data until an error tolerance is within a predetermined threshold.

7. The method of claim 6, wherein performing pair-wise coarse adjustment to align the first face scan data with the second face scan data includes iteratively aligning the first face scan data to the second face scan data under the guide of a constraining fraction overlap parameter and an error tolerance parameter until facial landmarks align.

8. A system for processing three-dimensional face scan data, the system comprising:

a three-dimensional scanner; and

a facial analysis device communicatively coupled to the scanner and configured to perform actions including;

receiving, from the three-dimensional scanner, first face scan data representing a three-dimensional scan of a face after application of a treatment to an eyebag area of the face;

determining a first model of an eyebag area of the first face scan data;

determining a first score based on the first model;

storing the first score in a scan data store;

comparing the first model to at least one stored model of previous face scan data representing a previous three-dimensional scan of the face from before application of the treatment to determine differences between the models;

presenting the determined differences in order to evaluate the treatment; and

storing the determined differences in the scan data store.

9. The system of claim 8, further comprising a display, and wherein the actions further include:

receiving, from the scanner, new face scan data representing a three-dimensional scan of a new face;

determining a new model of an eyebag area of the new face scan data;

determining predicted face scan data using the new model and the determined differences stored in the scan data store; and

presenting, on the display, the predicted face scan data.

10. The system of claim 8, wherein determining the first model of the eyebag area of the first face scan data includes determining a vertical cross-section of the eyebag area at a middle of the eyebag area relative to corners of an associated eye.

11. The system of claim 10, wherein determining the first score based on the first model includes at least one of:

determining an absolute area under curve (AUC) for the first model;

determining a maximum value or minimum value for the first model;

determining a slope between curve endpoints for the first model; and

determining an arc length for the first model.

12. The system of claim 8, wherein determining the first model of the eyebag area of the first face scan data includes:

determining a reference plane of the first face scan data;

determining a distance from each voxel within the first face scan data to the reference plane; and

determining a histogram of the distances; and

wherein determining the first score based on the first model includes determining a standard deviation of the histogram.

13. The system of claim 8, wherein the actions further include aligning the first face scan data to previous face scan data by:

performing pair-wise coarse adjustment to align the first face scan data with second face scan data; and

performing global fine adjustment to iteratively realign all face scan data until an error tolerance is within a predetermined threshold.

14. The system of claim 13, wherein performing pair-wise coarse adjustment to align the first face scan data with the second face scan data includes iteratively aligning the first face scan data to the second face scan data under the guide of a constraining fraction overlap parameter and an error tolerance parameter until facial landmarks align.

15. A system for processing three-dimensional face scan data, the system comprising:

circuitry for receiving first face scan data representing a three-dimensional scan of a face after application of a treatment to an eyebag area of the face;

circuitry for determining a first model of an eyebag area of the first face scan data;

circuitry for determining a first score based on the first model;

circuitry for storing the first score in a scan data store;

circuitry for comparing the first model to at least one stored model of previous face scan data representing a previous three-dimensional scan of the face from before application of the treatment to determine differences between the models;

circuitry for presenting the determined differences in order to evaluate the treatment; and

circuitry for storing the determined differences in the scan data store.

16. The system of claim 15, wherein the first model is a vertical cross-section of the eyebag area at the middle of the eyebag area relative to corners of an associated eye, and wherein determining the first score based on the first model includes at least one of:

determining a difference in absolute area under curve (AUC) for the first model and the at least one stored model;

determining a difference in max value or min value for the first model and the at least one stored model;

determining a difference in slope between curve endpoints for the first model and the at least one stored model; and

determining a difference in arc length for the first model and the at least one stored model.