Device and method for classifying a condition based on image analysis

• US 9,443,132 B2
• Filed: 02/05/2014
• Issued: 09/13/2016
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Children's National Medical Center
• Original Assignee: Children's National Medical Center
• Inventors: Linguraru, Marius George, Zhao, Qian, Rosenbaum, Kenneth, Summar, Marshall, Okada, Kazunori
• Primary Examiner(s): SHAH, UTPAL D
• Application Number:
  US14/173,482
• Publication Number:
  US 20140219526A1
• Time in Patent Office: 951 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An image analysis device includes circuitry that receives one or more input images and detects a plurality of anatomical landmarks on the one or more input images using a pre-determined face model. The circuitry extracts a plurality of geometric and local texture features based on the plurality of anatomical landmarks. The circuitry selects one or more condition-specific features from the plurality of geometric and local texture features. The circuitry classifies the one or more input images into one or more conditions based on the one or more condition-specific features.

Claims

1. An image analysis device comprising:

circuitry configured to;

receive one or more input images;

detect a plurality of anatomical landmarks on the one or more input images using a pre-determined face model;

extract a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

select one or more condition-specific features from the plurality of geometric and local texture features; and

classify the one or more input images into one or more conditions based on the selected one or more condition-specific geometric and local texture features to generate a classification result,

wherein the pre-determined face model is generated by the circuitry being configured to generate a statistical shape with non-Gaussian shape prior estimation,

and wherein the circuitry is configured to generate the statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE).

2. The image analysis device of claim 1, wherein

the classification result is provided as an input to a disease screening process.

3. The image analysis device of claim 1, wherein

the one or more conditions correspond to one or more genetic syndromes, and

the one or more condition-specific features are syndrome-specific or population-specific features corresponding to the one or more genetic syndromes.

4. The image analysis device of claim 1, wherein

the circuitry is further configured to generate the pre-determined face model by being further configured to:

receive a plurality of training images corresponding to a facial morphology and a texture of a baseline group and a group having at least one of the one or more condition-specific features;

further generate a statistical shape model and generate an appearance patch model based on the plurality of training images; and

train a classifier with one or more geometric and multi-scale local texture features based on the plurality of training images, the statistical shape model, and the appearance patch model.

5. The image analysis device of claim 4, wherein the plurality of training images include image areas with individuals of different ages, ethnicities, illumination, image background, resolution, or poses.

6. The image analysis device of claim 1, wherein

the pre-determined face model is based on a constrained local model.

7. The image analysis device of claim 1, wherein

the classification of the one or more input images into the one or more types of conditions is based on one or more of a support vector machines technique, a k-nearest neighbor technique, a linear discriminant analysis technique and a random forest technique.

8. The image analysis device of claim 1, wherein

the selection of the one or more condition-specific features is based on an area between a receiver operating characteristic curve and a random classifier slope.

9. The image analysis device of claim 1, wherein

the one or more types of conditions correspond to one or more emotional states and/or one or more psychological states.

10. The image analysis device of claim 9, wherein

the classification result is provided as an input to a surveillance or security screening process.

11. The image analysis device of claim 9, wherein

the one or more emotional states and psychological states correspond to one or more of happiness, pleasure, distress, disgust, anger, sadness, crying, or pain.

12. The image analysis device of claim 1, wherein

the circuitry is further configured to output a recommendation of a medical practitioner based on the classification result.

13. The image analysis device of claim 1, wherein

the circuitry is further configured to output a condition management recommendation based on the classification result.

14. The image analysis device of claim 13, wherein

the condition management recommendation includes a recommendation for treating the condition.

15. The image analysis device of claim 13, wherein

the condition management recommendation includes lifestyle management information related to the condition.

16. The image analysis device of claim 15, wherein

the lifestyle management information includes at least one of a recommendation and a reminder related to diet and/or exercise.

17. The image analysis device of claim 15, wherein

the lifestyle management information includes at least one of a recommendation and a reminder related to scheduling and/or attending an appointment related to treating the condition.

20. The image analysis device of claim 1, wherein the circuitry is configured to classify the one or more input images into one or more conditions based only on the selected one or more condition-specific geometric and local texture features.

21. The image analysis device of claim 1, wherein the pre-determined face model is generated by the circuitry being configured to

perform independent component analysis (ICA) to generate a statistical shape face model via data-driven selection that selects independent components, and

perform shape prior estimation using kernel density estimation (KDE) and Gaussian mixture model (GMM).

22. The image analysis device of claim 1, wherein the circuitry is configured to generate the statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE) and Gaussian mixture model (GMM).

23. The image analysis device of claim 1, wherein the pre-determined face model is generated by the circuitry being configured to perform independent component analysis (ICA) to generate a statistical shape face model.

24. The image analysis device of claim 1, wherein the pre-determined face model is generated by the circuitry being configured to perform data- driven selection that selects independent components to generate a statistical shape face model.

25. The image analysis device of claim 1, wherein the circuitry is further configured to control display of an app interface which implements obtaining of the one or more input images and displays the result of the classification.

18. An image analysis method comprising:

receiving one or more input images;

detecting, by circuitry, a plurality of anatomical landmarks on the one or more input images using a pre-determined face model;

extracting, by the circuitry, a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

selecting, by the circuitry, one or more condition-specific features from the plurality of geometric and local texture features; and

classifying, by the circuitry, the one or more input images into one or more conditions based on the selected one or more condition-specific geometric and local texture features to generate a classification result,

wherein the pre-determined face model is generated by the circuitry by generating a statistical shape with non-Gaussian shape prior estimation, and

wherein the circuitry is configured to generate the statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE).

19. A non-transitory computer readable medium having instructions stored therein that when executed by one or more processors cause a computer to perform an image analysis method, the method comprising:

receiving one or more input images;

detecting a plurality of anatomical landmarks on the one or more input images using a pre-determined face model;

extracting a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

selecting one or more condition-specific features from the plurality of geometric and local texture features; and

classifying, by the circuitry, the one or more input images into one or more conditions based on the selected one or more condition-specific geometric and local texture features to generate a classification result,

wherein the pre-determined face model is generated by the circuitry by generating a statistical shape with non-Gaussian shape prior estimation, and wherein the circuitry is configured to generate the statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE).