Device and method for classifying a condition based on image analysis

• US 10,204,260 B2
• Filed: 09/13/2016
• Issued: 02/12/2019
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Children's National Medical Center
• Original Assignee: Children's National Medical Center
• Inventors: Linguraru, Marius George, Zhao, Qian, Rosenbaum, Kenneth, Summar, Marshall, Okada, Kazunori
• Primary Examiner(s): Shah, Utpal D
• Application Number:
  US15/264,386
• Publication Number:
  US 20170068846A1
• Time in Patent Office: 882 Days
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An image analysis device includes circuitry that receives one or more input images and detects a plurality of anatomical landmarks on the one or more input images using a pre-determined face model. The circuitry extracts a plurality of geometric and local texture features based on the plurality of anatomical landmarks. The circuitry selects one or more condition-specific features from the plurality of geometric and local texture features. The circuitry classifies the one or more input images into one or more conditions based on the one or more condition-specific features.

Claims

1. A device comprising:

circuitry configured to;

receive one or more input images;

detect a plurality of anatomical landmarks on the one or more input images using a pre-determined face model, by generating a statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE) and/or Gaussian mixture model (GMM);

extract a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

select one or more features from the plurality of geometric and local texture features; and

determine a likelihood that a patient in the one or more input images has a genetic disorder based on the selected one or more features.

2. The device of claim 1, wherein

the one or more features are syndrome-specific, age-specific, or ethnicity-specific features corresponding to genetic disorders.

3. The device of claim 1, wherein

the circuitry is further configured to generate the pre-determined face model by being further configured to:

receive a plurality of training images corresponding to a facial morphology and a texture of a baseline group and a group having at least one of the one or more features;

further generate a statistical shape model and generate an appearance patch model based on the plurality of training images; and

train a classifier with one or more geometric and multi-scale local texture features based on the plurality of training images, the statistical shape model, and the appearance patch model.

4. The device of claim 3, wherein

the plurality of training images include image areas with individuals of different ages, genders, ethnicities, races, illumination, image background, resolution, or poses.

5. The device of claim 1, wherein

the pre-determined face model is based on a constrained local model.

6. The device of claim 1, wherein the circuitry is configured to

determine that the patient has the genetic disorder based on a machine learning technique.

7. The device of claim 1, wherein the circuitry is configured to

select the one or more features based on an area between a receiver operating characteristic curve and a random classifier slope.

8. The device of claim 1, wherein

the circuitry is configured to output a recommendation of a medical practitioner when the patient is determined to have the genetic disorder.

9. The device of claim 1, wherein

the circuitry is configured to output a condition management recommendation when the patient is determined to have the genetic disorder.

10. The device of claim 9, wherein

the condition management recommendation includes a recommendation for treating the genetic disorder.

11. The device of claim 9, wherein

the condition management recommendation includes lifestyle management information related to the genetic disorder.

12. The device of claim 11, wherein

the lifestyle management information includes at least one of a recommendation and a reminder related to diet and/or exercise.

13. The device of claim 11, wherein

the lifestyle management information includes at least one of a recommendation and a reminder related to scheduling and/or attending an appointment related to treating the genetic disorder.

14. The device of claim 1, wherein the circuitry is configured to determine the likelihood based only on the selected one or more features.

15. The device of claim 1, wherein the pre-determined face model is generated by the circuitry being configured to

perform independent component analysis (ICA) to generate a statistical shape face model via data-driven selection that selects independent components, and

perform shape prior estimation using the kernel density estimation (KDE) and the Gaussian mixture model (GMM).

16. The device of claim 1, wherein

the pre-determined face model is generated by the circuitry being configured to generate the statistical shape with the non-Gaussian shape prior estimation, and

the circuitry is configured to generate the statistical shape with the non-Gaussian shape prior estimation using the kernel density estimation (KDE).

17. The device of claim 1, wherein the circuitry is configured to generate the statistical shape with the non-Gaussian shape prior estimation using the kernel density estimation (KDE) and the Gaussian mixture model (GMM).

18. The device of claim 1, wherein the pre-determined face model is generated by the circuitry being configured to perform independent component analysis (ICA) to generate a statistical shape face model.

19. The device of claim 1, wherein the pre-determined face model is generated by the circuitry being configured to perform, data-driven selection that selects independent components to generate a statistical shape face model.

20. The device of claim 1, wherein the circuitry is further configured to control display of an app interface which implements obtaining of the one or more input images.

21. The device of claim 1, wherein the circuitry is further configured to control display of an app interface which displays a result of a determination that the patient has the genetic disorder.

22. The device of claim 1, wherein

the one or more input images correspond to a part of a body of the patient.

23. The device of claim 1, wherein the circuitry is configured to

determine a probability score representing the likelihood that the patient in the one or more input images has the genetic disorder;

and

determine that the patient has the genetic disorder when the probability score is greater than or equal to a predetermined threshold and determine that the patient does not have the genetic disorder when the probability score is less than the predetermined threshold.

24. The device of claim 1, wherein the circuitry is configured to

display, on a display, the likelihood that the patient in the one or more input images has the genetic disorder to allow a physician to determine whether the patient has the genetic disorder based on the likelihood and other information corresponding to the patient.

25. A method comprising:

receiving one or more input images;

detecting, by circuitry, a plurality of anatomical landmarks on the one or more input images using a pre-determined face model, the detecting including generating a statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE) and/or Gaussian mixture model (GMM);

extracting, by the circuitry, a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

selecting, by the circuitry, one or more features from the plurality of geometric and local texture features; and

determining, by the circuitry, a likelihood that a patient in the one or more input images has a genetic disorder based on the selected one or more features.

26. A non-transitory computer readable medium having instructions stored therein that when executed by one or more processors cause a computer to perform a method comprising:

receiving one or more input images;

detecting a plurality of anatomical landmarks on the one or more input images using a pre-determined face model, the detecting including generating a statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE) and/or Gaussian mixture model (GMM);

extracting a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

selecting one or more features from the plurality of geometric and local texture features; and

determining a likelihood that a patient in the one or more input images has a genetic disorder based on the selected one or more features.

27. A system comprising:

circuitry configured to;

receive one or more input images;

detect a plurality of anatomical landmarks on the one or more input images using a pre-determined face model, by generating a statistical shape with non-Gaussian shape prior estimation using kernel density estimation (KDE) and/or Gaussian mixture model (GMM);

extract a plurality of geometric and local texture features based on the plurality of anatomical landmarks;

select one or more features from the plurality of geometric and local texture features; and

determine a likelihood that a patient in the one or more input images has a genetic disorder based on the selected one or more features; and

a device that captures the one or more input images and transmits the one or more input images to the circuitry.