MODELING AND LEARNING CHARACTER TRAITS AND MEDICAL CONDITION BASED ON 3D FACIAL FEATURES

• US 20180190377A1
• Filed: 01/02/2018
• Published: 07/05/2018
• Est. Priority Date: 12/30/2016
• Status: Abandoned Application

• Current Assignee: Dirk Schneemann, LLC
• Original Assignee: Dirk Schneemann, LLC
• Application Number:
  US15/860,395
• Publication Number:
  US 20180190377A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/167 Personality evaluation
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7275 Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/7485 Automatic selection of region of interest
• CPC Class Codes: A61B 6/03 Computed tomography [CT] echo-tomography A61B8/14
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/08 Detecting organic movements or changes, e.g. tumours, cysts, swellings A61B8/02 - A61B8/06 take precedence
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: G06F 18/2413 : based on distances to training or reference patterns
• CPC Class Codes: G06F 18/41 : Interactive pattern learning with a human teacher
• CPC Class Codes: G06N 3/04 : Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 : Combinations of networks
• CPC Class Codes: G06N 3/084 : Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 7/0016 : involving temporal comparison
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/454 : Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 : using neural networks
• CPC Class Codes: G06V 20/653 : by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 40/161 : Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/169 : Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 : Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 : Facial expression recognition
• CPC Class Codes: G16H 20/70 : relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A computer-implemented method for identifying character traits associated with a target subject includes acquiring image data of a target subject from an image data source, rendering a 3D image data set, comparing each of a plurality of regions of interest within the 3D image set to a historical image data set to identify active regions of interest, grouping subsets of the regions of interest into one or more convolutional feature layers, wherein each convolutional feature layer probabilistically maps to a pre-identified character trait, and applying a convolutional neural network model to the convolutional feature layers to identify a pattern of active regions of interest within each convolutional feature layer to predict whether a target subject possesses the pre-identified character trait.

Claims

1. A computer-implemented method for identifying character traits associated with a target subject, the method comprising:

acquiring image data of a target subject from an image data source;

rendering a colored or textured 3D image data set;

comparing, with a characteristic recognition server, each of a plurality of regions of interest within the 3D image set to a historical image data set to identify active regions of interest;

grouping subsets of the regions of interest into one or more convolutional feature layers, wherein convolutional feature layers probabilistically map to pre-identified character traits; and

applying, with a prediction and learning engine, a convolutional neural network model to the convolutional feature layers to train and identify patterns of active regions of interest within each convolutional feature layer to predict whether a target subject possesses the pre-identified character trait.

2. The computer-implemented method of claim 1, further comprising:

storing the one or more convolutional neural networks; and

for each pre-defined character trait, extrapolating from the one or more convolutional neural networks, one or more regions of interest correlated to the pre-defined character trait.

3. The method of claim 2, wherein the extrapolating one or more regions of interest comprises applying a deep learning algorithm to the one or more convolutional neural networks.

4. The computer-implemented method of claim 1, further comprising obtaining, from a user interface, an indication as to whether the target subject possesses the pre-identified character trait.

5. The computer implemented method of claim 4, further comprising generating an error signal if the prediction as to whether the target subject possesses the pre-identified character trait does not match the indication from the user interface.

6. The computer implemented method of claim 5, further comprising tuning the convolutional neural network model by applying, with the prediction an learning engine, the error signal to the convolutional neural network model.

7. The computer-implemented method of claim 6, wherein the tuning of the convolutional neural network model comprises adjusting a set of probabilistic weightings for one or more convolutional layers, wherein a probabilistic weighting indicates a likelihood that the convolutional layer is included in the convolutional neural network model in relation to a corresponding pre-defined character trait.

8. A computer-implemented method for identifying early signs of diseases from features detected in human faces, the method comprising:

acquiring image data of a target subject from an image data sources;

rendering a colored or textured 3D image data set;

comparing each of a plurality of regions of interest within the 3D image set to a historical data set stored in an Electronic Health Record;

grouping subsets of the regions of interest into one or more convolutional feature layers, wherein convolutional feature layers probabilistically map to one or more medical diagnoses; and

applying a convolutional neural network algorithm to the convolutional feature layers to train and identify a pattern of active regions of interest within each convolutional feature layer to render a medical diagnosis.

9. The method of claim 8, further comprising:

storing a plurality of convolutional neural networks, each convolutional neural network comprising a set of convolutional feature layers and one or more corresponding medical diagnoses; and

for each medical diagnosis, extrapolating from the plurality of convolutional neural networks, one or more regions of interest correlated to the medical diagnosis.

10. The method of claim 9, wherein the extrapolating one or more regions of interest comprises applying a deep learning algorithm to the plurality of convolutional neural networks.

11. A system for identifying character traits associated with a target subject, the system comprising:

a characteristic recognition server, an image data source, a user interface, and a data store, wherein the characteristic recognition server comprises a processor and a non-transitory medium with computer executable instructions embedded thereon, the computer executable instructions configured to cause the processor to;

acquire image data of a target subject from the image data source;

render a textured or colored 3D image data set;

compare each of a plurality of regions of interest within the 3D image set to a historical image data set to identify active regions of interest;

group subsets of the regions of interest into one or more convolutional feature layers, wherein convolutional feature layers probabilistically map to pre-identified character traits; and

apply, with a prediction and learning engine, a convolutional neural network model to the convolutional feature layers to identify and train a pattern of active regions of interest within each convolutional feature layer to predict whether a target subject possesses the pre-identified character trait.

12. The system of claim 11, wherein the computer executable instructions are further configured to cause the processor to:

store the one or more convolutional neural networks in the data store; and

for each pre-defined character trait, extrapolate from the one or more convolutional neural networks, one or more regions of interest correlated to the pre-defined character trait.

13. The system of claim 12, wherein the computer executable instructions are further configured to cause the processor to apply a deep learning algorithm to the one or more convolutional neural networks.

14. The system of claim 11, wherein the computer executable instructions are further configured to cause the processor to obtain, from the user interface, an indication as to whether the target subject possesses the pre-identified character trait.

15. The system of claim 14, wherein the computer executable instructions are further configured to cause the processor to generate an error signal if the prediction as to whether the target subject possesses the pre-identified character trait does not match the indication from the user interface.

16. The system of claim 15, wherein the computer executable instructions are further configured to cause the processor to tune the convolutional neural network model by applying the error signal to the convolutional neural network model.

17. The system of claim 16, wherein the computer executable instructions are further configured to cause the processor to adjust a set of probabilistic weightings for one or more convolutional layers, wherein a probabilistic weighting indicates a likelihood that the convolutional layer is included in the convolutional neural network model in relation to a corresponding pre-defined character trait.

18. The system of claim 11, wherein the computer executable instructions are further configured to cause the processor to apply a convolutional neural network algorithm to the convolutional feature layers to identify a pattern of active regions of interest within each convolutional feature layer to render a medical diagnosis.

19. The system of claim 18, wherein the computer executable instructions are further configured to cause the processor to store a plurality of convolutional neural networks, each convolutional neural network comprising a set of convolutional feature layers and one or more corresponding medical diagnoses;

and

for each medical diagnosis, extrapolating from the plurality of convolutional neural networks, one or more regions of interest correlated to the medical diagnosis.

20. The system of claim 11, wherein the image data source comprises a still camera, video camera, an infrared camera, a 3D point cloud source, a laser scanner, a CAT scanner, a MRI scanner, or an ultrasound scanner.