Emotional and/or psychiatric state detection

• US 9,058,816 B2
• Filed: 08/23/2010
• Issued: 06/16/2015
• Est. Priority Date: 07/06/2010
• Status: Active Grant

• Current Assignee: RMIT University
• Original Assignee: RMIT University
• Inventors: Lech, Margaret, Allen, Nicholas Brian, Burnett, Ian Shaw, He, Ling
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US13/807,905
• Publication Number:
  US 20130166291A1
• Time in Patent Office: 1,758 Days
• Field of Search: 846/61, 846/63, 704/232, 704/261, 704/267, 704/270, 704/9
• US Class Current: 1/1
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Mental state of a person is classified in an automated manner by analysing natural speech of the person. A glottal waveform is extracted from a natural speech signal. Pre-determined parameters defining at least one diagnostic class of a class model are retrieved, the parameters determined from selected training glottal waveform features. The selected glottal waveform features are extracted from the signal. Current mental state of the person is classified by comparing extracted glottal waveform features with the parameters and class model. Feature extraction from a glottal waveform or other natural speech signal may involve determining spectral amplitudes of the signal, setting spectral amplitudes below a pre-defined threshold to zero and, for each of a plurality of sub bands, determining an area under the thresholded spectral amplitudes, and deriving signal feature parameters from the determined areas in accordance with a diagnostic class model.

Claims

1. A method of classifying a current mental state of a person by analysing natural speech of the person, the method comprising:

receiving a natural speech signal obtained from the person;

pre-processing the natural speech signal to effect frame extraction;

extracting a glottal waveform from frames of the speech signal;

retrieving pre-determined parameters, the parameters being derived from selected training glottal waveform features and the parameters defining at least one diagnostic class in accordance with a classifier;

extracting selected glottal waveform features from the glottal waveform of the speech signal; and

classifying the current mental state of the person by comparing the extracted selected glottal waveform features with the parameters and the classifier.

2. The method of claim 1, performed in near real-time upon utterance of the natural speech by the person.

3. The method of claim 1, wherein the classifying of the current mental state of the person involves classifying the person as being depressed;

at-risk of developing depression;

or non-depressed.

4. The method of claim 1, wherein extracting selected glottal waveform feature comprises deriving parameters from an area under a normalised Teager Energy operator (TEO) correlation in one or more sub-bands.

5. The method of claim 1 further comprising at least one further pre-processing step of:

signal noise removal;

channel compensation; and

pre-emphasis filtering.

6. The method of claim 1 wherein Wavelet Packet Analysis is employed in derivation of the parameters.

7. The method of claim 1 wherein the parameters are derived from the selected training glottal waveform features by at least one of:

stochastic modeling;

statistical modeling;

neural network modeling; and

analytical modelling.

8. The method of claim 1, further comprising selecting an optimal sub-set of the parameters, and utilising only the optimal sub-set in subsequent classification.

9. The method of claim 1, wherein the classifier comprises a diagnostic class model.

10. The method of claim 1, wherein the classifier comprises a trained neural network.

11. A computing device for classifying a current mental state of a person by analysing natural speech of the person, the computing device comprising a processor configured to:

receive a natural speech signal obtained from the person;

pre-process the natural speech signal to effect frame extraction;

extract a glottal waveform from frames of the speech signal;

retrieve pre-determined parameters, the parameters being derived from selected training glottal waveform features and the parameters defining at least one diagnostic class in accordance with a classifier;

extract selected glottal waveform features from the glottal waveform of the speech signal; and

classify the current mental state of the person by comparing the extracted selected glottal waveform features with the parameters and the classifier.

12. The computing device of claim 11, wherein the computing device comprises at least one of:

an electronic programmable device, a mobile phone, a personal computer, a telephone, a personal communications device;

a server effecting web-hosted processing; and

a point of care medical processor.

13. The computing device of claim 11, wherein the processor is further configured to perform the processes of claim 11 in near real-time upon utterance of the natural speech by the person.

14. A non-transitory computer storage medium comprising a plurality of computer program instructions to make a computer execute a process for classifying a current mental state of a person by analysing natural speech of the person, the process comprising:

receiving a natural speech signal obtained from the person;

pre-processing the natural speech signal to effect frame extraction;

extracting a glottal waveform from frames of the speech signal;

retrieving pre-determined parameters, the parameters being derived from selected training glottal waveform features and the parameters defining at least one diagnostic class in accordance with a classifier;

extracting selected glottal waveform features from the glottal waveform of the speech signal; and

classifying the current mental state of the person by comparing the extracted selected glottal waveform features with the parameters and the classifier.

15. A method of classifying a current mental state of a person by analysing natural speech of the person, the method comprising:

receiving a natural speech signal obtained from the person;

pre-processing the natural speech signal to effect frame extraction;

determining spectral amplitudes of frames of the speech signal;

setting the spectral amplitudes that are below a pre-defined threshold to zero, to produce thresholded spectral amplitudes;

for each of a plurality of sub bands, determining an area under the thresholded spectral amplitudes, and deriving signal feature parameters from the determined areas in accordance with a classifier;

retrieving pre-determined parameters, the parameters being derived from training signal features and the parameters defining at least one diagnostic class in accordance with the classifier; and

classifying the current mental state of the person by comparing derived signal feature parameters with the pre-determined parameters and the classifier.

16. A computing device for classifying a current mental state of a person by analysing natural speech of the person, the computing device comprising a processor configured to:

receive a natural speech signal obtained from the person;

pre-process the natural speech signal to effect frame extraction;

determine spectral amplitudes of frames of the speech signal;

set the spectral amplitudes that are below a pre-defined threshold to zero, to produce thresholded spectral amplitudes;

for each of a plurality of sub bands, determine an area under the thresholded spectral amplitudes, and derive signal feature parameters from the determined areas in accordance with a classifier;

retrieve pre-determined parameters, the parameters being derived from training signal features and the parameters defining at least one diagnostic class in accordance with the classifier; and

classify the current mental state of the person by comparing derived signal feature parameters with the pre-determined parameters and the classifier.

17. The device of claim 16, comprising at least one of:

an electronic programmable device, a mobile phone, a personal computer, a telephone, a personal communications device;

a server effecting web-hosted processing; and

a point of care medical processor.

18. A non-transitory computer storage medium comprising a plurality of computer program instructions to make a computer execute a process for classifying a current mental state of a person by analysing natural speech of the person, the process comprising:

receiving a natural speech signal obtained from the person;

pre-processing the natural speech signal to effect frame extraction;

determining spectral amplitudes of frame of the speech signal;

setting the spectral amplitudes that are below a pre-defined threshold to zero, to produce thresholded spectral amplitudes;

for each of a plurality of sub bands, determining an area under the thresholded spectral amplitudes, and deriving signal feature parameters from the determined areas in accordance with a classifier;

retrieving pre-determined parameters, the parameters being derived from training signal features and the parameters defining at least one diagnostic class in accordance with the classifier; and

classifying the current mental state of the person by comparing derived signal feature parameters with the pre-determined parameters and the classifier.