Method and apparatus for speech analysis

• US 20040249634A1
• Filed: 03/25/2004
• Published: 12/09/2004
• Est. Priority Date: 08/09/2001
• Status: Active Grant

• Current Assignee: VoiceSense Ltd.
• Original Assignee: VoiceSense Ltd.
• Inventors: Degani, Yoav, Zamir, Yishai
• Granted Patent: US 7,606,701 B2
• Time in Patent Office: Days
• US Class Current: 704/207
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

An apparatus for determining emotional arousal of a subject by speech analysis, and an associated method. In the method, a speech sample is obtained, the speech sample is pre-processed into silent and active speech segments and the active speech segments are divided into strings of equal length blocks (the blocks having primary speech parameters including pitch and amplitude parameters), a plurality of selected secondary speech parameters indicative of characteristics of equal-pitch are derived, rising-pitch and falling-pitch trends in the strings of blocks, the secondary speech parameters are compared with predefined, subject independent values representing non-emotional speech to generate a processing result indicative of emotional arousal, and the generated processed result is outputted to an output device.

Claims

1. A method for determining emotional arousal of a subject by speech analysis, comprising the steps of:

obtaining a speech sample;

pre-processing the speech sample into silent and active speech segments and dividing the active speech segments into strings of equal length blocks;

said blocks having primary speech parameters including pitch and amplitude parameters;

deriving a plurality of selected secondary speech parameters indicative of characteristics of equal-pitch, rising-pitch and falling-pitch trends in said strings of blocks;

comparing said secondary speech parameters with predefined, subject independent values representing non-emotional speech to generate a processing result indicative of emotional arousal, and outputting said generated processed result to an output device, wherein said plurality of selected secondary speech parameters is selected from the list of;

(a) Average length of short silences and/or short silences frequency;

(b) Average length of equal pitch segments and/or equal pitch segments frequency;

(c) Rising pitch segments length average and/or rising pitch segments frequency and/or falling pitch segments length average and/or falling pitch segments frequency; and

(d) The average amplitude dispersion within equal pitch segments of speech.

2. The method according to claim 1, wherein said step of deriving further includes deriving a plurality of selected secondary speech parameters indicative of pause and silence characteristics of the speech sample being analyzed.

3. The method according to claim 1, including analyzing irregularity of pace and rhythm, pitch, and amplitude of the speech sample being analyzed.

4. The method of claim 1, wherein said plurality of selected secondary speech parameters (is selected from the list of:

) further includes at least one of the list of average pause length and (/or) pause frequency.

5. The method according to claim 1, wherein said step of obtaining a sample of speech comprises the step of inputting a digitized voice file.

6. The method according to claim 1, wherein said step of obtaining a sample of speech comprises the step of capturing speech specimens and sampling and digitizing the speech specimens in a voice sampling and digitizing unit to form a digitized voice file.

7. The method according to claim 1, wherein the step of pre-processing includes:

obtaining digitized voice samples,

normalizing said voice samples,

data filtering,

noise-reduction,

segmenting said voice samples into silence and speech segments, dividing the speech segments into blocks, and

processing said blocks by auto-correlation, to calculate pitch and amplitude voice parameters per block.

8. The method according to claim 1, adapted for analyzing a speech signal including a plurality of interacting voices, further comprising:

separating the interacting voices into separate voice channels;

performing samples normalization for each channel of interest;

performing data filtering for each channel of interest;

performing noise-reduction for each channel of interest;

performing silence and speech segmentation and dividing the speech segments into blocks for each channel of interest, and

auto-correlation processing to calculate pitch and amplitude voice parameters per block for each channel of interest.

9. The method according to claim 1, wherein the step of deriving includes:

marking a speech segment of a pre-defined length for processing;

calculating pauses related parameters for said speech segment;

calculating silences related parameters for said speech segment;

joining blocks into strings of blocks categorized as being strings of blocks having rising pitch trends, strings of blocks having falling pitch trends and strings of blocks having equal pitch trends;

calculating pitch related parameters within the speech segment, said pitch related parameters selected from the list of frequency and average lengths of strings of blocks characterized by having rising, falling or equal pitch trends, and average amplitude dispersion of strings of blocks having equal pitch, and

classifying the speech segment into one of several categories of typical parameter range.

10. The method according to claim 1, wherein the step of comparing said secondary speech parameters with predefined, subject independent values representing non-emotional speech to generate a processing result indicative of emotional arousal includes comparing at least two secondary voice parameter categories with pre-defined values representing non-emotional speech, said categories being selected from the list of:

(a) Average pause length and/or pause frequency;

(b) Average length of short silences and/or short silences frequency;

(c) Average length of equal pitch segments and/or equal pitch segments frequency;

(d) Rising pitch segments length average and/or rising pitch segments frequency and/or falling pitch segments length average and/or falling pitch segments frequency; and

(e) The average amplitude dispersion within equal pitch segments of speech

11. The method according to claim 1, further comprising calculating a reliability grade based on at least one factor selected from the list of:

quality of voice segment;

significance of emotional arousal decision, and consistency of specific segment results with results of previous speech segments.

12. The method according to claim 11, wherein said quality of voice segment is determined, based on noise level, size of sampled data, and quality of sampled data.

13. The method according to claim 11, wherein said significance of emotional arousal decision is determined, based on number of participating parameters and degree of deviation within each parameter.

14. The method according to claim 1, further comprising pattern processing to detect emotional patterns that are revealed along a time axis.

15. An apparatus with means for executing the speech analysis method according to claim 1, comprising:

a voice input unit;

a pre-processing unit for pre-processing voice samples from voice input unit;

a main processing unit for processing said pre-processed voice samples and detecting emotional arousal therefrom; and

a main indicators output unit for outputting an indication of emotional arousal.

16. The apparatus according to claim 15, wherein said voice input unit includes a voice capturing unit and a voice sampling and digitizing unit coupled to said voice capturing unit for sampling and digitizing captured voice input.

17. The apparatus according to claim 15, wherein said voice input unit includes at least one of the following:

a microphone, an interface to an audio player, an interface to a wired, wireless or cellular telephone, an interface to the Internet or other network, an interface to a computer, an interface to an electronic personal organizer or to any other electronic equipment, or an interface to a toy.

18. The apparatus according to claim 16, wherein said voice sampling and digitizing unit is selected from a sound card, or a DSP chip-based voice sampling and digitizing device.

19. The apparatus according to claim 15, wherein said main indicators output unit is selected from a local output device, a display, a speaker, a file, a storage unit or monitoring device;

or an interface to a remote computer, to the Internet, to another network, to a wired, wireless or cellular telephone, to a computer game, to a toy, to an electronic personal organizer or to any other electronic output equipment.

20. The apparatus according to claim 15, wherein all said units are installed on a small, mobile, DSP chip based unit.

21. The apparatus according to claim 15, wherein some of said units are physically distanced from other units, and said apparatus further comprises an interface for allowing data communication between said units.

22. The apparatus according to claim 15, wherein said pre-processing and said processing units are incorporated within a software tool capable of integrating with an external source of digitized voice input and with an external output device.