Method and apparatus for determining emotional arousal by speech analysis

• US 7,606,701 B2
• Filed: 08/07/2002
• Issued: 10/20/2009
• Est. Priority Date: 08/09/2001
• Status: Active Grant

• Current Assignee: VoiceSense Ltd.
• Original Assignee: VoiceSense Ltd.
• Inventors: Degani, Yoav, Zamir, Yishai
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): He; Jialong
• Application Number:
  US10/485,254
• Publication Number:
  US 20040249634A1
• Time in Patent Office: 2,631 Days
• Field of Search: 704/207, 704/246, 704/270, 704/273, 704/217
• US Class Current: 704/207
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

An apparatus for determining emotional arousal of a subject by speech analysis, and an associated method. In the method, a speech sample is obtained, the speech sample is pre-processed into silent and active speech segments and the active speech segments are divided into strings of equal length blocks (the blocks having primary speech parameters including pitch and amplitude parameters), a plurality of selected secondary speech parameters indicative of characteristics of equal-pitch are derived, rising-pitch and falling-pitch trends in the strings of blocks, the secondary speech parameters are compared with predefined, subject independent values representing non-emotional speech to generate a processing result indicative of emotional arousal, and the generated processed result is outputted to an output device.

Claims

1. A method for determining emotional arousal of a subject by speech analysis, comprising the steps of:

obtaining a speech sample;

pre-processing the speech sample into silent and active speech segments and dividing the active speech segments into strings of equal length blocks;

said blocks having primary speech parameters including pitch and amplitude parameters;

deriving a plurality of selected secondary speech parameters indicative of characteristics of equal-pitch, rising-pitch and falling-pitch trends in said strings of blocks;

comparing said secondary speech parameters with predefined, subject independent values representing non-emotional speech to generate a processing result indicative of emotional arousal, and outputting said generated processed result to an output device, wherein said secondary speech parameters comprise;

(a) average length of short silences and number of short silences per unit of time;

(b) average length of equal pitch segments and number of equal pitch segments per unit of time;

(c) rising pitch segments length average and number of rising pitch segments per unit of time and falling pitch segments length average and number of falling pitch segments per unit of time; and

(d) average amplitude dispersion within equal pitch segments of speech.

2. The method according to claim 1, wherein said step of deriving further includes deriving a plurality of selected secondary speech parameters indicative of pause and silence characteristics of the speech sample being analyzed.

3. The method according to claim 1, including analyzing irregularity of pace and rhythm, pitch, and amplitude of the speech sample being analyzed.

4. The method according to claim 1, wherein said step of obtaining a sample of speech comprises the step of inputting a digitized voice file.

5. The method according to claim 1, wherein said step of obtaining a sample of speech comprises the step of capturing speech specimens and sampling and digitizing the speech specimens in a voice sampling and digitizing unit to form a digitized voice file.

6. The method according to claim 1, wherein the step of pre-processing includes:

obtaining digitized voice samples,

normalizing said voice samples,

reducing noise,

segmenting said voice samples into silence and speech segments, dividing the speech segments into blocks, and

processing said blocks by auto-correlation, to calculate pitch and amplitude speech parameters per block.

7. The method according to claim 1, adapted for analyzing a speech signal including a plurality of interacting voices, further comprising:

separating the interacting voices into separate voice channels;

performing samples normalization for each channel of interest;

performing noise-reduction for each channel of interest;

performing silence and speech segmentation and dividing the speech segments into blocks for each channel of interest, and

performing auto-correlation processing to calculate pitch and amplitude speech parameters per block for each channel of interest.

8. The method according to claim 1, wherein the step of deriving includes:

marking a speech segment of a pre-defined length for processing;

calculating pauses related parameters for said speech segment;

calculating silences related parameters for said speech segment;

joining blocks into strings of blocks categorized as being strings of blocks having rising pitch trends, strings of blocks having falling pitch trends and strings of blocks having equal pitch trends;

calculating pitch related parameters within the speech segment, said pitch related parameters comprising average length of equal pitch segments, number of equal pitch segments per unit of time, rising pitch segments length average, number of rising pitch segments per unit of time, falling pitch segments length average, number of falling pitch segments per unit of time and average amplitude dispersion within equal pitch segments of speech, and

classifying the speech segment into one of several categories of typical parameter range.

9. The method according to claim 1, further comprising calculating a reliability grade based on quality of voice segment, significance of emotional arousal decision, and consistency of specific segment results with results of previous speech segments.

10. The method according to claim 9, wherein said quality of voice segment is determined, based on noise level, size of sampled data, and quality of sampled data.

11. The method according to claim 9, wherein said significance of emotional arousal decision is determined, based on number of participating parameters and degree of deviation within each parameter.

12. The method according to claim 1, further comprising pattern processing to detect emotional patterns that are revealed along a time axis.

13. The method according to claim 1, wherein:

said obtaining step is carried out using a voice input unit;

said preprocessing is carried out using a pre-processing unit for pre-processing voice samples from voice input unit;

said deriving and comparing steps are carried out using a main processing unit for processing said pre-processed voice samples and detecting emotional arousal therefrom; and

said processing result is outputted to a main indicators output unit for outputting an indication of emotional arousal.

14. The method according to claim 13, wherein said voice input unit of the apparatus includes a voice capturing unit and a voice sampling and digitizing unit coupled to said voice capturing unit for sampling and digitizing captured voice input.

15. The method according to claim 13, wherein said voice input unit of the apparatus includes at least one element selected from the group consisting of:

a microphone, an interface to an audio player, an interface to a wired, wireless or cellular telephone, an interface to the Internet or other network, an interface to a computer, an interface to an electronic personal organizer or to any other electronic equipment, and an interface to a toy.

16. The method according to claim 14, wherein said voice sampling and digitizing unit of the apparatus is selected from the group consisting of a sound card, and a DSP chip-based voice sampling and digitizing device.

17. The method according to claim 13, wherein said main indicators output unit of the apparatus is selected from the group consisting of a local output device, a display, a speaker, a file, a storage unit, a monitoring device, and an interface to a remote computer, to the Internet, to another network, to a wired, wireless or cellular telephone, to a computer game, to a toy, to an electronic personal organizer or to any other electronic output equipment.

18. The method according to claim 13, wherein all said units of the apparatus are installed on a small, mobile, DSP chip based unit.

19. The method according to claim 13, wherein some of said units of the apparatus are physically distanced from other units, and said apparatus further comprises an interface for allowing data communication between said units.

20. The method according to claim 13, wherein said pre-processing and said processing units of the apparatus incorporate a software tool capable of integrating with an external source of digitized voice input and with an external output device.