APPARATUS AND METHOD FOR SPEECH ANALYSIS

• US 20120089396A1
• Filed: 06/16/2010
• Published: 04/12/2012
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Original Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Inventors: Shrivastav, Rahul, Patel, Sona
• Granted Patent: US 8,788,270 B2
• Time in Patent Office: Days
• US Class Current: 704/249
• CPC Class Codes: G10L 25/00 Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34

Abstract

A system that incorporates teachings of the present disclosure may include, for example, an interface for receiving an utterance of speech and converting the utterance into a speech signal, such as digital representation including a waveform and/or spectrum; and a processor for dividing the speech signal into segments and detecting the emotional information from speech. The system is designed by comparing the speech segments to a baseline to identify the emotion or emotions from the suprasegmental information (i.e., paralinguistic information) in speech, wherein the baseline is determined from acoustic characteristics of a plurality of emotion categories. Other embodiments are disclosed.

Claims

1. A method for dote, wining an emotion state of a speaker, comprising:

providing an acoustic space having one or more dimensions, wherein each dimension of the one or more dimensions of the acoustic space corresponds to at least one baseline acoustic characteristic;

receiving a subject utterance of speech by a speaker;

measuring one or more acoustic characteristics of the subject utterance of speech;

comparing each acoustic characteristic of the one or more acoustic characteristics of the subject utterance of speech to a corresponding one or more baseline acoustic characteristic; and

determining an emotion state of the speaker based on the comparison,

wherein determining the emotion state of the speaker based on the comparison occurs within one day of receiving the subject utterance of speech by the speaker.

2. The method according to claim 1, wherein providing an acoustic space comprises analyzing training data to determine the at least one baseline acoustic characteristic for each of the one or more dimensions of the acoustic space.

3. The method according to claim 1, wherein determining the emotion state of speaker based on the comparison comprises determining one or more emotions of the speaker based on the comparison.

4. The method according to claim 1, wherein the emotion state of the speaker comprises a category of emotion and an intensity of the category of emotion.

5. The method according to claim 1, wherein the emotion state of the speaker comprises at least one magnitude along a corresponding at least one of the one or more dimensions within the space.

6. The method according to claim 1, wherein each of the at least one baseline acoustic characteristic for each dimension of the one or more dimensions affects perception of the emotion state.

7. The method according to claim 2, wherein the training data comprises at least one training utterance of speech.

8. The method according to claim 7, wherein the at least one training utterance of speech comprises at least two training utterances of speech.

9. The method according to claim 7, wherein one or more of the at least one training utterance of speech is spoken by the speaker.

10. The method according to claim 7, wherein one or more of the at least one training utterance of speech is spoken by an additional speaker.

11. The method according to claim 7, wherein the subject utterance of speech comprises one or more of the at least one training utterance of speech.

12. The method according to claim 11, wherein semantic and/or syntactic content of the one or more of the at least one training utterance of speech is determined by the speaker.

13. The method according to claim 1, wherein the subject utterance of speech comprises a 2 to 10 second segment of speech.

14. The method according to claim 1, further comprising selecting a segment of speech from the subject utterance of speech, wherein measuring the one or more acoustic characteristics of the subject utterance of speech comprises measuring one or more acoustic characteristic of the segment of speech.

15. The method according to claim 14, wherein the segment of speech from the subject utterance of speech is a 2 to 10 second segment of speech from the subject utterance of speech.

16. The method according to claim 15, wherein the segment of speech from the subject utterance of speech is a 3 to 5 second segment of speech from the subject utterance of speech.

17. The method according to claim 14, further comprising:

selecting an additional segment of speech from the subject utterance of speech;

measuring one or more additional acoustic characteristics of the additional segment of speech, wherein each one or more additional acoustic characteristic of the additional segment of speech corresponds to a corresponding one or more baseline acoustic characteristic;

comparing each one or more additional acoustic characteristic of the additional segment of speech to the corresponding one or more baseline acoustic characteristic; and

determining an additional emotion state of the speaker based on the comparison.

18. The method according to claim 17, wherein the segment of speech from the subject utterance of speech and the additional segment of speech from the subject utterance of speech are of different lengths.

19. The method according to claim 1, wherein at least one of the one or more acoustic characteristic of the subject utterance of speech comprises a suprasegmental property of the subject utterance of speech, and corresponding at least one of the one or more baseline acoustic characteristic comprises a corresponding suprasegmental property.

20. The method according to claim 1, wherein each of the one or more acoustic characteristic of the subject utterance of speech is selected from the group consisting of:

fundamental frequency, pitch, intensity, loudness, and speaking rate.

21. The method according to claim 1, wherein each of the one or more acoustic characteristic of the subject utterance of speech is selected from the group consisting of:

number of peaks in the pitch, intensity contour, loudness contour, pitch contour, fundamental frequency contour, attack of the intensity contour, attack of the loudness contour, attack of the pitch contour, attack of the fundamental frequency contour, fall the intensity contour, fall of the loudness contour, fall of the pitch contour, fall of the fundamental frequency contour, duty cycle of the peaks in the pitch, normalized minimum pitch, normalized maximum of pitch, cepstral peak prominence (CPP), and spectral slope.

22. The method according to claim 1, wherein determining the emotion state of the speaker based on the comparison occurs within one minute of receiving the subject utterance of speech by the speaker.

23. The method according to claim 1, wherein determining the emotion state of the speaker based on the comparison occurs within 30 seconds of receiving the subject utterance of speech by the speaker.

24. The method according to claim 1, wherein determining the emotion state of the speaker based on the comparison occurs within 15 seconds of receiving the subject utterance of speech by the speaker.

25. The method according to claim 1, wherein determining the emotion state of the speaker based on the comparison occurs within 10 seconds of receiving the subject utterance of speech by the speaker.

26. The method according to claim 1, wherein determining the emotion state of the speaker based on the comparison occurs within 5 seconds of receiving the subject utterance of speech by the speaker.

27. A method for determining an emotion state of a speaker, comprising:

providing an acoustic space having one or more dimensions, wherein each dimension of the one or more dimensions of the acoustic space corresponds to at least one baseline acoustic characteristic;

receiving a subject utterance of speech by a speaker;

measuring one or more acoustic characteristic of the subject utterance of speech;

comparing each acoustic characteristic of the one or more acoustic characteristic of the subject utterance of speech to a corresponding one or more baseline acoustic characteristic; and

determining an emotion state of the speaker based on the comparison, wherein the emotion state of the speaker comprises at least one magnitude along a corresponding at least one of the one or more dimensions within the acoustic space.

28-48. -48. (canceled)

49. A method for determining an emotion state of a speaker, comprising:

providing an acoustic space having one or more dimensions, wherein each dimension of the one or more dimensions of the acoustic space corresponds to at least one baseline acoustic characteristic;

receiving a training utterance of speech by the speaker;

analyzing the training utterance of speech;

modifying the acoustic space based on the analysis of the training reference of speech to produce a modified acoustic space having one or more modified dimensions, wherein each modified dimension of the one or more modified dimensions of the modified acoustic space corresponds to at least one modified baseline acoustic characteristic;

receiving a subject utterance of speech by a speaker;

measuring one or more one acoustic characteristic of the subject utterance of speech;

comparing each acoustic characteristic of the one or more acoustic characteristics of the subject utterance of speech to a corresponding one or more one baseline acoustic characteristic; and

determining an emotion state of the speaker based on the comparison.

50-59. -59. (canceled)

60. A method of creating a perceptual space, comprising:

obtaining listener judgments of differences in perception in at least two emotions from one or more speech utterances;

measuring d′

values between each of the at least two creations, and each of the remain at least two emotions, wherein the d′

values represent perceptual distances between emotions;

applying a multidimensional scaling analysis to the measured d′

values; and

creating a n−

1 dimensional perceptual space.

61-62. -62. (canceled)