Modification of voice waveforms to change social signaling

• US 8,484,035 B2
• Filed: 09/06/2007
• Issued: 07/09/2013
• Est. Priority Date: 09/06/2007
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Pentland, Alex Paul
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US11/899,460
• Publication Number:
  US 20080044048A1
• Time in Patent Office: 2,133 Days
• Field of Search: 704/256, 704/256.1, 704/260, 704/270, 704/270.1, 704/275, 704/278, 704/258, 704/500, 704/235, 704/246, 704/247, 704/248, 379/265.06, 379/265.07
• US Class Current: 704/278
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

A method of altering a social signaling characteristic of a speech signal. A statistically large number of speech samples created by different speakers in different tones of voice are evaluated to determine one or more relationships that exist between a selected social signaling characteristic and one or more measurable parameters of the speech samples. An input audio voice signal is then processed in accordance with these relationships to modify one or more of controllable parameters of input audio voice signal to produce a modified output audio voice signal in which said selected social signaling characteristic is modified. In a specific illustrative embodiment, a two-level hidden Markov model is used to identify voiced and unvoiced speech segments and selected controllable characteristics of these speech segments are modified to alter the desired social signaling characteristic.

Claims

1. A method of altering a selected real-time social signaling characteristic of an input audio voice signal, which method comprises processing in real-time said input audio voice signal in to modify one or more measurable parameters of said input audio voice signal to produce a modified output audio voice signal in which said selected real-time social signaling characteristic is modified, wherein said input audio voice signal is not generated by a speech synthesizer.

2. The method of claim 1 in which said social signaling characteristic is selected from a group comprising:

interest, attraction, attentiveness, friendliness, submissiveness, and persuasiveness.

3. The method of claim 1 wherein said measurable parameters are selected from a group comprising activity level, engagement, emphasis, pause length entropy, and mirroring,

4. The method of claim 1, wherein the input audio voice signal is from a microphone.

5. The method of claim 4, wherein the input audio voice signal is processed in real time to produce the modified output audio voice signal.

6. The method of claim 1, wherein the input audio voice signal is a live signal.

7. The method of claim 1, wherein both the input audio voice signal and output audio voice signal are digital.

8. A system for modifying an audio input signal, the audio input signal comprising voiced segments and unvoiced segments, and said system comprising, in combination,

a microphone for producing the audio input signal, and

a signal processor for modifying said audio input signal to alter one or more attributes of at least selected ones of the voiced segments to alter one or more real-time social signaling characteristics of the audio input signal to form an audio output signal.

9. The system of claim 8 wherein said one or more social signaling characteristics is persuasiveness and wherein said signal processor alters the duration of said selected ones of said voiced segments.

10. The system of claim 9 wherein said signal processor alters the duration of said selected ones of said voiced segments without significant change in the spectral characteristics of selected ones of said voiced segments.

11. The system of claim 8 wherein:

said signal processor employs a phase vocoder to expand or contract the duration of said selected ones of said voiced segments, and

said phase vocoder performs a Fourier transform at fixed time intervals and calculates frequency changes between these intervals.

12. The system of claim 8 wherein the audio input signal is a live signal.

13. The system of claim 9 wherein said signal processor further regulates the volume of said voiced segments.

14. The system of claim 8, wherein the audio input signal is processed in real time to produce the modified audio output signal.

15. Apparatus for automatically modifying one or more real-time social signaling characteristics of an audio input signal to produce a modified audio output signal comprising, in combination,

a digital signal analyzer for determining the boundaries between speech segments and non-speech segments of said audio input signal,

a digital signal processor for modifying one or more controllable parameters of said speech segments to produce modified speech segments having one or more modified real-time social signaling characteristics, and

output means for combining said modified speech segments with said non-speech segments to produce said modified audio output signal, wherein said audio input signal is from a microphone.

16. The apparatus of in claim 15 wherein said digital signal analyzer further determines the boundaries between voiced and unvoiced segments of said speech segments and wherein said digital signal processor modifies one or more controllable parameters of said voiced segments.

17. The apparatus of claim 15, wherein the audio input signal is a live signal.

18. Apparatus for automatically modifying one or more social signaling characteristics of an audio input signal as set forth in-claim 15 further including means for determining and storing one or more relationships that exist between a given selected social signaling characteristic and at least one of said controllable parameters and wherein said digital signal processor modifies said at least one of said controllable parameters in accordance with at least one of said relationships to control said selected social signaling characteristic.

19. Apparatus for automatically modifying one or more social signaling characteristics of an audio input signal as set forth in claim 18 wherein said digital signal analyzer further determines the boundaries between voiced and unvoiced segments of said speech segments and wherein said digital signal processor modifies controllable parameters of said voiced segments.

20. The apparatus of claim 15, wherein the audio input signal is processed in real time to produce the modified audio output signal.