System and method for characterizing voiced excitations of speech and acoustic signals, removing acoustic noise from speech, and synthesizing speech

• US 6,711,539 B2
• Filed: 05/08/2001
• Issued: 03/23/2004
• Est. Priority Date: 02/06/1996
• Status: Expired due to Fees

• Current Assignee: Lawrence Livermore National Security LLC (Government of the United States of America)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Ng, Lawrence C., Burnett, Greg C., Holzrichter, John F.
• Primary Examiner(s): To, Doris H.
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/851,550
• Publication Number:
  US 20010021905A1
• Time in Patent Office: 1,050 Days
• Field of Search: 704/223, 704/226
• US Class Current: 704/223
• CPC Class Codes: A61B 5/0507 using microwaves or terahertz waves
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: G01N 2291/02491 Materials with nonlinear acoustic properties
• CPC Class Codes: G01N 2291/02872 Pressure
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

The present invention is a system and method for characterizing human (or animate) speech voiced excitation functions and acoustic signals, for removing unwanted acoustic noise which often occurs when a speaker uses a microphone in common environments, and for synthesizing personalized or modified human (or other animate) speech upon command from a controller. A low power EM sensor is used to detect the motions of windpipe tissues in the glottal region of the human speech system before, during, and after voiced speech is produced by a user. From these tissue motion measurements, a voiced excitation function can be derived. Further, the excitation function provides speech production information to enhance noise removal from human speech and it enables accurate transfer functions of speech to be obtained. Previously stored excitation and transfer functions can be used for synthesizing personalized or modified human speech. Configurations of EM sensor and acoustic microphone systems are described to enhance noise cancellation and to enable multiple articulator measurements.

Claims

1. A method for removing acoustic noise from speech, comprising the steps of:

obtaining a speech excitation function using an EM sensor;

identifying a first voiced-excitation onset time from the excitation function;

obtaining an acoustic speech signal, corresponding to the speech excitation function;

subtracting a first predetermined unvoiced time period from the first voiced-excitation onset time to obtain a corresponding first unvoiced-acoustic onset time within the acoustic speech signal;

defining a no-speech time period prior to the first unvoiced-acoustic onset time;

measuring a acoustic noise within the no-speech time period; and

reducing the acoustic noise in the acoustic speech signal.

2. The method of claim 1 wherein the step of defining a no-speech time period comprises the steps of:

identifying an initialization time prior to a beginning of speech; and

defining the no-speech time period between the initialization time and the first unvoiced-acoustic onset time.

3. The method of claim 1 wherein the step of defining a no-speech time period comprises the steps of:

identifying a second voiced-excitation end-time from the excitation function;

adding a second predetermined unvoiced time period to the second voiced-excitation end-time to obtain a corresponding second unvoiced-acoustic end-time within the acoustic speech signal; and

defining the no-speech time period as a period between second unvoiced-acoustic end-time and the first unvoiced-acoustic onset time.

4. The method of claim 1:

further comprising the step of, constructing an acoustic noise filter over a first set of time frames; and

wherein the reducing step is comprised of the steps of,

selecting an acoustic noise filter corresponding to the acoustic speech signal and the acoustic noise; and

filtering the acoustic noise from the acoustic signal over a second set of time frames using the acoustic noise filter.

5. The method of claim 1 further comprising the steps of:

defining time frames within the excitation function and acoustic speech signal based on glottal tissue configuration; and

identifying a subset of the time frames where the excitation function is substantially constant; and

wherein the reducing step is comprised of the step of averaging amplitudes of the acoustic speech signal over the subset.

6. The method of claim 1:

wherein the obtaining an acoustic speech signal step includes the step of capturing the acoustic speech signal with an audio system; and

further comprising steps of,

identifying positive growing instabilities in the acoustic speech signal; and

damping the instabilities by adjusting the audio system.

7. The method of claim 1:

wherein the measuring acoustic noise step, includes the step of,

detecting an echo signal within the acoustic speech signal corresponding to a voiced-speech portion of the acoustic speech signal;

further including the step of,

identifying a portion of the acoustic speech signal corresponding to the echo signal; and

wherein the reducing step, includes the step of,

sign, amplitude, and phase adjusting the portion of the acoustic speech signal to cancel the echo signal.

8. The method of claim 1:

wherein the measuring acoustic noise step, includes the step of,

measuring background acoustic-noise with a microphone; and

wherein the reducing step, includes the step of,

sign, amplitude, and phase adjusting the background acoustic-noise to reduce the acoustic noise.

9. The A method for removing acoustic noise from an acoustic speech signal, comprising the steps of:

selecting a first set of acoustic speech time frames with timing defined by an excitation function determined using an EM sensor;

characterizing qualities of an acoustic noise signal over a second set of time frames with timing defined by an excitation function determined using the EM sensor and by using the acoustic speech signal over said second set of time frames;

constructing an acoustic noise filter appropriate to the acoustic speech signal over the first set of time frames and to the characterized noise signal over the second set of time frames; and

filtering the acoustic noise signal from the acoustic speech signal over the first set of time frames using the acoustic noise filter, wherein;

the characterizing step includes the step of characterizing the qualities of the acoustic noise signal over the first set of time frames; and

the constructing step includes the step of constructing the acoustic noise filter using both acoustic speech signal and noise signal information over the first set of time frames, and

wherein the constructing step further includes the steps of;

selecting a first set of acoustic speech time frames corresponding to a set of voiced speech excitation functions;

constructing a speech band-pass filter using spectral information of the voiced speech excitation function obtained using the EM sensor over the first set of time frames;

characterizing the acoustic noise over the first set of acoustic speech time frames using the acoustic-signal spectral-information excluded by the speech band-pass filter that is constructed using spectral information of the voiced speech excitation function;

constructing the acoustic noise filter over the first set of time frames by using the band-pass filter and the characterized acoustic noise; and

filtering the acoustic noise from the acoustic signal over the first set of time frames using the acoustic noise filter.

10. The A method for removing acoustic noise from an acoustic speech signal, comprising the steps of:

selecting a first set of acoustic speech time frames with timing defined by an excitation function determined using an EM sensor;

characterizing qualities of an acoustic noise signal over a second set of time frames with timing defined by an excitation function determined using the EM sensor and by using the acoustic speech signal over said second set of time frames;

constructing an acoustic noise filter appropriate to the acoustic speech signal over the first set of time frames and to the characterized noise signal over the second set of time frames; and

filtering the acoustic noise signal from the acoustic speech signal over the first set of time frames using the acoustic noise filter,

partitioning the acoustic speech signal into time frames;

calculating an acoustic speech signal energy;

calculating an excitation function energy;

averaging the acoustic speech signal energy over a subset of the time frames;

averaging the excitation function energy over the subset of the time frames; and

replacing a portion of the acoustic speech signal in a first time frame with a portion of the acoustic speech signal in a second time frame, if a change in the acoustic speech signal energy in the first time frame exceeds a predetermined threshold, and if the corresponding excitation energy remains constant within predetermined threshold levels.

11. A system for removing acoustic noise from speech, comprising:

an EM sensor for generating a speech excitation function from measured movements of a predetermined portion of a vocal tract;

an acoustic sensor receiving an acoustic speech signal, corresponding to the speech excitation function from the vocal tract; and

a computer for,

identifying a first voiced-excitation onset time from the excitation function,

subtracting a first predetermined unvoiced time period from the first voiced-excitation onset time to obtain a corresponding first unvoiced-acoustic onset time within the acoustic speech signal;

defining a no-speech time period prior to the first unvoiced-acoustic onset time;

measuring acoustic noise within the no-speech time period; and

reducing the acoustic noise in the acoustic speech signal.