System and method for characterizing voiced excitations of speech and acoustic signals, removing acoustic noise from speech, and synthesizing speech

• US 7,089,177 B2
• Filed: 08/03/2005
• Issued: 08/08/2006
• Est. Priority Date: 02/06/1996
• Status: Expired due to Fees

• Current Assignee: Lawrence Livermore National Security LLC (Government of the United States of America)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Ng, Lawrence C., Burnett, Greg C., Holzrichter, John F.
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US11/198,287
• Publication Number:
  US 20050278167A1
• Time in Patent Office: 370 Days
• Field of Search: 704/200, 704/203, 704/207, 704/233, 704/270, 704/276
• US Class Current: 704/203
• CPC Class Codes: A61B 5/0507 using microwaves or terahertz waves
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: G01N 2291/02491 Materials with nonlinear acoustic properties
• CPC Class Codes: G01N 2291/02872 Pressure
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

The present invention is a system and method for characterizing human (or animate) speech voiced excitation functions and acoustic signals, for removing unwanted acoustic noise which often occurs when a speaker uses a microphone in common environments, and for synthesizing personalized or modified human (or other animate) speech upon command from a controller. A low power EM sensor is used to detect the motions of windpipe tissues in the glottal region of the human speech system before, during, and after voiced speech is produced by a user. From these tissue motion measurements, a voiced excitation function can be derived. Further, the excitation function provides speech production information to enhance noise removal from human speech and it enables accurate transfer functions of speech to be obtained. Previously stored excitation and transfer functions can be used for synthesizing personalized or modified human speech. Configurations of EM sensor and acoustic microphone systems are described to enhance noise cancellation and to enable multiple articulator measurements.

Claims

1. A method of measuring a sound source of human speech comprising:

directing one or more electromagnetic (EM) waves from an EM sensor toward a speech organ, that is a generating source of a speech sound,

detecting electromagnetic (EM) waves scattered from a sound generating speech organ to measure the conditions of said speech organ while it is generating a speech sound,

detecting acoustic speech output from the speaker to obtain acoustic speech information, and

combining the EM speech information with the acoustic speech information using a speech characterization algorithm.

2. A method of measuring a sound source of human speech comprising:

directing one or more electromagnetic (EM) waves from an EM sensor toward a speech organ, that is a generating source of a speech sound,

where the EM sensor

directs a near-field, non-propagating EM wave to the sound generating speech organ,

detecting electromagnetic (EM) waves scattered from a sound generating speech organ to measure the conditions of said speech organ while it is generating a speech sound, and

where the EM sensor measures the change in the detected EM wave properties caused by the changes in the targeted speech organ during its generation of speech sounds,

detecting acoustic speech output from the speaker to obtain acoustic speech information, and

combining the EM speech information with the acoustic speech information using a speech characterization algorithm.

3. The method of claim 2 where the EM sensor

directs a combined near field, non-propagating EM wave, and a propagating EM wave to the sound generating speech organ and

measures the change in the detected EM wave properties caused by the by the changes in the targeted speech organ during its generation of speech sounds.

4. The method of claim 2 where the detected EM wave property change is due to a change in the dielectric constant, the dielectric'"'"'s location, and or the conductivity of one or more speech organ tissues as it changes while generating speech sounds.

5. The method of claim 2 where the sound generating speech organ is the vocal folds.

6. The method of claim 2 where the sound generating speech organ is the location of a constricted air passage in the vocal tract.

7. The method of claim 2 where

the directed EM wave can be a slowly changing electric (E) field, associated with a voltage or current on a conducting antenna surface, and

the detection of changes in said electric field due to organ condition changes is measured by a change in voltage, current, circuit impedance, or other circuit variable in the circuit attached to the antenna.

8. The method of claim 7 by which the near field EM sensor is affixed to a hand held or bracket-held telephone that is used against the user'"'"'s skin surface.

9. The method of claim 2 where the sound source is related to an excitation function of voiced or unvoiced speech.

10. The method of claim 9 where the excitation function can be a volume air flow or pressure function of voiced speech.

11. The method of claim 2 where the sound generating speech organ measurement is obtaining by measuring an auxiliary tissue motion versus time, whose condition is related to the generating organ.

12. The method of claim 11 where the auxiliary tissue motion is that of a segment of the inside wall of the vocal tract whose condition is related to the sound generating speech organ.

13. The method of claim 11 where the auxiliary tissue is a segment of the speaker'"'"'s skin which is connected to a segment of inside vocal tract wall, said tract wall-motion being related to the condition of the sound generating speech organ.

14. The method of claim 11 where the auxiliary tissues are those surrounding the vocal folds.

15. The method of claim 12 where the auxiliary tissue is inside the section of the vocal tract called the oral cavity, including the tongue surface.

16. The method of claim 13 where the segment of the speaker'"'"'s skin is attached to the inside wall of the oral cavity and may comprise the cheek skin, the jaw skin, the lip surface, and the upper neck skin.