Multi-language speech recognition system

• US 5,758,023 A
• Filed: 09/21/1995
• Issued: 05/26/1998
• Est. Priority Date: 07/13/1993
• Status: Expired due to Fees

• Current Assignee: Theodore Austin Bordeaux
• Original Assignee: Theodore Austin Bordeaux
• Inventors: Bordeaux, Theodore Austin
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US08/532,867
• Time in Patent Office: 978 Days
• Field of Search: 395/209, 395/2.41, 395/2.44, 395/2.86, 395/2.5, 395/2.64-2.66
• US Class Current: 704/232
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band

Abstract

Input speech of an arbitrary speaker is automatically transcribed into one of many pre-determined spoken languages by determining, with the frequency discrimination and frequency response of human hearing, the radiated spectrum of a speech input signal, identifying continuously from that spectrum the phones in the signal, aggregating those phones into phonemes, and translating the phonemic string into the pre-determined spoken language for output.

Claims

1. A multi-language, speaker-independent, continuous speech, substantially unconstrained vocabulary speech recognition system comprising

a language selector for selecting from a plurality of languages a particular language to be recognized,

a store of neuron weights representing a plurality of artificial neural networks pre-trained to recognize phonemes, allophones of phonemes, and other speech sounds having meaning in each of said plurality of languages, each of said plurality of artificial neural networks being designed to operate without the use of a Markov model,

a store of phonetic-natural language dictionaries for each of said plurality of languages,

an analog-to-digital converter for converting speech sounds into digital information,

a high-resolution spectrum analyzer for receiving said digital information from said analog-to-digital converter and for determining, to within a range of plus or minus two hertz accuracy, frequencies contained in said digital information at successive time intervals of four milliseconds or less, and for determining amplitudes of each of said frequencies,

a phone identifier for receiving said frequency and amplitude information from said spectrum analyzer during each of said time intervals and identifying said phonemes, allophones of phonemes, and other speech sounds having meaning in the language being spoken in each of said time intervals, said phone identifier comprising a selected one of a plurality of artificial neural networks corresponding to said particular language to be recognized,

a phoneme integrator for determining when there has been a sufficient consecutive number of identifications of each particular said phoneme, allophone of a phoneme, or other speech sound having meaning in said particular language to be recognized, to meet an established duration criterion for existence of said particular phoneme, allophone of a phoneme, or other speech sound having meaning in said particular language to be recognized, said phoneme integrator providing as an output a phonetic representation of said speech sounds, and

a phoneme string translator for identifying, separating and displaying or storing to a file, natural language words represented by said phonetic representation of said speech sounds.

2. The system of claim 1, wherein said language selector is capable of accepting a selection as to which particular language is to be recognized.

3. The system of claim 1, wherein said spectrum analyzer comprises a plurality of narrowband filters, a distance between center frequencies of said narrowband filters being spaced according to a minimum frequency differential required for human hearing to distinguish between two tones of said center frequencies of said narrowband filters.

4. The system of claim 1, wherein said plurality of artificial neural networks are capable of being trained to recognize phonemes, allophones of phonemes, and other speech sounds having meaning in each of said plurality of languages.

5. The system of claim 1, wherein said phonetic-natural language dictionaries are capable of containing multiple pronunciations of each stored word.

6. The system of claim 1, wherein said system is capable of displaying or storing to a file, text of continuous conversational speech of an arbitrary speaker speaking one of said plurality of languages.

7. The system of claim 1, wherein each of said artificial neural networks is capable of identifying a specific number of time intervals within an overall occurrence of said phoneme, allophone of a phoneme, or a speech sound having meaning in the language being spoken, at which a vocal tract configuration most closely approaches a target configuration of said phoneme, allophone of a phoneme, or other speech sound having meaning in the language being spoken.

8. The system of claim 1, wherein said system is capable of addressing and accounting for coarticulation effects, such as gemination, which occur in conversational speech.

9. A method for multi-language, speaker-independent, continuous speech, unconstrained vocabulary speech recognition comprising the following steps:

selecting a particular language to be recognized from a plurality of languages,

receiving and converting speech sounds into digital information,

receiving said digital information and determining, to within a range of plus or minus two hertz accuracy, frequencies contained in said digital information, at successive time intervals of four milliseconds or less, and determining amplitudes of each of said frequencies,

receiving said frequency and amplitude information during each of said time intervals and identifying said phonemes, allophones of phonemes, and other speech sounds having meaning in said particular language to be recognized in each of said time intervals,

determining when there has been a sufficient consecutive number of identifications of each particular said phoneme, allophone of a phoneme, or other speech sound having meaning in said particular language to be recognized, to meet an established duration criterion for existence of said particular phoneme, allophone of a phoneme, or other speech sound having meaning in said particular language to be recognized,

providing as an output a phonetic representation of said speech sounds, and

identifying, separating and displaying or storing to a file, natural language words represented by said phonetic representation of said speech sounds.

10. The method of claim 9, wherein the step of identifying said phonemes, allophones of phonemes, and other speech sounds having meaning in said particular language to be recognized in each of said time intervals, further comprises the following steps:

identifying a specific number of time intervals within an overall occurrence of said phoneme, allophone of a phoneme, or a speech sound having meaning in said particular language to be recognized, at which a vocal tract configuration most closely approaches a target configuration of said phoneme, allophone of a phoneme, or other speech sound having meaning in said language to be recognized.

11. The method of claim 9, further comprising the following step:

addressing and accounting for coarticulation effects, such as gemination, which occur in conversational speech.

12. The method of claim 9, wherein said method is capable of transcribing continuous conversational speech of an arbitrary speaker into a written form of a predetermined language to be spoken by said arbitrary speaker.

13. A multi-language, speaker-independent, continuous speech, unconstrained vocabulary speech recognition system comprising

a language selector for selecting from a plurality of languages a particular language to be recognized,

a store of neuron weights representing a plurality of artificial neural networks pre-trained to recognize phonemes, allophones of phonemes, and other speech sounds having meaning in each of said plurality of languages, each of said plurality of artificial neural networks being designed to operate without the use of a Markov model,

a store of phonetic-natural language dictionaries for each of said plurality of languages,

an analog-to-digital converter for converting speech sounds into digital information,

a high-resolution spectrum analyzer for receiving said digital information from said analog-to-digital converter, and for determining, to within a range of plus or minus two hertz accuracy, frequencies contained in said digital information at successive time intervals, wherein each of said time intervals is a fixed duration, said duration being selected from a range between one-quarter (0.25) and eight (8) milliseconds, and for determining amplitudes of each of said frequencies,

a phone identifier for receiving said frequency and amplitude information from said spectrum analyzer during each of said time intervals and identifying said phonemes, allophones of phonemes, and other speech sounds having meaning in said particular language to be recognized in each of said time intervals, said phone identifier comprising a selected one of a plurality of artificial neural networks corresponding to said particular language to be recognized,

a phoneme integrator for determining when there has been a sufficient consecutive number of identifications of each particular said phoneme, allophone of a phoneme, or other speech sound having meaning in said language to be spoken, to meet an established duration criterion for existence of said particular phoneme, allophone of a phoneme, or other speech sound having meaning in said particular language to be recognized, said phoneme integrator providing as an output a phonetic representation of said speech sounds, and

a phoneme string translator for identifying, separating and displaying or storing to a file, natural language words represented by said phonetic representation of said speech sounds.