System and method for speech recognition

• US 4,181,813 A
• Filed: 05/08/1978
• Issued: 01/01/1980
• Est. Priority Date: 05/08/1978
• Status: Expired due to Term

• Current Assignee: John Marley
• Original Assignee: John Marley
• Inventors: Marley, John
• Primary Examiner(s): Brown, Thomas W.
• Assistant Examiner(s): Kemeny, E. S.
• Application Number:
  US05/903,697
• Time in Patent Office: 603 Days
• Field of Search: 179/1 SD, 179/1 SB, 179/1 SC, 179/1 SE
• US Class Current: 704/251
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

A system and method for speech recognition provides a means of printing phonemes in response to received speech signals utilizing inexpensive components. The speech signals are inputted into an amplifier which provides negative feedback to normalize the amplitude of the speech signals. The normalized speech signals are delta modulated at a first sampling rate to produce a corresponding first sequence of digital pulses. The negative feedback signal of the amplifier is delta modulated at a second sampling rate to produce a second sequence of digital pulses corresponding to amplitude information of the speech signals. The speech signals are filtered and utilized to produce a digital pulse corresponding to high frequency components of the speech signals having magnitudes in excess of a threshold voltage. A microprocessor contains an algorithm for detecting major slope transitions of the analog speech signals in response to the first sequence of digital signals by detecting information corresponding to presence and absence of predetermined numbers of successive slope reversals in the delta modulator producing the first sequence of digital pulses. The algorithm computes cues from the high frequency digital pulse and the second sequence of pulses. The algorithm computes a plurality of speech waveform characteristic ratios of time intervals between various slope transitions and compares the speech waveform characteristic ratios with a plurality of stored phoneme ratios representing a set of phonemes to detect matching therebetween. The order of comparing is determined on the basis of the cues and a configuration of a phoneme decision tree contained in the algorithm. When a matching occurs, a signal corresponding to the matched phoneme is produced and utilized to cause the phoneme to be printed. In one embodiment of the invention, the speech signals are produced by the earphone of a standard telephone headset.

Claims

1. A method for producing a signal representing a phoneme sound contained in a stream of voice signals, said method comprising the steps of:

(a) producing a first sequence of analog speech signals representing said voice signals;

(b) delta modulating said first sequence of analog speech signals to produce a sequence of digital pulses representing phonemic information contained in said analog speech signals;

(c) operating upon said sequence of digital pulses to detect major slope transitions of said analog speech signals;

(d) measuring time intervals between predetermined ones of said detected major slope transitions;

(e) computing a plurality of speech waveform characteristic ratios between predetermined ones of said time intervals;

(f) comparing said speech waveform characteristic ratios with a plurality of stored phoneme ratios to determine if said speech waveform characteristic ratios match any of said stored phoneme characteristic ratios; and

(g) producing a signal representing a phoneme sound corresponding to a matching one of said phoneme characteristic ratios.

2. The method of claim 1 wherein said delta modulating step includes the steps of:

(a) comparing a positively and negatively ramping signal to said sequence of digital pulses;

(b) periodically comparing the instantaneous level of said analog speech signals at a predetermined rate with the instantaneous level of said ramping signal;

(c) reversing the slope of said ramping signal if said ramping signal is positive-going and exceeds said instantaneous level, and reversing the slope of said ramping signal if said ramping signal is negative-going and is less than said instantaneous level; and

(d) producing said digital pulses in accordance with said slope reversing.

3. The method of claim 2 wherein said operating upon said sequence of digital pulses to detect said major slope transitions includes the steps of detecting presence and absence of predetermined numbers of successive slope reversals of said ramping signal.

4. The method of claim 1 further including sequentially repeating steps (a) through (f) to produce a sequence of signals representing a plurality of phoneme sounds in response to said first sequence of analog speech signals.

5. The method of claim 4 further including the steps of producing an amplitude envelope signal representing amplitude variations of said voice signals, delta modulating said amplitude envelope signal to produce a second sequence of digital pulses representing amplitude information contained in said amplitude envelope signal, and utilizing said second sequence of digital pulses to determine transition points in said voice signals.

6. The method of claim 5 further including the step of utilizing said second sequence of digital pulses to determine the order of said comparing in step (f).

7. The method of claim 3 further including a plurality of feature extraction steps after said time interval measuring, said feature extraction steps including:

(a) producing a first sequence of first logic variable segments each defined by a respective one of said time intervals and a logic state determined by the slope of said ramping signal if said slope remains unreversed for a predetermined time and storing said first sequence of first logic variable segments in a first memory stack;

(b) determining a period of said analog speech signals by summing the ones of said stored first logic variable segments between a most recently stored one of said first logic variable segments and a like previously stored one of said first logic variable segments; and

(c) dividing predetermined ones of said first logic variable segments by said period to obtain certain ones of said speech waveform characteristic ratios.

8. The method of claim 7 further including the steps of:

(a) producing a second sequence of second logic variable segments each defined by a respective one of said time intervals and a logic state determined by occurrence of a predetermined number of consecutive slope reversals of said ramping signal, and storing said second sequence of second logic variable segments in a second memory stack; and

(b) dividing predetermined ones of said second logic variable segments by said period to obtain certain other ones of said waveform characteristic ratios.

9. The method of claim 1 wherein said comparing is in accordance with a phoneme classifier subroutine, said phoneme classifier subroutine selecting various ones of said stored phoneme ratios for comparison with said speech waveform characteristic ratios according to a phoneme classifier decision tree, said phoneme classifier decision tree including a plurality of branches each representing a plurality of phoneme ratios of related phoneme sounds, the phoneme ratios of each branch being selected in decreasing order of their average frequency of occurrence in a spoken language.

10. The method of claim 9 further including the steps of:

(a) producing an amplitude envelope signal representing amplitude variations of said voice signals;

(b) serially encoding said amplitude envelope signal to produce a second sequence of serial digital pulses representing amplitude information of said amplitude envelope signals;

(c) analyzing said second sequence of serial digital pulses to produce transition information representing transitions and transitional glides of said voice signals;

(d) utilizing said transition information in said phoneme classifier subroutine to aid in selecting one of said branches; and

(e) sequentially selecting stored phoneme ratios in said branch for said comparing.

11. The method of claim 9 further including the steps of:

(a) detecting frequency components of said voice signals in a predetermined frequency range and exceeding a predetermined threshold magnitude;

(b) producing a digital signal representing said detecting frequency component; and

(c) utilizing said frequency component representing digital signals in said phoneme classifier subroutine to aid in selecting one of said branches and in distinguishing between predetermined ones of said stored phoneme ratios and said speech waveform characteristic ratios.

12. The method of claim 9 further including the printing a phoneme symbol in response to said phoneme sound representing signal.

13. The method of claim 12 wherein said comparing step includes:

(a) sequentially comparing said selected ones of said stored phoneme ratios with said speech waveform characteristic ratios until an adequate matching occurs;

(b) if an adequate matching results from any of said sequential comparing steps, comparing said matched phoneme ratios with a most recently matched previous phoneme ratio, and repeating said sequential comparing steps if no adequate matching results; and

(c) if said matched phoneme ratio is the same as said previous phoneme ratio, producing said phoneme sound representing signal to cause printing of said phoneme symbol, and repeating said sequential comparing steps if said matched phoneme ratio is not the same as said previous phoneme ratio, said printed phoneme symbol corresponding to said previous phoneme ratio.

14. The method of claim 1 wherein said delta modulating of said first sequence of analog speech signals occurs in accordance with a sampling rate of approximately fifty kilohertz.

15. The method of claim 5 wherein said delta modulating of said amplitude envelope signal occurs in accordance with a sampling rate of approximately 50 to 500 Hertz.

16. The method of claim 3 wherein step (a) further includes the step of controlling the magnitude of the slope of said ramping signal inversely with respect to amplitude variations of said voice signals to reduce the effect of amplitude variations of said voice signals on said sequence of digital pulses.

17. A method for producing a signal representing a phoneme in response to a sequence of analog speech signals, said method comprising the steps of:

(a) serially encoding said sequence of analog speech signals to produce a corresponding sequence of serial digital pulses corresponding to ramp slope reversals of a delta modulator circuit;

(b) detecting major slope transitions of said analog speech signals by detecting presence and absence of predetermined numbers of successive slope reversals corresponding to said sequence of digital pulses;

(c) computing a speech waveform characteristic ratio of time intervals between certain ones of said slope transitions;

(d) comparing said speech waveform characteristic ratio with a stored phoneme ratio to determine if said speech waveform characteristic ratio matches said stored phoneme ratio; and

(e) producing a signal representing a phoneme corresponding to said phoneme ratio if said matching occurs.

18. A system for producing signals representing a phoneme contained in a stream of voice signals, said system comprising in combination:

(a) means for producing a first sequence of analog speech signals representing said voice signals;

(b) means for delta modulating said first sequence of analog speech signals to produce a sequence of digital pulses representing phonemic information contained in said analog speech signals;

(c) means for detecting major slope transitions of said analog speech signals in response to said sequence of digital pulses;

(d) means for measuring time intervals between predetermined ones of said detected major slope transitions;

(e) means for computing a plurality of speech waveform characteristic ratios between predetermined ones of said time intervals;

(f) means for comparing said speech waveform characteristic ratios with a plurality of stored phoneme ratios to determine if said speech waveform characteristic ratios match any of said stored phoneme characteristic ratios; and

(g) means responsive to said comparing means for producing a signal representing a phoneme corresponding to a matching one of said phoneme characteristic ratios.

19. The system of claim 18 wherein said delta modulating means includes:

(a) means for comparing a positively and negatively ramping signal to said sequence of digital pulses;

(b) means for periodically comparing the instantaneous level of said analog speech signals at a predetermined rate with the instantaneous level of said ramping signal;

(c) means for reversing the slope of said ramping signal if said ramping signal is positive-going and exceeds said instantaneous level, and reversing the slope of said ramping signal if said ramping signal is negative-going and is less than said instantaneous level; and

(d) means for producing said digital pulses in accordance with said slope reversing.

20. The method of claim 19 wherein said major slope transition detecting means includes means for detecting presence and absence of predetermined numbers of successive slope reversals of said ramping signal.

21. The system of claim 18 further including means for producing an amplitude envelope signal representing amplitude variations of said voice signals, means for delta modulating said amplitude envelope signal to produce a second sequence of digital pulses representing amplitude information contained in said amplitude envelope signal, and means for determining transition points in said voice signals in response to said second sequence of digital pulses.

22. The system of claim 20 wherein said speech waveform characteristic ratio computing means includes feature extraction means, said feature extraction means including:

(a) means for producing a first sequence of first logic variable segments each defined by a respective one of said time intervals and a logic state determined by the slope of said ramping signal if said slope remains unreversed for a predetermined time;

(b) first means for storing said first sequence of first logic variable segments;

(c) means for determining a period of said analog speech signals by summing the ones of said stored first logic variable segments between a most recently stored one of said first logic variable segments and a like previously stored one of said first logic variable segments; and

(d) means for dividing predetermined ones of said first logic variable segments by said period to obtain certain ones of said speech waveform characteristic ratios.

23. The system of claim 22 wherein said speech waveform characteristic ratio computing means further includes:

(a) means for producing a second sequence of second logic variable segments each defined by a respective one of said time intervals and a logic state determined by occurrence of a predetermined number of consecutive slope reversals of said ramping signal;

(b) second means for storing said second sequence of second logic variable segments; and

(c) means for dividing predetermined ones of said second logic variable segments by said period to obtain certain other ones of said waveform characteristic ratios.

24. The system of claim 18 wherein said comprising means includes means for classifying phonemes, said phoneme classifying means selecting various ones of said stored phoneme ratios for comparison with said speech waveform characteristic ratios according to a phoneme classifier decision tree, said phoneme classifier decision tree including a plurality of branches each representing a plurality of phoneme ratios of related phoneme sounds, the phoneme ratios of each branch being selected in decreasing order of their average frequency of occurrence in a spoken language.

25. The system of claim 24 further including:

(a) means for producing an amplitude envelope signal representing amplitude variations of said voice signals;

(b) means for serially encoding said amplitude envelope signal to produce a second sequence of serial digital pulses representing amplitude information of said amplitude envelope signals;

(c) means in said comparing means for analyzing said second sequence of serial digital pulses to produce transition information representing transitions and transitional glides of said voice signals.

26. The system of claim 24 further including:

(a) means for detecting frequency components of said voice signals in a predetermined frequency range and exceeding a predetermined threshold magnitude;

(b) means for producing a digital signal representing said detected frequency component; and

(c) means in said comparing means responsive to said frequency component representing signals for selecting one of said branches and distinguishing between predetermined ones of said stored phoneme ratios and said speech waveform characteristic ratios.

27. The system of claim 24 further including means for printing a phoneme symbol in response to said phoneme representing signal.

28. The method of claim 27 wherein said comparing means further includes:

(a) means for sequentially comparing said selected ones of said stored phoneme ratios with said speech waveform characteristic ratios until an adequate matching occurs;

(b) means for comparing said matched phoneme ratios with a most recently matched previous phoneme ratio, if an adequate matching results from any of said sequential comparing steps, and repeating said sequential comparing steps if no adequate matching results; and

(c) means for producing said phoneme sound representing signal to cause printing of said phoneme symbol, if said matched phoneme ratio is the same as said previous phoneme ratio, and repeating said sequential comparing steps if said matched phoneme ratio is not the same as said previous phoneme ratio, said printed phoneme symbol corresponding to said previous phoneme ratio.

31. The system of claim 27 incorporated into a telephone system, wherein said analog speech signal producing means includes a receiving unit of a telephone headset and an acoustical coupling device responsive to sound waves produced by said receiving unit for producing said analog speech signals and conducting them to said delta modulating means.

32. The system of claim 31 further including switching means for electrically coupling signals produced in response to a microphone unit to the input of said delta modulating means and decoupling said receiving unit from said delta modulating means to permit a person to use said system as a dictacting machine, speech training machine, or to use said system to practice reading phoneme symbols produced in response to speech.

33. The system of claim 27 incorporated in a dictating machine.

34. The system of claim 33 wherein said dictating machine is portable, and incorporates a thermal phoneme symbol printing head.

29. A system for producing a signal representing a phoneme in response to a sequence of analog speech signals, said system comprising in combination:

(a) means for serially encoding said sequence of analog speech signals to produce a corresponding sequence of serial digital pulses corresponding to ramp slope reversals of a delta modulator circuit;

(b) means for detecting major slope transitions of said analog speech signals by detecting presence and absence of predetermined numbers of successive slope reversals corresponding to said sequence of digital pulses;

(c) means for computing a speech waveform characteristic ratio of time intervals between certain ones of said slope transitions;

(d) means for comparing said speech waveform characteristic ratio with a stored phoneme ratio to determine if said speech waveform characteristic ratio matches said stored phoneme ratio; and

(e) means responsive to said comparing means for producing a signal representing a phoneme corresponding to said phoneme ratio if said matching occurs.

30. A method for producing a signal representing a phoneme in response to a sequence of speech signals, such method comprising the steps of:

(a) encoding said sequence of speech signals to produce a sequence of digital pulses representing phonemic information continued in said analog speech signals, said encoding including

i. comparing a positively and negatively ramping signal to said sequence of substantially repetitive speech signals;

ii. periodically comparing the instantaneous level of said analog speech signals at a predetermined rate with the instantaneous level of said ramping signal;

iii. reversing the slope of said ramping signal if said ramping signal is positive-going and exceeds said instantaneous level, and reversing the slope of said ramping signal if said ramping signal is negative going and is less than said level;

iv. producing said digital pulses in accordance with said slope reversing;

(b) detecting major slope transitions of said speech wave form in response to said sequence of digital pulses by detecting presence and absence of successive slope reversals;

(c) measuring time intervals between predetermined ones of said detected major slope transitions;

(d) computing a plurality of speech waveform characteristic ratios between predetermined ones of said measured time intervals;

(e) comparing said speech waveform characteristic ratios with a plurality of stored groups of phoneme characteristic ratios in a predetermined order to determine if said speech waveform characteristic ratios match any of said stored phoneme ratios; and

(f) producing a signal representing the phoneme sound corresponding to a matching one of said phoneme ratios.