Method of recognizing pre-recorded announcements

• US 5,825,855 A
• Filed: 01/30/1997
• Issued: 10/20/1998
• Est. Priority Date: 01/30/1997
• Status: Expired due to Term

• Current Assignee: Toshiba America Information Systems Incorporated (Toshiba Corporation)
• Original Assignee: Toshiba America Information Systems Incorporated (Toshiba Corporation)
• Inventors: Astarabadi, Shaun
• Primary Examiner(s): Tsang, Fan S.
• Assistant Examiner(s): Hoosain, Allan
• Application Number:
  US08/790,200
• Time in Patent Office: 628 Days
• Field of Search: 379/67, 379/88, 379/89, 379/67.1, 379/88.01, 379/88.02, 379/88.08, 379/88.09, 379/88.22, 379/88.23, 382/30
• US Class Current: 379/67.1
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence
• CPC Class Codes: H04M 1/645 with speech synthesis
• CPC Class Codes: H04M 1/6505 storing speech in digital form
• CPC Class Codes: H04M 2201/40 using speech recognition
• CPC Class Codes: H04M 3/533 Voice mail systems

Abstract

A method of recognizing correspondence between a pre-recorded sample announcement and the identical pre-recorded reference announcement is shown. The reference pre-recorded sample announcement is characterized in terms of: (1) Start of speech (SP) where the transition from silence to voicing occurs, (2) average frame power (AFP) for each power frame length of a voicing segment of speech (3) silence gap length (SGL), the duration of non-voicing between two consecutive voicing segments and (4) end of speech (EOS). The sample pre-recorded announcement is also characterized in terms of SP, AFP, SGL, and EOS. The SP, AFPs, SGLs, and EOS of the reference pre-recorded announcement is compared with that of the sample pre-recorded announcement to determine if a match exists.

Claims

1. A method of recognizing a specific pre-recorded announcement using silence gaps in an end user'"'"'s computer, the method comprising the steps of:

in a learning phase using the computer to do the following steps;

(a) accepting as input for training a signal representing a reference pre-recorded announcement having at least one silence period to be received;

(b) determining characteristics of a starting point (SP) for a transition from silence to speech in the reference pre-recorded announcement;

(c) determining characteristics of a first voicing segment of the reference pre-recorded announcement;

(d) determining a silence gap length (SGL) for a period of silence following the first voicing segment of the reference pre-recorded announcement;

(e) determining characteristics of a second voicing segment of the reference pre-recorded announcement;

(f) characterizing an end of speech (EOS) of the reference pre-recorded announcement;

(g) storing the characteristics of the SP for the transition from silence to speech in the reference pre-recorded announcement;

(h) storing the characteristics of the first voicing segment of the reference pre-recorded announcement;

(i) storing the SGL for the period of silence following the first voicing segment of the reference pre-recorded announcement;

(j) storing the characteristics of the second voicing segment of the reference pre-recorded announcement;

(k) storing the characteristics of the FOS of the reference pre-recorded announcement;

in a recognition phase using the computer to do the following steps;

(a) accepting as input from the second device for recognition a signal representing a sample prerecorded announcement;

(b) recognizing the SP for the transition from silence to speech in the sample pre-recorded announcement;

(c) determining characteristics of a first voicing segment of the sample pre-recorded announcement;

(d) comparing the characteristics of the first voicing segment of the sample pre-recorded announcement with the stored characteristics of the first voicing segment of the reference pre-recorded announcement;

(e) testing if the characteristics of the first voicing segment of the sample pre-recorded announcement match the stored characteristics of the first voicing segment of the reference pre-recorded announcement, and if so then;

comparing the characteristics of the SGL for the period of silence following the first voicing segment of the sample pre-recorded announcement with the stored characteristics of the SGL for the period of silence following first voicing segment of the reference pre-recorded announcement;

(f) testing if the characteristics of the SGL for the period of silence following the first voicing segment of the sample prerecorded announcement match the stored characteristics of the SGL for the period of silence following first voicing segment of the reference pre-recorded announcement, and if so then comparing the characteristics of the second voicing segment of the sample pre-recorded announcement from with the stored characteristics of the second voicing segment of the reference pre-recorded announcement;

(g) testing if the characteristics of the second voicing segment of the sample pre-recorded announcement match the stored characteristics of the second voicing segment of the reference pre-recorded announcement, and if so then recognizing the EOS of the sample announcement; and

(h) recognizing a match between the sample pre-recorded announcement and the reference pre-recorded announcement.

2. The method of claim 1 comprising the additional step of determining a level of confidence when there is an incomplete match between the reference pre-recorded announcement and the sample pre-recorded announcement.

3. The method of claim 1 comprising the additional steps of:

normalizing the signal representing the reference pre-recorded announcement; and

normalizing the signal representing the sample pre-recorded announcement.

4. The method of claim 1 where the step of recognizing a match between the sample pre-recorded announcement and the reference pre-recorded announcement comprises the steps of:

recognizing transitions between voicing segments and silence and between silence and voicing segments;

at each transition evaluating a calculated signal derived from the sample pre-recorded announcement from the second device with an expected, stored signal derived from the reference pre-recorded announcement;

advancing a state machine if a match between the calculated signal derived from the sample pre-recorded announcement with an expected, stored signal derived from the reference pre-recorded announcement is obtained; and

resetting the state machine if a match between the calculated signal derived from the sample pre-recorded announcement with an expected, stored signal derived from the reference pre-recorded announcement is not obtained.

5. The method of claim 4 further comprising the step of the first device determining that there is a match between the sample pre-recorded announcement and the reference pre-recorded announcement when the EOS of the sample pre-recorded announcement matches the EOS of the reference pre-recorded announcement.

6. The method of claim 4 wherein the computer obtains characteristics of a number of different pre-recorded reference announcements and stores the obtained characteristics in memory, and the computer launching state machines for each of the pre-recorded reference announcements upon initiation of the acceptance of input from a pre-recorded sample announcement.

7. The method of claim 1 where the step of recognizing a match between the sample pre-recorded announcement and the reference pre-recorded announcement is accomplished by cross correlation.

8. The method of claim 1 further comprising the steps of:

determining in at least one test case if proper recognition of the sample pre-recorded announcement is made;

testing if proper recognition of the sample announcement is not made and if so then;

extracting further details of the voicing segments to uniquely identify the sample pre-recorded announcement.

9. The method of claim 8 wherein extracting further details of the voicing segments consists of subdividing each voicing segment into equal length frames.

10. The method of claim 8 wherein extracting further details of the voicing segments consists of subdividing each voicing segment into fixed length frames.

11. The method of claim 1 wherein each voicing segment comprises at least one power frame length, each power frame length having an average frame power (AFP) associated therewith.

12. A method of recognizing a specific phrase generated within a pre-recorded announcement using silence gaps in an end user'"'"'s computer comprising the steps of:

in a learning phase using the computer to do the following steps;

(a) accepting as input a reference phrase having at least one silence period forming a portion of a pre-recorded announcement to be received;

(b) determining a point where speech representing a beginning of the reference phrase occurs;

(c) calculating mean energy between two points for voicing portions of speech in the reference phrase;

(d) calculating a silence gap length within the reference phrase;

(e) determining an end of the reference phrase;

(f) storing as parameters for the reference phrase the point where speech representing a beginning of the reference phrase occurs, the mean energy between two points for voicing portions of speech, the silence gap length, and the end of the reference phrase;

in a recognition phase using the computer to do the following steps;

(a) accepting as input a sample phrase having silence periods forming a portion of a sample pre-recorded announcement;

(b) determining a point where speech representing a beginning of the sample phrase occurs;

(c) calculating mean energy between two points for voicing portions of speech in the sample phrase;

(d) calculating a silence gap length within the sample phrase;

(e) determining an end of the sample phrase; and

(f) comparing the stored parameter values for the reference phrase with the point where speech representing a beginning of the sample phrase occurs, the mean energy between two points for voicing portions of speech for the sample phrase, the silence gap length in the sample phrase, and the end of the sample phrase.

13. The method of claim 12 comprising the additional step of determining a level of confidence when there is an incomplete match between the sample phrase and the reference phrase.

14. The method of claim 12 comprising the additional steps of:

normalizing the signal representing the reference phrase; and

normalizing the signal representing the sample phrase.

15. The method of claim 12 further comprising the steps of:

determining in at least one test case if proper recognition of the sample phrase is made;

testing if proper recognition of the sample phrase is not made and if so then;

extracting further details of the voicing segments to uniquely identify the sample phrase.

16. The method of claim 15 wherein extracting further details of the voicing segments consists of subdividing each voicing segment into equal length frames.

17. The method of claim 15 wherein extracting further details of the voicing segments consists of subdividing each voicing segment into fixed length frames.

18. A method of a first device responding in an appropriate pre-determined fashion to a specific pre-recorded announcement from a second device using silence gaps in an end user'"'"'s computer, the method comprising the steps of:

in a learning phase using the computer to do the following steps;

(a) accepting as input a reference pre-recorded announcement having at least one silence period;

(b) obtaining time domain characteristics of voicing segments of the reference pre-recorded announcement;

(c) storing in memory the obtained characteristics of voicing segments of the reference pre-recorded announcement obtained;

(d) obtaining time domain characteristics of silence segments of the reference pre-recorded announcement;

(e) storing in memory the obtained characteristics of silence segments of the reference pre-recorded announcement;

in a recognition phase using the computer to do the following steps;

(a) accepting as input a sample pre-recorded announcement;

(b) obtaining time domain characteristics of voicing segments of the sample pre-recorded announcement;

(c) comparing the time domain characteristics of the voicing segments of the sample pre-recorded announcement with the time domain characteristics of voicing segments of the reference pre-recorded announcement;

(d) obtaining time domain characteristics of silence segments of the sample pre-recorded announcement;

(e) comparing the time domain characteristics of the silence segments of the sample pre-recorded announcement with the time domain characteristics of silence segments of the reference prerecorded announcement;

(f) determining from the comparisons if the sample pre-recorded announcement is a match with the reference pre-recorded announcement; and

(g) testing if the sample pre-recorded announcement matches the reference pre-recorded announcement and if so then responding in the predetermined fashion.

19. The method of claim 18 where the step of recognizing a match between the sample pre-recorded announcement and the reference pre-recorded announcement comprises the steps of:

recognizing transitions between voicing segments and silence and between silence and voicing segments;

at each transition evaluating a calculated signal derived from the sample pre-recorded announcement with an expected, stored signal derived from the reference pre-recorded announcement;

advancing a state machine if a match between the calculated signal derived from the sample pre-recorded announcement and an expected, stored signal derived from the reference pre-recorded announcement is obtained; and

resetting the state machine if a match between the calculated signal derived from the sample pre-recorded announcement with an expected, stored signal derived from the reference pre-recorded announcement is not obtained.

20. The method of claim 18 further comprising the step of determining that there is a match between the sample pre-recorded announcement and the reference pre-recorded announcement when the EOS of the sample pre-recorded announcement matches the EOS of the reference pre-recorded announcement.

21. The method of claim 20 wherein characteristics of a number of different pre-recorded reference announcements are all obtained and stored in memory and state machines for each of the pre-recorded reference announcements are all launched upon initiation of the acceptance of input from a pre-recorded announcement.

22. The method of claim 18 where the step of recognizing a match between the sample pre-recorded announcement and the reference pre-recorded announcement is accomplished by cross correlation.

23. The method of claim 18 comprising the additional step of determining a level of confidence when there is an incomplete match between the sample pre-recorded announcement and the reference pre-recorded announcement.

24. The method of claim 18 comprising the additional steps of:

normalizing the signal representing the reference pre-recorded announcement; and

normalizing the signal representing the sample pre-recorded announcement.

25. The method of claim 18 further comprising the steps of:

determining that excess detail is being used to discriminate between different pre-recorded announcements; and

adjusting calculated parameters so as to decrease detail levels and thus usage of storage and calculation resources.

26. The method of claim 25 wherein various characteristics are stored as parameters and usage of storage and calculation resources is decreased by normalizing the parameters.

27. The method of claim 25 wherein usage of storage and calculation resources is decreased by increasing the number of samples in an average power value for a voicing segment of speech calculated for each power frame length segment of the signal so that fewer average frame power values (AFPs) are calculated and stored.

28. A method comparing a sample pre-recorded voice announcement having at least one silence period with a reference pre-recorded voice announcement having at least one silence period in an end user'"'"'s computer, the method comprising the steps of;

in a learning phase using the computer to do the following steps;

(1) determining a key phrase within the reference pre-recorded verbal announcement;

(2) accepting as input for training a signal representative of the key phrase within the reference pre-recorded announcement;

(3) determining characteristics of a starting point (SP) for a transition from silence to speech representing initiation of the key phrase in the reference pre-recorded announcement;

(4) determining characteristics of a first voicing segment of the key phrase of the reference pre-recorded announcement;

(5) determining a first silence gap length (SGL) for a period of silence following the first voicing segment of the key phrase of the reference pre-recorded announcement;

(6) determining characteristics of a next voicing segment of the key phrase of the reference pre-recorded announcement;

(7) repeating steps 5 and 6 of the learning phase until an end of the key phrase is noted;

(8) characterizing an end of speech (EOS) of the key phrase of the reference pre-recorded announcement;

(9) storing the characteristics of the SP for the transition from silence to speech in the key phrase of the reference pre-recorded announcement;

(10) storing the characteristics of the first voicing segment of the key phrase of the reference pre-recorded announcement;

(11) storing the SGL for the period of silence following the first voicing segment of the key phrase of the reference pre-recorded announcement;

(12) storing the characteristics of all succeeding voicing segments and SGLs of the key phrase of the reference pre-recorded announcement;

(13) storing the characteristics of the EOS of the key phrase of the reference pre-recorded announcement;

in a recognition phase using the computer to do the following steps;

(1) accepting as input for recognition a signal representing a sample pre-recorded announcement;

(2) recognizing the SP for the transition from silence to speech in the sample pre-recorded announcement;

(3) determining characteristics of a first voicing segment of the sample pre-recorded announcement;

(4) comparing the characteristics of the first voicing segment of the sample pre-recorded announcement with the stored characteristics of the first voicing segment of the key phrase of the reference pre-recorded announcement;

(5) testing if the characteristics of the first voicing segment of the sample pre-recorded announcement do not match the stored characteristics of the first voicing segment of the key phrase of the reference pre-recorded announcement, and if so then;

(6) repeating steps 2 through 5 of the recognition phase designating successive voicing segments as the first voicing segment of the sample pre-recorded announcement until the characteristics of the first voicing segment of the sample pre-recorded announcement matches the stored characteristics of the first voicing segment of the key phrase of the reference pre-recorded announcement indicating that the key phrase of the sample pre-recorded announcement has been reached;

(6) comparing the characteristics of the SGL for the period of silence following the first voicing segment of the key phrase of the sample pre-recorded announcement with the stored characteristics of the SGL for the period of silence following the first voicing segment of the key phrase of the reference pre-recorded announcement;

(7) testing if the characteristics of the SGL for the period of silence following the first voicing segment of the sample pre-recorded announcement match the stored characteristics of the SGL for the period of silence following first voicing segment of the reference pre-recorded announcement, and of so then;

(8) for all successive voicing segments and SGLs of the key phrase of the sample pre-recorded announcement, comparing the characteristics of each voicing segment and SGL of the sample pre-recorded announcement with respective stored characteristics of voicing segments and SGLs of the key phrase of the reference pre-recorded announcement and if characteristics of each voicing segment and SGL of the sample pre-recorded announcement matches the respective stored characteristics of voicing segments and SGLs of the key phrase of the reference pre-recorded announcement;

then

(9) recognizing the EOS of the key phrase of the sample pre-recorded announcement; and

(10) recognizing a match between the sample pre-recorded announcement and the reference pre-recorded announcement.

29. The method of claim 28 wherein steps 1 through 8 of the learning phase are performed at least twice, each time using a different voicing threshold point.

30. The method of claim 28 wherein voicing segments are broken down into power frame length segments, each having an average frame powers (AFP) and relative percent measures are used for neighboring AFP values.

31. A computer device located at an end user location, the computer device for responding in an appropriate pre-determined fashion to a pre-recorded sample announcement, the computer device comprising:

at the end user location in said computer device;

a processor;

a memory coupled to the processor for storing a program and data therein;

a data input device coupled to the processor;

an audio input subsystem coupled to the processor for accepting input speech signals; and

a stored program resident in the memory, the program for performing the steps of;

accepting as input a pre-recorded reference announcement;

obtaining time domain characteristics of voicing segments of the reference announcement;

storing in memory the obtained time domain characteristics of voicing segments of the reference announcement;

obtaining time domain characteristics of silence segments of the reference announcement;

storing in memory the obtained time domain characteristics of silence segments of the reference announcement;

accepting as input the pre-recorded sample announcement;

obtaining time domain characteristics of voicing segments of the sample announcement;

comparing the time domain characteristics of the voicing segments of the sample announcement from the pre-recorded announcement playing device with the time domain characteristics of voicing segments of the reference announcement;

obtaining time domain characteristics of silence segments of the sample announcement;

comparing the time domain characteristics of the silence segments of the sample announcement with the characteristics of silence segments of the reference announcement;

determining from the comparisons if the sample announcement is a match with the reference announcement; and

if the sample announcement matches the reference announcement then responding in a pre-determined fashion.

32. A computer device located at an end user location the computer device for recognizing a sample phrase generated within a sample pre-recorded announcement, the computer device comprising:

at the end user location in said computer device;

a processor;

a memory coupled to the processor for storing a program and data therein;

a data input device coupled to the processor;

an audio input subsystem coupled to the processor for accepting input speech signals; and

a stored program resident in the memory, the program for performing the steps of;

accepting as input a reference phrase forming a portion of a pre-recorded reference announcement;

determining a point where speech representing a beginning of the reference phrase occurs;

calculating mean energy between two points for voicing portions of speech in the reference phrase;

calculating a silence gap length within the reference phrase;

determining an end of the reference phrase;

storing as parameters for the reference phrase the point where speech representing a beginning of the reference phrase occurs, the mean energy between two points for voicing portions of speech, the silence gap length, and the end of the reference phrase;

accepting as input the sample phrase forming a portion of the sample pre-recorded announcement;

determining a point where speech representing a beginning of the sample phrase occurs;

calculating mean energy between two points for voicing portions of speech in the sample phrase;

calculating a silence gap length within the sample phrase;

determining an end of the sample phrase; and

comparing the stored parameter values for the reference phrase with the point where speech representing a beginning of the sample phrase occurs, the mean energy between two points for voicing portions of speech for the sample phrase, the silence gap length in the sample phrase, and the end of the sample phrase.

33. The computer system of claim 32 wherein the stored program determines a level of confidence when there is an incomplete match between the phrase in the sample pre-recorded announcement and the phrase in the reference pre-recorded announcement.

34. A program storage device readable by an end user'"'"'s machine, tangibly embodying a program of instructions executable by the machine to perform method steps for responding in an appropriate pre-determined fashion to a pre-recorded sample announcement, the method steps comprising:

in a learning subroutine using the end user'"'"'s machine to do the following steps;

accepting as input a pre-recorded reference announcement;

obtaining time domain characteristics of voicing segments of the reference announcement;

storing in memory the obtained time domain characteristics of voicing segments of the reference announcement;

obtaining time domain characteristics of silence segments of the reference announcement;

storing in memory the obtained time domain characteristics of silence segments of the reference announcement;

in a recognition subroutine using the end user'"'"'s machine to do the following steps;

accepting as input the pre-recorded sample announcement in a time domain;

obtaining time domain characteristics of voicing segments of the sample announcement from the pre-recorded announcement playing device;

comparing the time domain characteristics of the voicing segments of the sample announcement with the time domain characteristics of voicing segments of the reference announcement;

obtaining time domain characteristics of silence segments of the sample announcement;

comparing the time domain characteristics of the silence segments of the sample announcement with the time domain characteristics of silence segments of the reference announcement;

determining from the comparisons if the sample announcement is a match with the reference announcement; and

if the sample announcement matches the reference announcement then responding in a pre-determined fashion.

35. A computer program product comprising a computer usable medium having computer readable program code means embodied therein comprising computer readable program code means for causing an end user'"'"'s computer to recognize a sample phrase generated within a sample pre-recorded announcement the recognition being performed by steps comprising:

in a learning subroutine using the end user'"'"'s computer to do the following steps;

accepting as input a reference phrase forming a portion of a pre-recorded reference announcement;

determining a point where speech representing a beginning of the reference phrase occurs;

calculating mean energy between two points for voicing portions of speech in the reference phrase;

calculating a silence gap length within the reference phrase;

determining an end of the reference phrase;

storing as parameters for the reference phrase the point where speech representing a beginning of the reference phrase occurs, the mean energy between two points for voicing portions of speech, the silence gap length, and the end of the reference phrase;

in a recognition subroutine using the end user'"'"'s computer to do the following steps;

accepting as input the sample phrase forming a portion of the pre-recorded announcement;

determining a point where speech representing a beginning of the sample phrase occurs;

calculating mean energy between two points for voicing portions of speech in the sample phrase;

calculating a silence gap length within the sample phrase;

determining an end of the sample phrase; and

comparing the stored parameter values for the reference phrase with the point where speech representing a beginning of the sample phrase occurs, the mean energy between two points for voicing portions of speech for the sample phrase, the silence gap length in the sample phrase, and the end of the sample phrase.

36. A method of training a computer to recognize a pre-recorded announcement from a pre-recorded announcement playing device comprising using the computer to do the following the steps of:

accepting as input a phrase from the pre-recorded announcement playing device forming a portion of the pre-recorded announcement;

determining a point where speech representing a beginning of the phrase from the pre-recorded announcement playing device occurs;

calculating mean energy between two points for voicing portions of speech in the phrase from the pre-recorded announcement playing device;

calculating a silence gap length within the phrase from the pre-recorded announcement playing device;

determining an end of the phrase from the pre-recorded announcement playing device; and

storing as parameters for the phrase from the pre-recorded announcement playing device the point where speech representing a beginning of the phrase occurs, the mean energy between two points for voicing portions of speech, the silence gap length, and the end of the phrase from the pre-recorded announcement playing device.

37. A method of comparing a sample pre-recorded announcement with a previously analyzed reference pre-recorded announcement wherein parameters representing a key phrase of the reference pre-recorded announcement have been stored using an end user'"'"'s computer, comprising the steps of:

accepting as input a sample phrase forming a portion of the sample pre-recorded announcement;

determining a point where speech representing a beginning of the sample phrase;

calculating mean energy between two points for voicing portions of speech in the sample phrase;

calculating a silence gap length within the sample phrase;

determining an end of the sample phrase; and

comparing the stored parameter values for the reference phrase with the point where speech representing a beginning of the sample phrase occurs, the mean energy between two points for voicing portions of speech for the sample phrase, the silence gap length in the sample phrase, and the end of the sample phrase.