Phonetic searching

• US 7,324,939 B1
• Filed: 01/27/2006
• Issued: 01/29/2008
• Est. Priority Date: 03/04/2000
• Status: Expired due to Term

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Cardillo, Peter S., Price, William E., Clements, Mark A.
• Primary Examiner(s): Harper; V. Paul
• Application Number:
  US11/341,311
• Time in Patent Office: 732 Days
• Field of Search: None
• US Class Current: 704/236
• CPC Class Codes: G06F 16/3343 using phonetics
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

An improved method and apparatus is disclosed which uses probabilistic techniques to map an input search string with a prestored audio file, and recognize certain portions of a search string phonetically. An improved interface is disclosed which permits users to input search strings, linguistics, phonetics, or a combination of both, and also allows logic functions to be specified by indicating how far separated specific phonemes are in time.

Claims

1. A method for finding a desired audio event in an audio signal comprising:

accepting audio data representing a recording of the audio signal in a recording interval having a beginning and an end, wherein a first end time comprises a time of one of the beginning and the end of the recording interval and a second end time comprises a time of the other of the beginning and the end;

preprocessing the audio data, including

computing preprocessing data associated with each of a series of times in the recording interval, including computing preprocessing data that include scores associated with each of the times based on the audio signal between said each of the series of times and the first end time of the recording interval, and

storing said preprocessing data;

determining a representation of the desired audio event; and

searching the audio signal for the desired audio event, including

computing scores for each of the series of times in the recording interval based on the representation of the desired audio event and associated with the audio signal between said each of the series of times and the second end time of the recording interval,

retrieving the stored preprocessing data, and

combining the scores based on the representation of the desired audio event and scores from the retrieved preprocessing data to form scores for the desired audio event each associated with a different time in the recording interval.

2. The method of claim 1 further comprising finding occurrences of the desired audio event based on the scores for said events.

3. The method of claim 1 wherein finding occurrences of the desired audio event based on the scores for said events includes ranking times in the recording interval according to the scores for the desired audio event at said times.

4. The method of claim 1 wherein computing the preprocessing data is performed independently of the desired audio event.

5. The method of claim 4 wherein computing the preprocessing data is performed prior to determining the representation of the desired audio event.

6. The method of claim 1 wherein computing the preprocessing data includes computing scores associated with each of a series of equally spaced times.

7. The method of claim 6 wherein the equally spaced times are spaced at intervals of 10 milliseconds.

8. The method of claim 1 wherein the first end time is earlier than the second end time in the recording interval.

9. The method of claim 1 wherein computing the preprocessing data includes computing probabilities of model states associated with each of said times.

10. The method of claim 1 wherein computing preprocessing data associated with each of the series of times in the recording interval further includes computing preprocessing data that includes scores associated with each of the series of times based on the audio signal between said time and the second end time of the recording interval.

11. The method of claim 10 wherein computing preprocessing data that includes scores associated with each of the series of times based on the audio signal between said time and the second end time of the recording interval includes computing a probability associated with each time.

12. The method of claim 1 wherein computing the preprocessing data further includes computing preprocessing data associated with each time and associated with each of a plurality of model states based on the audio signal in the vicinity of said time.

13. The method of claim 12 wherein computing preprocessing data associated with each time and associated with each of the plurality of model states includes computing a quantity associated with a likelihood of observing the audio signal in the vicinity of said time in each of the model states.

14. The method of claim 1 wherein preprocessing the audio data includes quantizing the computed data before storing said data.

15. The method of claim 1 wherein the computing the preprocessing data includes processing the audio data using a hidden Markov model.

16. The method of claim 15 wherein the hidden Markov model includes phonetic models.

17. The method of claim 15 wherein computing the preprocessing data associated with a time based on the audio signal between said time and the first end time of the recording interval includes computing a probability of observing the audio signal between said time and the first end time according to the hidden Markov model.

18. The method of claim 17 wherein computing a probability of observing the audio signal between said time and the first end time according to the hidden Markov model includes computing a probability of observing said audio signal and occupying a state of the model that is associated with an endpoint of the desired audio event.

19. The method of claim 17 wherein the computing the preprocessing data includes applying a recursive computation for successive of the times of the series of times in the recording interval.

20. The method of claim 1 wherein determining a representation of the desired audio event includes determining a phonetic representation of the desired audio event.

21. The method of claim 20 wherein determining a representation of the desired audio event includes accepting a text representation.

22. The method of claim 21 wherein accepting the text representation includes accepting one or more words.

23. The method of claim 22 wherein accepting words includes accepting a phrase.

24. The method of claim 21 wherein accepting the text representation includes accepting text representation of a pronunciation.

25. The method of claim 21 wherein accepting the text representation of a pronunciation includes accepting a representation in terms of phonetic units.

26. The method of claim 20 wherein determining a representation of the desired audio event includes accepting an audio signal.

27. The method of claim 26 wherein accepting the audio signal includes accepting data encoding a spoken word.

28. The method of claim 1 wherein determining a representation of the desired audio event includes determining a representation of multiple components of the desired audio event, and characteristics of a desired temporal relationship of the components.

29. The method of claim 28 wherein determining a representation of multiple components includes determining a phonetic representation of each of the components.

30. The method of claim 28 wherein determining characteristics of a desired temporal relationship of the components includes determining a maximum temporal separation of said components.

31. A method for finding a desired audio event in an audio signal comprising:

preprocessing the audio signal independently of the desired audio event, including

computing preprocessing data associated with each of a series of times in a recording interval, including computing preprocessing data that include scores associated with each of the series of times and associated with different model states, and

storing said preprocessing data; and

accepting a representation of the desired audio event;

searching the audio signal for the desired audio event, including

retrieving the scores from the stored preprocessing data,

computing scores using the accepted representation of the desired event, and

computing scores for occurrences at corresponding times of the desired audio event by combining the retrieved scores and scores computed using the accepted representation of the desired event.

32. The method of claim 31 wherein computing preprocessing data that include scores associated with each of the times and associated with different model states includes computing scores associated with phonetic models.

33. The method of claim 31 wherein computing scores for occurrences of the desired audio event includes evaluating an expression for a quantity characterizing a probability of occurrence of the desired audio event, wherein the expression includes a first term determined from the scores of the preprocessing data and second term determined using the representation of the desired audio event.

34. The method of claim 31 wherein the computing of the preprocessing data requires substantially more computation than the computing of the scores using the accepted representation of the desired event.