Phonetic searching

• US 7,263,484 B1
• Filed: 03/05/2001
• Issued: 08/28/2007
• Est. Priority Date: 03/04/2000
• Status: Expired due to Term

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Cardillo, Peter S., Price, William E., Clements, Mark A.
• Primary Examiner(s): Harper; V. Paul
• Application Number:
  US09/799,385
• Time in Patent Office: 2,367 Days
• Field of Search: 704/231, 704/242, 704/251, 704/252, 704/255, 704/9, 707/2, 707/3, 707/5
• US Class Current: 704/236
• CPC Class Codes: G06F 16/3343 using phonetics
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

An improved method and apparatus is disclosed which uses probabilistic techniques to map an input search string with a prestored audio file, and recognize certain portions of a search string phonetically. An improved interface is disclosed which permits users to input search strings, linguistics, phonetics, or a combination of both, and also allows logic functions to be specified by indicating how far separated specific phonemes are in time.

Claims

1. A method of searching an audio source for occurrence of a search string, the method comprising:

processing the audio source according to a plurality of audio states independently of the search string, including

forming a plurality of time frames from the audio source, and

storing, for each of the time frames, quantities characterizing probabilities associated with the occurrence of each of the plurality of audio states within said each time frame;

accepting the search string;

using the search string and the stored quantities characterizing the probabilities, determining for each of a series of the time frames a quantity characterizing a probability that the search string ended or began at said time frame.

2. The method of claim 1 comprising the step of first modeling the audio source by utilizing a Gaussian model and plural Mel Frequency Cepstral Coefficients (MFCCs).

3. The method of claim 1 wherein said audio states comprise R plus M time N states, where M is a number of states per phoneme, N is a number of different phonemes, and R is a number of bridge states.

4. The method of claim 1 wherein the storing includes quantization of the quantities characterizing the probabilities.

5. The method of claim 4 wherein the quantization includes using four bit logarithmic linear quantization of the quantities.

6. The method of claim 1 wherein forward and backward probabilities associated with a bridge state are quantized and stored with different format than probabilities associated with other of the audio states.

7. The method of claim 1 wherein said time frames are overlapping.

8. The method of claim 1 wherein:

processing the audio source includes constructing a matrix having a time frame dimension and a state dimension, each entry representing a quantity characterizing a probability of an audio state occurring in a time frame, ascertaining audio states contained within the search string, and selecting entries in said matrix that correspond to audio states that occur in said search string; and

the method includes processing said search string against said selected entries in said matrix corresponding to said occurring states.

9. The method of claim 8 wherein said processing comprises ascertaining which time frames have a maximum probability of a predetermined search string state occurring therewithin, and further comprising sorting and displaying said time frames.

10. The method of claim 1 wherein the audio source comprises a file.

11. The method of claim 1 wherein determining for each of a series of the time frames a quantity characterizing the probability includes determining said quantities of each of the time frames.

12. The method of claim 11 wherein the time fames are of substantially equal duration.

13. The method of claim 1 wherein determining the quantity characterizing the probability that the specified search string ended or began at said time frame includes computing a quantity associated with a final or a beginning audio state of the search string.

14. The method of claim 1 further comprising:

generating a display that indicates, in sorted order, where in the file the specified search string is most likely to occur.

15. The method of claim 1 wherein:

processing the audio source includes constructing a data structure arranged according to a time frame dimension and a state dimension, each entry in the data structure representing a quantity characterizing a probability of an audio state occurring in a time frame, ascertaining audio states contained within the search string, and selecting portions of the data structure correspond to audio states that occur in said search string; and

the method includes processing said search string against said selected portions of the data structure corresponding to said occurring states.

16. The method of claim 1 further comprising:

accepting a further search string;

using the further search string and the stored quantities characterizing the probabilities, determining for each of a series of a series of the time frames a quantity characterizing the probability that the further search string ended or began at said time frame.