Vocabulary-independent search of spontaneous speech

• US 7,584,098 B2
• Filed: 11/29/2004
• Issued: 09/01/2009
• Est. Priority Date: 11/29/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Seide, Frank Torsten, Yu, Roger Peng
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): YEN, ERIC L
• Application Number:
  US10/998,451
• Publication Number:
  US 20060116997A1
• Time in Patent Office: 1,737 Days
• Field of Search: 704/259, 704/215, 704/231, 704/232, 704/233, 704/240, 704/242
• US Class Current: 704/240
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection

Abstract

A method of identifying a location of a query string in an audio signal is provided. Under the method, a segment of the audio signal is selected. A score for a query string in the segment of the audio signal is determined by determining the product of probabilities of overlapping sequences of tokens. The score is then used to decide if the segment of the audio signal is likely to contain the query string.

Claims

1. A method of identifying a location of a query string in an audio signal, the method comprising:

receiving a query string;

selecting a segment of the audio signal from a plurality of segments of the audio signal;

determining a probability of the query string given the segment of the audio signal by determining the product of probabilities of overlapping sequences of tokens, the probabilities of overlapping sequence of tokens formed through steps comprising;

applying the audio speech signal to a speech recognizer that identifies a lattice of tokens from the audio speech signal and that assigns a probability to each token in the lattice based on the degree to which the audio speech signal matches an acoustic model for the token;

determining expected term frequencies for overlapping sequences of tokens in the lattice through steps comprising;

for each path through the lattice;

determining a probability of the path by multiplying the probabilities of the tokens along the path together; and

at each token along the path updating an expected term frequency for an overlapping sequence of tokens that ends at the token by adding the probability of the path to a current expected term frequency for the overlapping sequence of tokens; and

determining the probability of an overlapping sequence of tokens based on the expected term frequency for the overlapping sequence of tokens;

using the probability of the query string given the segment of the audio speech signal to identify whether the segment of the audio speech signal is likely to contain the query string.

2. The method of claim 1 further comprising selecting a plurality of segments of the audio signal, determining a probability for the query string in each segment, and using the probabilities to rank the segments.

3. The method of claim 2 further comprising selecting at least one segment that ranks higher than another segment and searching the selected segment to identify at least one possible location of the query in the selected segment.

4. The method of claim 2 further comprising dividing an audio signal into segments of substantially similar durations.

5. The method of claim 4 wherein dividing the audio signal comprises identifying points of silence in the audio signal and dividing the audio signal at the points of silence.

6. The method of claim 4 wherein the step of dividing the audio signal is performed by a speech recognizer.

7. The method of claim 1 wherein the probabilities of overlapping sequences of tokens are determined from values stored in an index for the segment of the audio signal.

8. The method of claim 7 further comprising forming the index by first identifying a set of sequences of tokens found in a chosen set.

9. The method of claim 8 further comprising forming the chosen set using a background dictionary.

10. The method of claim 8 further comprising determining a set of backoff weights for sequences of tokens that are not found in the chosen set.

11. The method of claim 8 wherein at least one probability of an overlapping sequence of tokens is determined using a backoff weight.

12. The method of claim 1 wherein the probability of at least one overlapping sequence of tokens is estimated using the probability of a shorter sequence of tokens.

14. The method of claim 1 wherein the query string comprises multiple words and determining a probability for the query string given the segment of the audio signal comprises multiplying a product of probabilities of overlapping sequences of tokens associated with a first word by a product of probabilities of overlapping sequences of tokens associated with a second word in the query string, and further by a probability for a unigram of silence.

15. The method of claim 14 further comprising determining a second probability for the query string by determining a product of overlapping sequences of tokens associated with the first and second word while assuming no silence between the first and second word.

16. The method of claim 15 further comprising determining a first expected term frequency from the probability for the query string and determining a second expected term frequency from the second probability for the query string and selecting one of the first expected term frequency and the second expected term frequency as a final expected term frequency for the query string in the segment.

17. The method of claim 15 further comprising determining a first expected term frequency from the probability for the query string and determining a second expected term frequency from the second probability for the query string and combining the first expected term frequency and the second expected term frequency to form a final expected term frequency for the query string in the segment.

18. The method of claim 17 wherein combining the first expected term frequency and the second expected term frequency comprises summing the first expected term frequency and the second expected term frequency to form the final expected term frequency.

19. The method of claim 18 wherein combining the first expected term frequency and the second expected term frequency comprises forming a weighted sum of the first expected term frequency and the second expected term frequency to form the final expected term frequency.

20. The method of claim 1 wherein the tokens comprise sub-word units.

21. The method of claim 20 wherein the tokens comprise phonemes.

22. The method of claim 1 further comprising determining an expected term frequency for the query string by determining a first expected term frequency component for overlapping sequences of sub-word units and a second expected term frequency component for overlapping sequences of words and combining the first expected term frequency component and the second expected term frequency component to form the expected term frequency component for the query string.

13. The method of 1 wherein the probability of at least one overlapping sequence of tokens is estimated using the probability of a single token.