Indexing and searching audio using text indexers

• US 8,060,494 B2
• Filed: 12/07/2007
• Issued: 11/15/2011
• Est. Priority Date: 12/07/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Thambiratnam, Albert Joseph Kishan, Seide, Frank T. B., Yu, Peng
• Primary Examiner(s): AL HASHEMI, SANA A
• Application Number:
  US11/952,817
• Publication Number:
  US 20090150337A1
• Time in Patent Office: 1,439 Days
• Field of Search: 707/706, 707/708, 707/715, 707/723, 707/728
• US Class Current: 707/715
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00

Abstract

A full-text lattice indexing and searching system and method for indexing word lattices using a text indexer to enable enhance searching of audio content. The system and method utilize a Time-Anchored Lattice Expansion (TALE) method that represents word lattices such that they can be indexed with existing text indexers with little or no modification. Embodiments of system and method include an indexing module for generating and indexing word lattices based on audio content and a searching module for allowing searching of a full-text index containing indexed word lattices. The indexing module includes a custom IFilter and a custom Wordbreaker. Embodiments of the searching module include an ExpandQuery function for decorating an input query and a custom Stemmer. Embodiments of the searching module also include a GenerateSnippets module that extracts information from the indexed word lattices to enable the creation of clickable snippets.

Claims

1. A computer-implemented method for processing audio content, comprising:

generating word lattices for the audio content, the word lattices containing a plurality of word candidates and nodes that represent connections between the word candidates and encoding times;

expanding a decorated query that was decorated with desired confidence levels into a plurality of decorated queries corresponding to the desired confidence levels; and

indexing the word lattices using a text indexer of a full-text search engine and the plurality of decorated queries such that the audio content is contained in a full-text index as indexed audio content.

4. The computer-implemented method of claim 1, further comprising binning the word lattice to organize each word lattice in columns containing the words candidates and associated confidence scores, each confidence score defined as a likelihood that the word candidate is a possible substitute for a query term.

5. The computer-implemented method of claim 1, further comprising:

inputting a query from a user containing at least one search term and a desired confidence level; and

searching the indexed audio content to obtain search results corresponding to the query and the desired confidence level.

11. The computer-implemented method of claim 4, further comprising:

quantizing a plurality of confidence scores into a discrete number of confidence scores; and

decorating a word candidate with a confidence score by appending to the word candidate a suffix corresponding to that confidence score.

12. The computer-implemented method of claim 5, further comprising decorating the query with the desired confidence level and all confidence levels better than the desired confidence level by appending to the search term a suffix corresponding to the desired confidence level and all confidence levels better than the desired confidence level to obtain a decorated query.

16. The computer-implemented method of claim 11, further comprising:

encoding each word lattice column as a single linear text string by concatenating each word in the column with a separation character; and

outputting the single linear text string from a text ingestion plug-in of the full-text search engine.

17. The computer-implemented method of claim 12, further comprising:

inputting the decorated query into a Wordbreaker of the full-text search engine; and

expanding the decorated query into a plurality of decorated queries corresponding whose number is equal to a sum of the desired confidence level plus a number of all confidence levels better than the desired confidence level.

18. The computer-implemented method of claim 16, further comprising:

inputting the single linear text string to a Wordbreaker of the full-text search engine;

decoding the single linear text string to recover the word lattice columns and associated confidence scores; and

storing the word lattice columns and associated confidence scores in the full-text index.

19. The computer-implemented method of claim 17, further comprising:

inputting the plurality of decorated queries into a custom Stemmer;

stripping off the decorations of each of the plurality of decorated queries by removing each suffix to obtain stripped queries; and

storing each suffix associated with each of the plurality of decorated queries for future appending.

20. The computer-implemented method of claim 19, further comprising:

inputting the stripped queries into an original Stemmer of the full-text search engine;

obtaining from the original Stemmer variations of each of the stripped queries to obtain query variations;

inputting the stripped queries and the query variations into the custom Stemmer;

appending the removed suffixes to their proper stripped queries and the query variations to obtain an expanded set of decorated queries; and

searching the full-text index using the expanded set of decorated queries.

2. A computer-implemented method for processing audio content, comprising:

generating word lattices for the audio content, the word lattices containing a plurality of word candidates and nodes that are connections between the word candidates;

inputting a query from a user containing at least one search term and a desired confidence level; and

decorating the query with the desired confidence level and all confidence levels better than the desired confidence level to obtain a decorated query;

expanding the decorated query into a plurality of decorated queries based on the desired confidence level; and

obtaining search results corresponding to the decorated query and the desired confidence level and displaying the search results to the user.

6. The computer-implemented method of claim 2, further comprising:

indexing the word lattices such that the audio content is contained in a full-text index as indexed audio content; and

searching the indexed audio content to obtain the search results.

7. The computer-implemented method of claim 2, further comprising appending to the at least one search term a suffix corresponding to the desired confidence level and all confidence levels better than the desired confidence level.

8. The computer-implemented method of claim 2, further comprising:

inputting the plurality of decorated queries into a custom Stemmer;

stripping off the decorations of each of the plurality of decorated queries by removing each suffix to obtain stripped queries; and

storing each suffix associated with each of the plurality of decorated queries for future appending.

9. The computer-implemented method of claim 2, further comprising binning the word lattice to organize each word lattice in columns containing the words candidates and associated confidence scores, each confidence score defined as a likelihood that the word candidate is a possible substitute for a query term.

13. The computer-implemented method of claim 8, further comprising:

inputting the stripped queries into an original Stemmer of the full-text search engine;

obtaining from the original Stemmer variations of each of the stripped queries to obtain query variations;

inputting the stripped queries and the query variations into the custom Stemmer;

appending the removed suffixes to their proper stripped queries and the query variations to obtain an expanded set of decorated queries; and

searching the full-text index using the expanded set of decorated queries.

14. The computer-implemented method of claim 9, further comprising:

quantizing a plurality of confidence scores into a discrete number of confidence scores; and

decorating a word candidate with a confidence score by appending to the word candidate a suffix corresponding to that confidence score.

3. A computer-implemented method for processing audio content, comprising:

generating word lattices for the audio content, the word lattices containing a plurality of word candidates and nodes that are connections between the word candidates;

indexing the word lattices using a text indexer of a full-text search engine such that the audio content is contained in a full-text index as indexed audio content;

inputting a query from a user containing at least one search term and a desired confidence level;

searching the indexed audio content to obtain search results corresponding to the query and the desired confidence level;

decorating the query with the desired confidence level and all confidence levels better than the desired confidence level by appending to the search term a suffix corresponding to the desired confidence level and all confidence levels better than the desired confidence level to obtain a decorated query;

inputting the decorated query into a Wordbreaker of the full-text search engine; and

expanding the decorated query into a plurality of decorated queries corresponding whose number is equal to a sum of the desired confidence level plus a number of all confidence levels better than the desired confidence level.

10. The computer-implemented method of claim 3, further comprising:

inputting the plurality of decorated queries into a custom Stemmer; and

stripping off the decorations of each of the plurality of decorated queries by removing each suffix to obtain stripped queries; and

storing each suffix associated with each of the plurality of decorated queries for future appending.

15. The computer-implemented method of claim 10, further comprising:

inputting the stripped queries into an original Stemmer of the full-text search engine;

obtaining from the original Stemmer variations of each of the stripped queries to obtain query variations;

inputting the stripped queries and the query variations into the custom Stemmer;

appending the removed suffixes to their proper stripped queries and the query variations to obtain an expanded set of decorated queries; and

searching the full-text index using the expanded set of decorated queries.