Indexing and searching speech with text meta-data

• US 20070106509A1
• Filed: 11/08/2005
• Published: 05/10/2007
• Est. Priority Date: 11/08/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chelba, Ciprian, Sanchez, Jorge, Acero, Alejandro
• Granted Patent: US 7,809,568 B2
• Time in Patent Office: Days
• US Class Current: 704/240
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

An index for searching spoken documents having speech data and text meta-data is created by obtaining probabilities of occurrence of words and positional information of the words of the speech data and combining it with at least positional information of the words in the text meta-data. A single index can be created because the speech data and the text meta-data are treated the same and considered only different categories.

Claims

1. A method of indexing a spoken document comprising speech data and text meta-data, the method comprising:

generating information pertaining to recognized speech from the speech data, the information comprising probabilities of occurrence of words and positional information of the words in the recognized speech;

generating information pertaining to at least positional information of words in the text meta-data in substantially the same format as the information pertaining to recognized speech; and

building an index based on the information pertaining to recognized speech and the information pertaining to the text meta-data.

2. The method of claim 1 and further comprising categorizing at least one of speech data and text meta-data.

3. The method of claim 2 wherein categorizing includes categorizing different types of speech data.

4. The method of claim 2 wherein categorizing includes categorizing different types of text meta-data.

5. The method of claim 2 wherein building the index includes building the index with category information.

6. The method of claim 1 wherein generating information pertaining to recognized speech from the speech data comprises generating a lattice.

7. The method of claim 4 wherein generating information pertaining to the text meta-data comprises generating a lattice.

8. The method of claim 1 wherein generating information pertaining to recognized speech from the speech data includes identifying at least two alternative speech unit sequences based on the same portion of speech data;

and wherein building an index based on the information pertaining to recognized speech includes for each speech unit in the at least two alternative speech unit sequences, placing information in an entry in the index that indicates a position of the speech unit in at least one of the two alternative speech unit sequences.

9. The method of claim 1 wherein generating information pertaining to recognized speech from the speech data includes determining the probability of the speech unit appearing at the position by summing at least two probabilities, each probability associated with the speech unit appearing in a separate speech unit sequence.

10. A computer-readable medium having computer-executable instructions for performing steps comprising:

receiving a search query;

searching an index for an entry associated with a word in the search query, the index comprising;

information pertaining to a document identifier for a spoken document having speech data and text meta-data;

a category type identifier identifying at least one of different types of speech data, and speech data relative to text meta-data; and

a position for the word, and a probability of the word appearing at the position,

using the probabilities to rank spoken documents relative to each other; and

returning search results based on the ranked spoken documents.

11. The computer-readable medium of claim 10 wherein using the probabilities to rank the spoken documents comprises calculating a collection of composite n-gram scores for each spoken document.

12. The computer-readable medium of claim 11 wherein each composite n-gram score is formed by summing individual n-gram scores over all possible formations of an n-gram.

13. The computer-readable medium of claim 12 wherein the collection of composite n-gram scores is calculated based on different category types.

14. The computer-readable medium of claim 13 wherein a score for a category type is calculated by summing together each of the composite n-gram scores of each respective category type.

15. The computer-readable medium of claim 10 wherein using the probabilities to rank the spoken documents comprises calculating a document score as a combination of the category type scores.

16. The computer-readable medium of claim 15 wherein the category type scores are weighted.

17. A method of retrieving spoken documents based on a search query, the method comprising:

receiving the search query;

searching an index containing probabilities of positions for words generated from speech data in the spoken documents, the probabilities of positions for words referenced to different categories of speech data in the spoken document;

scoring each spoken document based on a set of probabilities for a word from the index for each category; and

returning search results based on the ranked spoken documents

18. The method of claim 17 wherein scoring each spoken document comprises calculating a document score as a weighted combination of scores for each different category of speech data.

19. The method of claim 17 wherein the index further includes probabilities of positions for words generated from text meta-data in the spoken documents, the probabilities of positions for words referenced to different categories of text meta-data in the spoken document.

20. The method of claim 19 wherein scoring each spoken document comprises calculating a document score as a weighted combination of scores for each different category of speech data and each different category of text meta-data.