Speech recognition training method for audio and video file indexing on a search engine

• US 7,272,558 B1
• Filed: 03/23/2007
• Issued: 09/18/2007
• Est. Priority Date: 12/01/2006
• Status: Active Grant

• Current Assignee: Coveo Solutions, Inc.
• Original Assignee: Coveo Solutions, Inc.
• Inventors: Soucy, Pascal, Simoneau, Laurent, Tessier, Richard
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US11/690,235
• Time in Patent Office: 179 Days
• Field of Search: 704/231, 704/235, 704/243, 704/244, 704/251, 704/255, 707/2, 707/3
• US Class Current: 704/235
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/735 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing

Abstract

A method and a related system to index audio and video documents and to automatically train the language model of a speech recognition system according to the context of the documents being indexed.

Claims

1. A method for indexing audio/video documents through the use of a search engine, the method comprising:

providing, to the search engine, a source of training documents comprising textual content;

the search engine retrieving at least some of the training documents from the source of training documents;

the search engine extracting the textual content from the retrieved training documents;

the search engine indexing the textual content;

training a speech recognition profile using the indexed textual content;

providing, to the search engine, a source for the audio/video documents each of which comprise an associated audio content;

the search engine retrieving at least some of the audio/video documents from the source of documents;

the search engine extracting the associated content from the audio/video documents;

converting the associated audio content into transcriptions using the trained speech recognition profile;

the search engine indexing the transcriptions thereby resulting in an indexing of the audio/video documents; and

saving the indexed transcriptions;

wherein the training of the speech recognition profile comprises using summary sentences and comparing the number of sentences to a threshold to determine if all sentences will be kept for the training.

2. The method of claim 1, wherein the providing a source of training documents comprises crawling and retrieving Web documents on a Web site.

3. The method of claim 1, wherein the providing a source of training documents comprises crawling and retrieving files on at least one of a local computer network, Intranet, Extranet, file repository or email messaging system.

4. The method of claim 1, wherein the providing a source of training documents comprises crawling and retrieving data from a database.

5. The method of claim 1, wherein the retrieving of the audio content comprises retrieving the audio content from the source of training documents.

6. The method of claim 1, wherein the retrieving of the audio content comprises retrieving the audio content from a source other than the source of training documents.

7. The method of claim 1, further comprising;

obtaining user input queries;

using the queries, finding sentences from the indexed textual content; and

using the found sentences as input to perform the speech recognition profile training.

8. The method of claim 1, wherein the speech recognition profile comprises at least one of a dictionary and a language model.

9. The method of claim 1, further comprising a sentence selection process which comprises calculating a score for each sentence and keeping only those sentences which achieve a given score.

10. The method of claim 9, wherein the score is calculated using at least one of the number of words and the number of non-alphabetic characters in each of the sentences.

11. The method of claim 9, wherein the score is calculated by using the number of proper nouns in each of the sentences.

12. The method of claim 11, wherein the score is calculated by using the frequency of the proper nouns contained in each sentence in a pool of previously selected sentences.

13. The method of claim 11, wherein the score calculation results in a higher score when the sentence contains proper nouns that are infrequent in the pool of already selected sentences.

14. A method of searching for audio/video documents comprising:

querying an index with a given search criterion, the database comprising transcriptions indexed according to the method of claim 1; and

displaying the search results to a user.

15. A search engine system for indexing audio/video documents comprising:

a search engine for;

receiving a source of training documents comprising textual content;

retrieving at least some of the training documents from the source of training documents;

extracting the textual content from the retrieved training documents;

indexing the textual content;

receiving a source for the audio/video documents each of which comprise an associated audio content;

retrieving at least some of the audio/video documents from the source of documents; and

extracting the associated audio content from the audio/video documents;

a training engine for training a speech recognition profile using the indexed textual content;

a speech recognition engine converting the associated audio content into transcriptions using the trained speech recognition profile;

the search engine further for indexing the transcriptions thereby resulting in an indexing of the audio/video documents; and

an index for saving the indexed transcriptions;

wherein the training of the speech recognition profile comprises using summary sentences and comparing the number of sentences to a threshold to determine if all sentences will be kept for the training.

16. The search engine system of claim 15 further comprising a user interface for receiving a query from a user, the query comprising a given search criterion and being used by the search engine for querying the index with the given search criterion, the search engine returning a search result for display by the user interface.