Systems and methods for improving recognition results via user-augmentation of a database

• US 8,001,066 B2
• Filed: 08/13/2010
• Issued: 08/16/2011
• Est. Priority Date: 07/03/2002
• Status: Active Grant

• Current Assignee: Francis G. Kubala, Scott Shepard, Sean Colbath
• Original Assignee: Francis G. Kubala, Scott Shepard, Sean Colbath
• Inventors: Kubala, Francis G., Shepard, Scott, Colbath, Sean
• Primary Examiner(s): Gaffin; Jeffrey A
• Assistant Examiner(s): BROWN JR, NATHAN H
• Application Number:
  US12/806,465
• Publication Number:
  US 20110004576A1
• Time in Patent Office: 368 Days
• Field of Search: 706/20, 706/16
• US Class Current: 706/20
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 2201/42 Graphical user interfaces
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/305 Recording playback features, e.g. increased speed
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A system improves recognition results. The system receives multimedia data and recognizes the multimedia data based on training data to generate documents. The system receives user augmentation relating to one of the documents or new documents from a user. The system supplements the training data with the user augmentation or new documents and retrains based on the supplemented training data.

Claims

1. A multimedia recognition system comprising:

a plurality of indexers configured to;

receive multimedia data, the multimedia data including at least two of audio data, video data, and text data, and

analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers,

receive user augmentation relating to one of the documents, the user augmentation including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents, and

provide the user augmentation to one or more of the indexers for retraining based on the user augmentation.

2. The system of claim 1 wherein the indexers include at least two of:

an audio indexer configured to perform speech recognition on the audio data based on the training data, a video indexer configured to perform at least one of video recognition and speech recognition on the video data based on the training data, and a text indexer configured to perform text recognition on the text data based on the training data.

3. A multimedia recognition system comprising:

means for receiving a plurality of types of multimedia data;

means for recognizing the multimedia data based on training data to generate recognition results;

means for storing the recognition results;

means for receiving user augmentation relating to some of the recognition results, the user augmentation including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents;

means for adding the user augmentation to the training data to obtain new training data; and

means for retraining based on the new training data.

4. A method for improving recognition results comprising:

receiving multimedia data, the multimedia data including at least two of audio data, video data, and text data;

recognizing the multimedia data based on training data to generate a first plurality of documents;

receiving user augmentation relating to one of the documents;

supplementing the training data with the user augmentation to obtain supplemented training data, the user augmentation including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents; and

recognizing the multimedia data based on the supplemented training data to generate a second plurality of documents.

5. The method of claim 4 wherein the recognizing the multimedia data includes at least two of:

performing speech recognition on the audio data based on the training data, performing at least one of video recognition and speech recognition on the video data based on the training data, and performing text recognition on the text data based on the training data.

6. The method of claim 5 further comprising:

performing at least one of speech recognition, video recognition, and text recognition on the attachment.

7. A computer-readable medium that stores instructions executable by one or more processors for improving recognition of multimedia data comprising:

instructions for acquiring multimedia data;

instructions for recognizing the multimedia data based on training data to generate a plurality of documents;

instructions for obtaining user augmentation relating to one of the documents, the user augmentation including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents;

instructions for adding the user augmentation to the training data to obtain new training data; and

instructions for retraining based on the new training data.

8. A multimedia recognition system comprising:

a plurality of indexers configured to;

receive multimedia data, the multimedia data including at least two of audio data, video data, and text data, and

analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers,

obtain new documents, the new documents including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents,

store the new documents, and

provide the new documents to one or more of the indexers for retraining based on the new documents.

9. The system of claim 8 wherein the indexers include at least two of:

an audio indexer configured to perform speech recognition on the audio data based on the training data, a video indexer configured to perform at least one of video recognition and speech recognition on the video data based on the training data, and a text indexer configured to perform text recognition on the text data based on the training data.

10. The system of claim 8 wherein when obtaining the new documents, the memory system is configured to:

employ an agent to actively seek out and retrieve new documents.

11. A multimedia recognition system comprising:

means for receiving a plurality of types of multimedia data;

means for recognizing the multimedia data based on training data to generate recognition results;

means for obtaining new documents from one or more users, the new documents including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents;

means for adding the new documents to the training data to obtain new training data; and

means for retraining based on the new training data.

12. A method for improving recognition results comprising:

receiving multimedia data, the multimedia data including at least two of audio data, video data, and text data;

recognizing the multimedia data based on training data to generate a plurality of documents;

obtaining new documents;

supplementing the training data with the new documents to obtain supplemented training data, the new documents including one or more of correction and/or enhancement of the documents, annotation of the documents with bookmarks, highlights, and notes, attachment of rich documents and insertion of rich documents; and

recognizing the multimedia data based on supplemented training data to generate a second plurality of documents.

13. The method of claim 12 wherein the recognizing the multimedia data includes at least two of:

performing speech recognition on the audio data based on the training data,

performing at least one of video recognition and speech recognition on the video data based on the training data, and

performing text recognition on the text data based on the training data.

14. The method of claim 12 wherein the obtaining the new documents includes:

actively seeking out and retrieving the new documents.