Multimedia recognition system comprising a plurality of indexers configured to receive and analyze multimedia data based on training data and user augmentation relating to one or more of a plurality of generated documents

• US 7,801,838 B2
• Filed: 07/02/2003
• Issued: 09/21/2010
• Est. Priority Date: 07/03/2002
• Status: Expired due to Fees

• Current Assignee: Ramp Holdings Incorporated (Clean Harbors Incorporated)
• Original Assignee: Ramp Holdings Incorporated (Clean Harbors Incorporated)
• Inventors: Kubala, Francis G., Shepard, Scott, Colbath, Sean
• Primary Examiner(s): Sparks; Donald
• Assistant Examiner(s): Brown, Jr.; Nathan H
• Application Number:
  US10/610,533
• Publication Number:
  US 20040006737A1
• Time in Patent Office: 2,638 Days
• Field of Search: 706/20
• US Class Current: 706/20
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 2201/42 Graphical user interfaces
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/305 Recording playback features, e.g. increased speed
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A system improves recognition results. The system receives multimedia data and recognizes the multimedia data based on training data to generate documents. The system receives user augmentation relating to one of the documents or new documents from a user. The system supplements the training data with the user augmentation or new documents and retrains based on the supplemented training data.

Claims

1. A multimedia recognition system, comprising:

a plurality of indexers configured to;

receive multimedia data, and analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers, receive user augmentation relating to one of the documents, and provide the user augmentation to one or more of the indexers for retraining based on the user augmentation, wherein when receiving user augmentation relating to one of the documents, the memory system is configured to;

receive correction of the one of the documents, as a corrected document, from a user, and store the corrected document.

2. The system of claim 1, wherein when providing the user augmentation to one or more of the indexers, the memory system is configured to send the corrected document to the one or more of the indexers for retraining based on the corrected document.

3. The system of claim 2, wherein the one or more of the indexers are configured to:

add the corrected document to the training data, and

retrain based on the training data.

4. A multimedia recognition system, comprising:

a plurality of indexers configured to;

receive multimedia data, and analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers, receive user augmentation relating to one of the documents, and provide the user augmentation to one or more of the indexers for retraining based on the user augmentation, wherein when receiving user augmentation relating to one of the documents, the memory system is configured to;

receive enhancement of the one of the documents, as an enhanced document, from a user, and store the enhanced document.

5. The system of claim 4, wherein when providing the user augmentation to one or more of the indexers, the memory system is configured to send the enhanced document to the one or more of the indexers for retraining based on the enhanced document.

6. The system of claim 5, wherein the one or more of the indexers are configured to:

add the enhanced document to the training data, and

retrain based on the training data.

7. A multimedia recognition system, comprising:

a plurality of indexers configured to;

receive multimedia data, and analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers, receive user augmentation relating to one of the documents, and provide the user augmentation to one or more of the indexers for retraining based on the user augmentation, wherein when receiving user augmentation relating to one of the documents, the memory system is configured to;

receive annotation of the one of the documents, as an annotated document, from a user, and store the annotated document.

8. The system of claim 7, wherein when providing the user augmentation to one or more of the indexers, the memory system is configured to send the annotated document to the one or more of the indexers for retraining based on the annotated document.

9. The system of claim 8, wherein the one or more of the indexers are configured to:

add the annotated document to the training data, and

retrain based on the training data.

10. The system of claim 7, wherein when receiving annotation of the one of the documents, the memory system is configured to at least one of:

receive a bookmark relating to the one of the documents, receive highlighting regarding one or more portions of the one of the documents, and receive a note relating to at least a portion of the one of the documents.

11. The system of claim 10, wherein the note includes one of comments from the user, one of an audio, video, and text file, and a reference to another one of the documents.

12. A multimedia recognition system, comprising:

a plurality of indexers configured to;

receive multimedia data, and analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers, receive user augmentation relating to one of the documents, and provide the user augmentation to one or more of the indexers for retraining based on the user augmentation, wherein when receiving user augmentation relating to one of the documents, the memory system is configured to;

receive an attachment for the one of the documents from a user, and store the attachment.

13. The system of claim 12, wherein when providing the user augmentation to one or more of the indexers, the memory system is configured to send the attachment to the one or more of the indexers for retraining based on the attachment.

14. The system of claim 13, wherein the one or more of the indexers are configured to:

add the attachment to the training data, and retrain based on the training data.

15. The system of claim 12, wherein the attachment includes one of an audio document, a video document, and a text document, or a reference to the audio document, the video document, or the text document.

16. The system of claim 12, wherein the memory system is further configured to:

send the attachment for analysis by one or more of the indexers.

17. A method for improving recognition results, comprising:

receiving multimedia data;

recognizing the multimedia data based on training data to generate a plurality of documents;

receiving user augmentation relating to one of the documents, wherein the receiving user augmentation relating to one of the documents includes;

receiving correction of the one of the documents, as a corrected document, from a user, and storing the corrected document;

supplementing the training data with the user augmentation to obtain supplemented training data; and

retraining based on the supplemented training data.

18. The method of claim 17, wherein the supplementing the training data includes:

adding the corrected document to the training data.

19. A method for improving recognition results, comprising:

receiving multimedia data;

recognizing the multimedia data based on training data to generate a plurality of documents;

receiving user augmentation relating to one of the documents wherein the receiving user augmentation relating to one of the documents includes;

receiving enhancement of the one of the documents, as an enhanced document, from a user, and storing the enhanced document;

supplementing the training data with the user augmentation to obtain supplemented training data; and

retraining based on the supplemented training data.

20. The method of claim 19, wherein the supplementing the training data includes:

adding the enhanced document to the training data.

21. A method for improving recognition results, comprising:

receiving multimedia data;

recognizing the multimedia data based on training data to generate a plurality of documents;

receiving user augmentation relating to one of the documents wherein the receiving user augmentation relating to one of the documents includes receiving annotation of the one of the documents, as an annotated document, from a user, and storing the annotated document;

supplementing the training data with the user augmentation to obtain supplemented training data; and

retraining based on the supplemented training data.

22. The method of claim 21, wherein the supplementing the training data includes:

adding the annotated document to the training data.

23. The method of claim 21, wherein the receiving annotation of the one of the documents includes at least one of:

receiving a bookmark relating to the one of the documents, receiving highlighting regarding one or more portions of the one of the documents, and receiving a note relating to at least a portion of the one of the documents.

24. The method of claim 23, wherein the note includes one of comments from the user, one of an audio, video, and text file, and a reference to another one of the documents.

25. A method for improving recognition results, comprising:

receiving multimedia data;

recognizing the multimedia data based on training data to generate a plurality of documents;

receiving user augmentation relating to one of the documents wherein the receiving user augmentation relating to one of the documents includes receiving an attachment for the one of the documents from a user, and storing the attachment;

supplementing the training data with the user augmentation to obtain supplemented training data; and

retraining based on the supplemented training data.

26. The method of claim 25, wherein the supplementing the training data includes:

adding the attachment to the training data.

27. The method of claim 25, wherein the attachment includes one of an audio document, a video document, and a text document, or a reference to the audio document, the video document, or the text document.

28. A multimedia recognition system, comprising:

a plurality of indexers configured to;

receive multimedia data, and analyze the multimedia data based on training data to generate a plurality of documents; and

a memory system configured to;

store the documents from the indexers, obtain new documents, store the new documents, and provide the new documents to one or more of the indexers for retraining based on the new documents, wherein when obtaining one of the new documents, the memory system is configured to at least one of;

receive text that has been cut-and-pasted, receive a file containing the one of the new documents, receive a link to the one of the new documents and employ an agent to actively seek out and retrieve new documents.

29. The system of claim 28, wherein the one or more of the indexers are configured to:

add the new documents to the training data, and retrain based on the training data.

30. A method for improving recognition results, comprising:

receiving multimedia data;

recognizing the multimedia data based on training data to generate a plurality of documents;

obtaining new documents, wherein the obtaining new documents includes at least one of receiving text that has been cut-and-pasted, receiving one or more files containing the new documents, receiving one or more links to-the new documents and actively seeking out and retrieving the new documents;

supplementing the training data with the new documents to obtain supplemented training data; and

retraining based on the supplemented training data.