Method and apparatus for fusion of recognition results from multiple types of data sources

• US 20050125224A1
• Filed: 11/08/2004
• Published: 06/09/2005
• Est. Priority Date: 11/06/2003
• Status: Abandoned Application

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Franco, Horacio F., Venkataraman, Anand, Myers, Gregory K., Stolcke, Andreas, Bratt, Harry, Rao Gadde, Venkata Ramana
• Application Number:
  US10/983,505
• Publication Number:
  US 20050125224A1
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A method and apparatus are provided for fusion of recognition results from multiple types of data sources. In one embodiment, the inventive method implementing a first processing technique to recognize at least a portion of terms contained in a first media source, implementing a second processing technique to recognize at least a portion of terms contained in a second media source that contains a different type of data than that contained in the first media source, and adapting the first processing technique based at least in part on results generated by the second processing technique.

Claims

1. A method for fusing recognition results from at least two different media sources, the method comprising:

recognizing at least a portion of at least one term contained in a first media source using a first processing technique;

recognizing at least a portion of at least one term contained in a second media source using a second processing technique, where said second media source contains a type of data that is different from a type of data contained in said first media source; and

adapting said first processing technique based at least in part on a result generated by said second processing technique.

2. The method of claim 1, further comprising the step of:

implementing said adapted first processing technique to re-recognize at least a portion of at least one term contained in said first media source.

3. The method of claim 2, where said re-recognition is performed on said first media source in an original form.

4. The method of claim 2, wherein said re-recognition is performed on said first media source in an intermediate form.

5. The method of claim 1, wherein said first processing technique and said second processing technique are implemented sequentially.

6. The method of claim 1, wherein said first processing technique and said second processing technique are implemented in parallel.

7. The method of claim 1, wherein said first media source includes at least one of an audio signal, a video signal, a still image, a document, an internet web page and manually inputted text.

8. The method of claim 1, wherein said second media source includes at least one of an audio signal, a video signal, a still image, a document, an internet web page and manually inputted text.

9. The method of claim 1, wherein said adapting step comprises:

searching recognition results produced by said second processing technique for new words not contained within a vocabulary of said first processing technique; and

adding said new words to said vocabulary of said first processing technique.

10. The method of claim 1, wherein at least one of said first processing technique and said second processing technique is implemented to recognize sub-word elements.

11. The method of claim 10, wherein said sub-word elements comprise at least one of characters contained within text-based words or phones contained within spoken words.

12. The method of claim 11, wherein said first processing technique produces a first result lattice comprising one or more potential sub-word elements contained within said first media source, and said second processing technique produces a second result lattice comprising one or more potential sub-word elements contained within said second media source.

13. The method of claim 12, wherein said adapting step comprises:

generating a first spelling lattice based on said first result lattice;

generating a second spelling lattice based on said second result lattice; and

combining said first spelling lattice and said second spelling lattice to form a combined spelling lattice.

14. The method of claim 13, further comprising the steps of:

identifying a most probable path within said combined spelling lattice, where said most probable path represents a likely spelling of a word contained within said first and second media sources;

selecting recognized sub-word elements from said second media source that correspond to said most probable path; and

adding a word produced by said recognized sub-word elements to a vocabulary of said first processing technique.

15. The method of claim 1, wherein said at least one term is at least one of a phone, a word, a phrase a sentence, a character and a number.

24. The computer readable medium of claim 15, wherein said adapting step comprises:

searching recognition results produced by said second processing technique for new words not contained within a vocabulary of said first processing technique; and

adding said new words to said vocabulary of said first processing technique.

16. A computer readable medium containing an executable program for fusing recognition results from at least two different media sources, where the program performs the steps of:

recognizing at least a portion of at least one term contained in a first media source using a first processing technique;

recognizing at least a portion of at least one term contained in a second media source using a second processing technique, where said second media source contains a type of data that is different from a type of data contained in said first media source; and

adapting said first processing technique based at least in part on a result generated by said second processing technique.

17. The computer readable medium of claim 16, further comprising the step of:

implementing said adapted first processing technique to re-recognize at least a portion of at least one term contained in said first media source.

18. The computer readable medium of claim 17, where said re-recognition is performed on said first media source in an original form.

19. The computer readable medium of claim 17, wherein said re-recognition is performed on said first media source in an intermediate form.

20. The computer readable medium of claim 16, wherein said first processing technique and said second processing technique are implemented sequentially.

21. The computer readable medium of claim 16, wherein said first processing technique and said second processing technique are implemented in parallel.

22. The computer readable medium of claim 16, wherein said first media source includes at least one of an audio signal, a video signal, a still image, a document, an internet web page and manually inputted text.

23. The computer readable medium of claim 16, wherein said second media source includes at least one of an audio signal, a video signal, a still image, a document, an internet web page and manually inputted text.

25. The computer readable medium of claim 16, wherein at least one of said first processing technique and said second processing technique is implemented to recognize sub-word elements.

26. The computer readable medium of claim 25, wherein said sub-word elements comprise at least one of characters contained within text-based words or phones contained within spoken words.

27. The computer readable medium of claim 26, wherein said first processing technique produces a first result lattice comprising one or more potential sub-word elements contained within said first media source, and said second processing technique produces a second result lattice comprising one or more potential sub-word elements contained within said second media source.

28. The computer readable medium of claim 27, wherein said adapting step comprises:

generating a first spelling lattice based on said first result lattice;

generating a second spelling lattice based on said second result lattice; and

combining said first spelling lattice and said second spelling lattice to form a combined spelling lattice.

29. The computer readable medium of claim 28, further comprising the steps of:

identifying a most probable path within said combined spelling lattice, where said most probable path represents a likely spelling of a word contained within said first and second media sources;

selecting recognized sub-word elements from said second media source that correspond to said most probable path; and

adding a word produced by said recognized sub-word elements to a vocabulary of said first processing technique.

30. The computer readable medium of claim 16, wherein said at least one term is at least one of a phone, a word, a phrase a sentence, a character and a number.

31. Apparatus for fusing recognition results from at least two different media sources, the apparatus comprising:

means for recognizing at least a portion of at least one term contained in a first media source using a first processing technique;

means for recognizing at least a portion of at least one term contained in a second media source using a second processing technique, where said second media source contains a type of data that is different from a type of data contained in said first media source; and

means for adapting said first processing technique based at least in part on a result generated by said second processing technique.