Spoken document retrieval using multiple speech transcription indices

• US 9,405,823 B2
• Filed: 06/17/2008
• Issued: 08/02/2016
• Est. Priority Date: 07/23/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Mamou, Jonathan Joseph, Mass, Yosi
• Primary Examiner(s): Smith, Brannon W
• Application Number:
  US12/140,282
• Publication Number:
  US 20090030894A1
• Time in Patent Office: 2,968 Days
• Field of Search: 707/740, 707/999.102, 704/236
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3343 using phonetics
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method and system are provided of spoken document retrieval using multiple search transcription indices. The method includes receiving a query input formed of one or more query terms and determining a type of a query term, wherein a type includes a term in a speech recognition vocabulary or a term not in a speech recognition vocabulary. One or more indices of search transcriptions are selected for searching the query term based on the type of the query term. The one or more indices are generated using different speech transcription methods. The results for the query term are scored by the one or more indices and the results of the one or more indices for the query term are merged. The results of the one or more query terms are then merged to provide the results for the query.

Claims

1. A method of spoken document retrieval by a system using multiple search transcription indices, the method comprising:

receiving an input phrase as a search query formed of query terms;

identifying a first type of query term in the input phrase, wherein the first type includes a query term in a speech recognition vocabulary of the system;

identifying one or more first search transcription indices for searching the first type of query term;

identifying a second type of query term in the input phrase, wherein the second type includes a query term not in the speech recognition vocabulary;

identifying a plurality of second search transcription indices for searching the second type of query term;

receiving at least a first list of results from the one or more first search transcription indices;

receiving plural lists of results from the plurality of second search transcription indices;

ranking results from a phonetic index of the plurality of second search transcription indices based on temporal proximity of phones, wherein the ranking computes a score in accordance with the following expression;

2. The method as claimed in claim 1, wherein combining the first scores according to the first method comprises using an aggregate sum and OR semantics and combining the one or more third scores according to the second method comprises using an aggregate sum with AND semantics.

3. The method as claimed in claim 1, wherein the plurality of second search transcription indices are generated using different speech transcription methods.

4. The method as claimed in claim 1, wherein the one or more first search transcription indices are taken from the group of:

word indices, sub-word word-fragment indices, sub-word phonetic indices, a combination of word and sub-word indices using timestamps, and a combination of word and sub-word indices using offsets.

5. The method as claimed in claim 1, wherein combining the first scores and combining the one or more third scores further comprise using a Threshold Algorithm to simultaneously scan the results from different indices to aggregate retrieved document scores.

6. The method as claimed in claim 5, wherein the Threshold Algorithm uses AND or OR semantics between the indices.

7. The method as claimed in claim 5, wherein combining the first scores and combining the one or more third scores further comprise using a weighted sum with weights allocated for different indices.

8. The method as claimed in claim 1, further comprising scoring results from the received lists of results, wherein the scoring includes an act selected from the group of:

scoring results from a word index using weighted term frequency;

scoring results for a sub-word index using computed confusion cost of sub-word in a query term;

Boolean model scoring;

vector space scoring;

or edit distance scoring.

9. The method as claimed in claim 1, wherein for a query term formed of a term in the speech recognition vocabulary, one or more word indices and one or more sub-word indices are selected.

10. The method as claimed in claim 1, wherein for a query term formed of a term not in the speech recognition vocabulary, one or more sub-word indices are selected.

11. The method as claimed in claim 1, further comprising:

identifying a third search transcription index for searching at least one of the query terms, wherein the third search transcription index includes words and sub-words; and

receiving a third list of results from the third index.

12. A non-transitory computer-readable storage medium comprising computer-executable instructions that, when executed by a computer, cause the computer to execute acts of:

receiving an input phrase as a search query formed of query terms;

identifying a first type of query term in the input phrase, wherein the first type includes a query term in a speech recognition vocabulary of the system;

identifying one or more first search transcription indices for searching the first type of query term;

identifying a second type of query term in the input phrase, wherein the second type includes a query term not in the speech recognition vocabulary;

identifying a plurality of second search transcription indices for searching the second type of query term;

receiving at least a first list of results from the one or more first search transcription indices;

receiving plural lists of results from the plurality of second search transcription indices;

ranking results from a phonetic index of the plurality of second search transcription indices based on temporal proximity of phones, wherein the ranking computes a score in accordance with the following expression;

13. A method of providing a service to a customer over a network for spoken document retrieval, the service comprising:

receiving an input phrase as a search query formed of query terms;

identifying a first type of query term in the input phrase, wherein the first type includes a query term in a speech recognition vocabulary of the system;

identifying one or more first search transcription indices for searching the first type of query term;

identifying a second type of query term in the input phrase, wherein the second type includes a query term not in the speech recognition vocabulary;

identifying a plurality of second search transcription indices for searching the second type of query term;

receiving at least a first list of results from the one or more first search transcription indices;

receiving plural lists of results from the plurality of second search transcription indices;

ranking results from a phonetic index of the plurality of second search transcription indices based on temporal proximity of phones, wherein the ranking computes a score in accordance with the following expression;

14. A search system for spoken document retrieval using multiple search transcription indices, the system comprising:

a processor;

a memory containing instructions that, when executed by the processor, adapt the search system to;

receive a search query formed of one or more query terms,

determine a first type and a second type of query term by reference to a speech recognition vocabulary, wherein the first type includes a term in a speech recognition vocabulary and the second type includes a term not in the speech recognition vocabulary,

identify multiple search transcription indices for searching a first query term based on the type of the first query term, wherein the multiple indices include a first phonetic index and a second index that is different from the first index,

rank results from the second index based on temporal proximity of phones, wherein the ranking computes a score in accordance with the following expression;

15. The search system as claimed in claim 14, wherein the multiple indices are taken from the group of:

word indices, sub-word word-fragment indices, sub-word phonetic indices, word and sub-word indices using timestamps, and word and sub-word indices using offsets.

16. The search system as claimed in claim 14, wherein the merging of the lists uses a Threshold Algorithm to scan results from different indices for a same document to aggregate document scores for the same document.