FAST OUT-OF-VOCABULARY SEARCH IN AUTOMATIC SPEECH RECOGNITION SYSTEMS

• US 20140188475A1
• Filed: 05/02/2013
• Published: 07/03/2014
• Est. Priority Date: 12/29/2012
• Status: Active Grant

• Current Assignee: Genesys Telecommunications Laboratories Incorporated (Genesys Cloud Services Incorporated)
• Original Assignee: Genesys Telecommunications Laboratories Incorporated (Genesys Cloud Services Incorporated)
• Inventors: Konig, Yochai, Lev-Tov, Amir, Faizakof, Avi
• Granted Patent: US 9,542,936 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A method including: receiving, on a computer system, a text search query, the query including one or more query words; generating, on the computer system, for each query word in the query, one or more anchor segments within a plurality of speech recognition processed audio files, the one or more anchor segments identifying possible locations containing the query word; post-processing, on the computer system, the one or more anchor segments, the post-processing including: expanding the one or more anchor segments; sorting the one or more anchor segments; and merging overlapping ones of the one or more anchor segments; and searching, on the computer system, the post-processed one or more anchor segments for instances of at least one of the one or more query words using a constrained grammar.

Claims

1. A method comprising:

receiving, on a computer system, a text search query, the query comprising one or more query words;

generating, on the computer system, for each query word in the query, one or more anchor segments within a plurality of speech recognition processed audio files, the one or more anchor segments identifying possible locations containing the query word;

post-processing, on the computer system, the one or more anchor segments, the post-processing comprising;

expanding the one or more anchor segments;

sorting the one or more anchor segments; and

merging overlapping ones of the one or more anchor segments; and

performing, on the computer system, speech recognition on the post-processed one or more anchor segments for instances of at least one of the one or more query words using a constrained grammar.

2. The method of claim 1, wherein the audio files are processed by a speech recognizer engine, and

wherein the generating, for each query word in the query, the one or more anchor segments of the processed audio files comprises:

determining if the query word is in a vocabulary of the speech recognizer engine;

when the query word is in the vocabulary, identifying one or more high confidence anchor segments corresponding to the query word; and

when the query word is not in the vocabulary, generating a search list of one or more sub-words of the query word and identifying one or more anchor segments containing at least one of the one or more sub-words.

3. The method of claim 2, wherein the generating the one or more anchor segments further comprises:

collecting low confidence words in the audio files, the low confidence words having word confidences below a threshold, and

wherein the identifying the one or more anchor segments corresponding to each of the sub-words comprises searching the low confidence words for only the sub-words of the query word when the query word is not in the vocabulary.

4. The method of claim 2, wherein the constrained grammar comprises one or more out-of-vocabulary query words of the query, wherein each of the out-of-vocabulary query words is not in the vocabulary.

5. The method of claim 1, wherein the searching comprises computing one or more event confidence levels, each of the event confidence levels corresponding to a confidence that an anchor segment of the one or more anchor segments contains a particular query word of the one or more query words of the query.

6. The method of claim 5, further comprising outputting, from the computer system, a result of the searching, wherein the result comprises the instances of the one or more query words in the audio file, sorted by event confidence level.

7. The method of claim 1, further comprising:

applying, on the computer system, a utility function to each of the one or more anchor segments to compute one or more corresponding anchor utility values; and

sorting, on the computer system, the one or more anchor segments in accordance with the one or more anchor utility values.

8. The method of claim 7, wherein the searching the one or more post-processed anchor segments only searches the one or more anchor segments having best anchor utility values of the one or more anchor utility values.

9. The method of claim 1, wherein the expanding the one or more anchor segments comprises:

for each query word in the query;

counting a first number of characters in the query before the query word and a second number of characters after the query word;

multiplying the first number of characters by an average character duration to obtain a first expansion amount; and

multiplying the second number of characters by the average character duration to obtain a second expansion amount; and

for each anchor segment, each anchor segment being identified by an anchor word, a start time, and an end time;

subtracting the first expansion amount and a first constant expansion duration from the start time; and

adding the second expansion amount and a second constant expansion duration to the end time.

10. A system comprising a computer system comprising a processor, memory, and storage, the system being configured to:

receive a text search query, the query comprising one or more query words;

generate, for each query word in the query, one or more anchor segments within a plurality of speech recognition processed audio files, the one or more anchor segments identifying possible locations containing the query word;

post-process the one or more anchor segments, the post-process comprising;

expanding the one or more anchor segments;

sorting the one or more anchor segments; and

merging overlapping ones of the one or more anchor segments; and

perform speech recognition on the post-processed one or more anchor segments for instances of at least one of the one or more query words using a constrained grammar.

11. The system of claim 10, wherein the system is configured to process the audio files using a speech recognizer engine, and

wherein the system is further configured to generate, for each query word in the query, the one or more anchor segments of the processed audio files by:

determining if the query word is in a vocabulary of a learning model of the speech recognizer engine;

when the query word is in the vocabulary, identifying one or more high confidence anchor segments corresponding to the query word; and

when the query word is not in the vocabulary, generating a search list of one or more sub-words of the query word and identifying one or more anchor segments corresponding to each of the one or more sub-words.

12. The system of claim 11, wherein the system is further configured to collect low confidence words in the audio files, the low confidence words having word confidences below a threshold, and

wherein the identifying the one or more anchor segments corresponding to each of the sub-words comprises searching the low confidence words for only the sub-words of the query word when the query word is not in the vocabulary.

13. The system of claim 11, wherein the constrained grammar comprises one or more out-of-vocabulary query words of the query, wherein each of the out-of-vocabulary query words is not in the vocabulary.

14. The system of claim 10, wherein the system is further configured to search the one or more post-processed anchor segments by computing one or more event confidence levels, each of the event confidence levels corresponding to a confidence that an anchor segment of the one or more anchor segments contains a particular query word of the one or more query words of the query.

15. The system of claim 14, wherein the system is further configured to output a result of the search, wherein the result comprises the instances of the query words in the audio file, sorted by event confidence level.

16. The system of claim 10, wherein the system is further configured to:

apply a utility function to each of the one or more anchor segments to compute one or more corresponding anchor utility values; and

sort the one or more anchor segments in accordance with the one or more anchor utility values.

17. The system of claim 16, wherein the system is configured to search the one or more post-processed anchor segments by only searching the one or more anchor segments having best anchor utility values of the one or more anchor utility values.

18. The system of claim 10, wherein the system is further configured to expand the one or more anchor segments by:

for each query word in the query;

counting a first number of characters in the query before the query word and a second number of characters after the query word;

multiplying the first number of characters by an average character duration to obtain a first expansion amount; and

multiplying the second number of characters by the average character duration to obtain a second expansion amount; and

for each anchor segment, each anchor segment being identified by an anchor word, a start time, and an end time;

subtracting the first expansion amount and a first constant expansion duration from the start time; and

adding the second expansion amount and a second constant expansion duration to the end time.

19. A system comprising

means for receiving a text search query, the query comprising one or more query words;

means for generating, for each query word in the query, one or more anchor segments identifying possible locations within a plurality of speech recognition processed audio files, the one or more anchor segments identifying possible locations containing the query word;

means for post-processing the one or more anchor segments comprising;

means for expanding the one or more anchor segments;

means for sorting the one or more anchor segments; and

means for merging overlapping ones of the one or more anchor segments; and

means for searching the post-processed one or more anchor segments for instances of at least one of the one or more query words using a constrained grammar.