“At least” operator for combining audio search hits

• US 9,535,987 B2
• Filed: 01/25/2016
• Issued: 01/03/2017
• Est. Priority Date: 03/30/2012
• Status: Active Grant

• Current Assignee: Avaya Incorporated
• Original Assignee: Avaya Incorporated
• Inventors: Ponting, Keith Michael
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US15/005,661
• Publication Number:
  US 20160140223A1
• Time in Patent Office: 344 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/632 Query formulation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

System and method to search audio data, including: receiving audio data representing speech; receiving a search query related to the audio data; compiling, by use of a processor, the search query into a hierarchy of scored speech recognition sub-searches; searching, by use of a processor, the audio data for speech identified by one or more of the sub-searches to produce hits; and combining, by use of a processor, the hits by use of at least one combination function to provide a composite search score of the audio data. The combination function may include an at-least-M-of-N function that produces a high score when at least M of N function inputs exceed a predetermined threshold value. The composite search score employ a soft time window such as a spline function.

Claims

1. A method to search audio data, comprising:

receiving, by a server, audio data representing speech;

receiving, by the server, a search query related to the audio data, the search query comprising a plurality of search terms;

compiling, by the server, the search query into a hierarchy of scored speech recognition sub-searches;

searching, by an audio search engine executing on the server, the audio data for speech identified by one or more of the sub-searches to produce hits, each hit being associated with a respective search term in the search query;

determining, by the server, whether to record each hit in a hit store, wherein a particular hit is recorded only when no previous hits associated with the respective search term are found or when a relevancy score of the particular hit is higher than a relevancy score of all previous hits associated with the respective search term;

determining, by the server, whether the hits recorded in the hit store exceed a predetermined number; and

when the hits exceed the predetermined number, combining, by the server, the hits by use of at least one combination function to provide a composite search score of the audio data, wherein at least one of the combination functions comprises an at-least-M-of-N function, the predetermined number of hits comprising hits for at least M members of a set comprising N members, wherein N is greater than or equal to M, and wherein the composite search score is determined based on the M members having a highest relevancy score.

2. The method of claim 1, wherein the high score is produced by applying a transform function to at least one of N function inputs.

3. The method of claim 1, wherein the at-least-M-of-N function comprises an ALL-function, wherein the ALL-function produces a high score when all inputs to the at-least-M-of-N function exceed the predetermined threshold value within a predetermined time window.

4. The method of claim 1, wherein the search query comprises one or more search terms and each hit comprises an identifier that indicates a match to one or more of the plurality of search terms, a time interval over which the match is declared, and the relevancy score.

5. The method of claim 4, further comprising:

applying, by the server, a respective threshold to one or more of the hits obtained from the step of searching the audio data for speech identified by one or more of the sub-searches; and

blocking, by the server, the hits having relevance scores below their respective threshold.

6. The method of claim 1 wherein the audio data comprises content-dependent metadata and content-independent metadata.

7. The method of claim 6 wherein the content-dependent metadata comprises:

an identification of a predetermined speaker;

a time interval during which the speaker was speaking; and

a quality score of the identification of the predetermined speaker.

8. The method of claim 1 wherein the composite search score comprises an attribute with an associated score and one or more associated time intervals.

9. The method of claim 1 wherein the composite search score may be determined in accordance with the following relationship:

10. The method of claim 9 wherein the sensitivity vectors are obtained from user feedback that indicates a correctness of the composite search score.

11. The method of claim 10, further comprising:

self-learning, by the server, of the sensitivity vectors based upon the user feedback.

12. The method of claim 1, further comprising:

scoring, by the server, the speech recognition by use of a soft time window.

13. The method of claim 12 wherein the soft time window comprises a window definable by a spline function.

14. A system to search audio data, comprising:

a computer-readable storage medium, storing executable instructions;

a processor in communication with the computer-readable storage medium, the processor when executing the executable instructions;

receives audio data representing speech;

receives a search query related to the audio data, the search query comprising a plurality of search terms;

compiles the search query into a hierarchy of scored speech recognition sub-searches;

searches, by an audio search engine executing on the processor, the audio data for speech identified by one or more of the sub-searches to produce hits, each hit being associated with a respective search term in the search query; and

determines whether to record each hit in a hit store, wherein a particular hit is recorded only when no previous hits associated with the respective search term are found or when a relevancy score of the particular hit is higher than a relevancy score of all previous hits associated with the respective search term;

determines whether the hits recorded in the hit store exceed a predetermined number; and

when the hits exceed the predetermined number, combines the hits by use of at least one combination function to provide a composite search score of the audio data, wherein at least one of the combination functions comprises an at-least-M-of-N function, the predetermined number of hits comprising hits for at least M members of a set comprising N members, wherein N is greater than or equal to M, and wherein the composite search score is determined based on the M members having a highest relevancy score.

15. The system of claim 14, wherein the high score is produced by applying a transform function to at least one of N function inputs.

16. The system of claim 14 wherein the audio data comprises content-dependent metadata and content-independent metadata.

17. The system of claim 14 wherein the composite search score comprises an attribute with an associated score and or more associated time intervals.

18. The system of claim 14 wherein the composite search score may be determined in accordance with the following relationship:

19. The system of claim 18 wherein the sensitivity vectors are obtained from user feedback that indicates a correctness of the composite search score.

20. The system of claim 19 further comprising a self-learning module to process the sensitivity vectors based upon the user feedback.