Methods and apparatus for audio data analysis and data mining using speech recognition

• US 7,133,828 B2
• Filed: 10/20/2003
• Issued: 11/07/2006
• Est. Priority Date: 10/18/2002
• Status: Expired due to Term

• Current Assignee: RingCentral Incorporated
• Original Assignee: SER Solutions Incorporated (Atos SE)
• Inventors: Scarano, Robert, Mark, Lawrence
• Primary Examiner(s): HARPER, V PAUL
• Application Number:
  US10/687,703
• Publication Number:
  US 20040083099A1
• Time in Patent Office: 1,114 Days
• Field of Search: 379/265.06, 379/266.08, 379/309, 379/32.01, 379/88.01, 704/231, 704/240, 704/243, 704/245, 704/251, 704/275, 704/252
• US Class Current: 704/251
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2203/301 Management of recordings

Abstract

The present invention provides an audio analysis intelligence tool that provides ad-hoc search capabilities using spoken words as an organized data form. The present invention provides an SQL like interface to process and search audio data and combine it with other traditional data forms.

Claims

1. A method of searching audio data, comprising the step of:

defining a phrase to use for searching;

defining a minimum confidence level for searching;

defining search criteria containing said phrase and data values to use for searching;

searching a set of audio segments for said phrase and said data values; and

producing a set of results of all occurrences of the phrase within the set of audio segments matching the data criteria such that a given occurrence is a match for the search phrase and the confidence that a given occurrence is a match for the search phrase.

2. The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all occurrences of the plurality of phrases identified in a specified sequential order within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

3. The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all audio segments including (i) at least one occurrence of a selected required one of the plurality of phrases and (ii) non-occurrences of at least one selected forbidden one of said plurality of phrases to be excluded from within the audio segments, said occurrence and non-occurrence determined with respect to said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search, phrases.

4. The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

5. The method according to claim 1 wherein said step of defining includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all audio segments lacking occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

6. The method according to claim 5 wherein said temporal relationship is with respect to said phrases.

7. The method according to claim 5 wherein said temporal relationship is with respect to said audio segment.

8. The method according to claim 1 wherein said step of defining a phrase includes defining a plurality of phrases, said step of searching includes searching said set of audio segments for said plurality of phrases, and said step of producing includes producing a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

9. The method according to claim 8 wherein said temporal relationship is with respect to said phrases.

10. The method according to claim 8 wherein said temporal relationship is with respect to said audio segment.

11. The method according to claim 1 wherein the step of identifying said set of audio segments comprises a step of constraining said set of audio segments to ones of said audio segments selected for processing based on said intrinsic data prior to performing said searching step.

12. The method according to claim 1 wherein said intrinsic data comprises metadata.

13. The method according to claim 1 wherein said intrinsic data comprises Computer Telephony Integration (CTI) data.

14. The method according to claim 13 wherein said CTI data selected from the set consisting of (i) called number (DNIS) and, calling number (ANI), and (iii) Agent ID.

15. A method of operating contact center, comprising the step of:

connecting a plurality of calls to at least one customer service representative;

recording audio segments from each of said plurality of calls;

defining a phrase and intrinsic data to use for searching;

defining a minimum confidence level for searching;

searching said set of audio segments for said phrase and said intrinsic data; and

producing a set of results of all occurrences of the phrase within the audio segments and the confidence that a given occurrence is a match for the search phrase.

16. A system for searching audio data comprising:

control logic operable to (i) define a phrase to use for searching;

(ii) define a minimum confidence level for searching; and

a search engine operable to search a set of audio segments and intrinsic data associated therewith for (i) said phrase and (ii) ones of said audio segments satisfying match criteria for said intrinsic data and, in response, produce a set of results of all occurrences of the phrase within the set of audio segments satisfying said match criteria and the confidence that a given occurrence is a match for the search phrase.

17. The system according to claim 16 wherein said control logic is further operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and produce a set of results of all occurrences of the plurality of phrases identified in a specified sequential order within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

18. The system according to claim 16 wherein said control logic is further operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and said produce a set of results of all audio segments including (i) at least one occurrence of a selected required one of the plurality of phrases and (ii) non-occurrences of at least one selected forbidden one of said plurality of phrases to be excluded from within the audio segments, said occurrence and non-occurrence determined with respect to said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

19. The system according to claim 16 wherein said control logic is further operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and produce a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

20. The system according to claim 16 wherein said control logic is operable to define a plurality of phrases, said search engine further operable to search said set of audio segments for said plurality of phrases and produce a set of results of all audio segments lacking occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

21. The system according to claim 20 wherein said temporal relationship is with respect to said phrases.

22. The system according to claim 20 wherein said temporal relationship is with respect to said audio segment.

23. The system according to claim 16 wherein said control logic is operable to define a plurality of phrases, said search engine operable to search said set of audio segments for said plurality of phrases and produce a set of results of all occurrences of the plurality of phrases identified in a specified temporal relationship within the audio segments with said minimum confidence that a given occurrence within said audio segments is a match for a corresponding one of said plurality of search phrases.

24. The system according to claim 23 wherein said temporal relationship is with respect to said phrases.

25. The system according to claim 23 wherein said temporal relationship is with respect to said audio segment.

26. The system according to claim 16 wherein said processor is further operable to identify said set of audio segments based on metadata associated with said audio segments prior to said search engine operating to search said set of audio segments.

27. The system according to claim 16 wherein said processor is responsive to Computer Telephony Integration (CTI) data for identifying said set of audio segments.

28. The system according to claim 27 wherein said CTI data selected from the set consisting of (i) called number (DNIS), (ii) calling number (ANI), and (iii) Agent ID.

29. A contact center comprising:

a switch configured to connect each of a plurality of calls to a customer service representative workstation;

a memory connected to said switch and configured to record audio segments from each of said plurality of calls;

a supervisory terminal configured to (i) define a phrase to use for searching;

(ii) a minimum confidence level for searching; and

(iii) identify criteria used to select a set of said audio segments from ones of said plurality of calls based on intrinsic data associated with respective ones of said calls;

a search engine connected to said supervisory terminal and to said memory for searching said calls based on said intrinsic data and for said phrase; and

a display connected to said search engine and configured to produce a set of results of all occurrences of the phrase within the calls identified by said search engine based on said intrinsic data and the confidence that a given occurrence is a match for the search phrase.

30. A method for analyzing audio data, comprising:

storing audio segments in a speech repository;

storing information regarding each of the audio segments in a database;

establishing a search criteria including (i) speech and Structured Query Language (SQL) criteria for locating spoken words or phrases in said audio segment using speech recognition technology and (ii) information regarding each of the audio segments;

searching said set of audio segments and said database in accordance with said search criteria; and

providing a report based on said search.