Compliance management of recorded audio using speech analytics

• US 9,544,438 B1
• Filed: 06/18/2015
• Issued: 01/10/2017
• Est. Priority Date: 06/18/2015
• Status: Active Grant

• Current Assignee: Alvaria Cayman
• Original Assignee: Noble Systems Corporation
• Inventors: Andraszek, Joseph, Koster, Karl H.
• Primary Examiner(s): Elahee, Md S
• Assistant Examiner(s): Pope, Kharye
• Application Number:
  US14/743,200
• Time in Patent Office: 572 Days
• Field of Search: 379/265.09, 379/88.01, 379/142.05, 379/265.02, 379/32.01, 370/352, 370/356, 370/351, 704/275, 704/231, 704/251, 704/257
• US Class Current: 1/1
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04M 2203/303 Marking
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/5183 Call or contact centers with computer-telephony arrangements
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4542 Blocking scenes or portions of the received content, e.g. censoring scenes
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

Various embodiments of the invention provide methods, systems, and computer-program products for identifying an occurrence during a conversation being conducted between a first party and a second party where sensitive information is conveyed by the first party or the second party. In particular embodiments, speech analytics is used to monitor audio of the conversation and a first speech characteristic is detected indicating a start event identifying a start of an occurrence where sensitive information is conveyed by the first party or the second party. Following the detection of first speech characteristic, a second speech characteristic is detected in the audio indicating an end event identifying an end of the occurrence. Accordingly, identifiers for the start and end events are then recorded in various embodiments as metadata along with a recording of the audio of the conversation to identify where in the recording the occurrence takes place.

Claims

1. A system for identifying an occurrence during a conversation being conducted between a first party and a second party where sensitive information is conveyed by at least one of the first party and the second party, the system comprising:

a speech analytics component configured to;

monitor audio of the conversation for at least one of the first party and the second party;

detect a first speech characteristic in the audio of the conversation for at least one of the first party and the second party indicating a start event identifying a start of the occurrence where the sensitive information is conveyed by at least one of the first party and the second party;

detect a second speech characteristic in the audio of the conversation for at least one of the first party and the second party indicating an end event identifying an end of the occurrence where the sensitive information is conveyed by at least one of the first party and the second party;

record a first identifier for the start event and a second identifier for the end event as metadata along with a recording of the audio of the conversation for at least one of the first party and the second party to identify where in the recording the occurrence takes place;

detect a third speech characteristic in the audio of the conversation for at least one of the first party and the second party indicating a second start event identifying a start of a second occurrence where sensitive information is conveyed by at least one of the first party and the second party; and

based on a confidence score associated with detecting the third speech characteristic being less than a threshold value and greater than a review score, mark the recording of the audio for review beginning at the second start event; and

an audio playback component configured to;

read the first identifier and the second identifier from the metadata;

play the recording of the audio for an individual to the start event based on the first identifier;

after playing the recording of the audio to the start event, apply a mask to the recording of the audio from the start event to the end event based on the second identifier without altering the recording so that the occurrence of sensitive information is permanently removed and so that the individual listening to the recording of the audio is unable to comprehend the sensitive information conveyed during the occurrence; and

after applying the mask to the recording of the audio from the start event to the end event, play the recording of the audio for the individual from the end event to an end of the recording of the audio.

2. The system of claim 1, wherein the speech analytics component is configured to detect the first speech characteristic in the audio of the conversation for at least one of the first party and the second party based on a confidence score associated with detecting the first speech characteristic being greater than a threshold value.

3. The system of claim 1, wherein the conversation comprises a telephone call, the first party comprises an agent at a contact center, and the second party comprises a party remote from the contact center.

4. A non-transitory, computer-readable medium comprising computer-executable instructions for identifying an occurrence during a conversation being conducted between a first party and a second party where sensitive information is conveyed by at least one of the first party and the second party, the computer-executable instructions, when executed by at least one computer processor, cause the at least one computer processor to:

monitor audio of the conversation for at least one of the first party and the second party;

detect a first speech characteristic in the audio of the conversation for at least one of the first party and the second party indicating a start event identifying a start of the occurrence where the sensitive information is conveyed by at least one of the first party and the second party;

detect a second speech characteristic in the audio of the conversation for at least one of the first party and the second party indicating an end event identifying an end of the occurrence where the sensitive information is conveyed by at least one of the first party and the second party;

record a first identifier for the start event and a second identifier for the end event as metadata along with a recording of the audio of the conversation for at least one of the first party and the second party to identify where in the recording the occurrence takes place;

detect a third speech characteristic in the audio of the conversation for at least one of the first party and the second party indicating a second start event identifying a start of a second occurrence where sensitive information is conveyed by at least one of the first party and the second party;

based on a confidence score associated with detecting the third speech characteristic being less than a threshold value and greater than a review score, mark the recording of the audio for review beginning at the second start event;

read the first identifier and the second identifier from the metadata;

play the recording of the audio for an individual to the start event based on the first identifier;

after playing the recording of the audio to the start event, apply a mask to the recording of the audio from the start event to the end event based on the second identifier without altering the recording so that the occurrence of sensitive information is permanently removed and so that the individual listening to the recording of the audio is unable to comprehend the sensitive information conveyed during the occurrence; and

after applying the mask to the recording of the audio from the start event to the end event, play the recording of the audio for the individual from the end event to an end of the recording of the audio.

5. The non-transitory, computer-readable medium of claim 4, wherein the computer-executable instructions, when executed by the at least one computer processor, cause the at least one computer processor to detect the first speech characteristic in the audio of the conversation for at least one of the first party and the second party based on a confidence score associated with detecting the first speech characteristic being greater than a threshold value.

6. The non-transitory, computer-readable medium of claim 4, wherein the conversation comprises a telephone call, the first party comprises an agent at a contact center, and the second party comprises a party remote from the contact center.

7. A method for identifying an occurrence during a conversation being conducted between a first party and a second party where sensitive information is conveyed by at least one of the first party and the second party, the method comprising the steps of:

monitoring audio of the conversation for at least one of the first party and the second party by a speech analytics system;

detecting a first speech characteristic in the audio of the conversation for at least one of the first party and the second party by the speech analytics system, the first speech characteristic indicating a start event identifying a start of the occurrence where the sensitive information is conveyed by at least one of the first party and the second party;

detecting a second speech characteristic in the audio of the conversation for at least one of the first party and the second party by the speech analytics system, the second speech characteristic indicating an end event identifying an end of the occurrence where the sensitive information is conveyed by at least one of the first party and the second party;

recording a first identifier for the start event and a second identifier for the end event by the speech analytics system as metadata along with a recording of the audio of the conversation for at least one of the first party and the second party to identify where in the recording the occurrence takes place;

detecting a third speech characteristic in the audio of the conversation for at least one of the first party and the second party by the speech analytics system, the third speech characteristic indicating a second start event identifying a start of a second occurrence where sensitive information is conveyed by at least one of the first party and the second party; and

based on a confidence score associated with detecting the third speech characteristic being less than a threshold value and greater than a review score, marking the recording of the audio for review beginning at the second start event by the speech analytics system;

reading the first identifier and the second identifier from the metadata by an audio playback system;

playing the recording of the audio for an individual to the start event based on the first identifier by the audio playback system;

after playing the recording of the audio to the start event, applying a mask to the recording of the audio from the start event to the end event based on the second identifier by the audio playback system without altering the recording so that the occurrence of sensitive information is permanently removed and so that the individual listening to the recording of the audio is unable to comprehend the sensitive information conveyed during the occurrence; and

after applying the mask to the recording of the audio from the start event to the end event, playing the recording of the audio for the individual from the end event to an end of the recording of the audio by the audio playback system.

8. The method of claim 7, wherein the step of detecting the first speech characteristic in the audio of the conversation for at least one of the first party and the second party is carried out by the speech analytics system based on a confidence score associated with detecting the first speech characteristic being greater than a threshold value.

9. The method of claim 7, wherein the conversation comprises a telephone call, the first party comprises an agent at a contact center, and the second party comprises a party remote from the contact center.