Computer-implemented system and method for masking special data

• US 8,954,332 B2
• Filed: 11/04/2013
• Issued: 02/10/2015
• Est. Priority Date: 07/13/2005
• Status: Active Grant

• Current Assignee: Avaya LLC (Avaya Incorporated), Avaya Management L.P. (Avaya Incorporated)
• Original Assignee: Intellisist, Inc. dba Spoken Communications (Avaya Incorporated)
• Inventors: Lee, Howard M., Lutz, Steven, Odinak, Gilad
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Serrou, Abdelali
• Application Number:
  US14/071,639
• Publication Number:
  US 20140129219A1
• Time in Patent Office: 463 Days
• Field of Search: 704/273, 704/235, 704/248, 704/249, 704/251, 704/253, 704/254, 704/270, 704/270.1, 704/233
• US Class Current: 704/273
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: H04L 2209/04 Masking or blinding
• CPC Class Codes: H04L 2209/56 Financial cryptography, e.g. electronic payment or e-cash
• CPC Class Codes: H04L 9/3231 Biological data, e.g. fingerprint, voice or retina network architectures or network communication protocols for supporting authentication of entities using biometrical features in a packet data network H04L63/0861

Abstract

A computer-implemented system and method for masking special data is provided. Speakers of a call recording are identified. The call recording is separated into strands corresponding to each of the speakers. A prompt list of elements that prompt the speaker of the other strand to utter special information is applied to one of the strands. At least one of the elements of the prompt list is identified in the one strand. A special information candidate is identified in the other strand and is located after a location in time where the element was found in the voice recording of the one strand. A confidence score is assigned to the element located in the one strand and to the special information candidate in the other strand. The confidence scores are combined and a threshold is applied. The special information candidate is rendered unintelligible when the combined confidence scores satisfy the threshold.

Claims

1. A computer-implemented method for masking special data comprising:

identifying speakers of a voice recording;

separating the voice recording into strands corresponding to each of the speakers, wherein one strand is associated with one of the speakers and another strand is associated with a different speaker;

applying to one of the strands, a prompt list comprising elements that prompt a speaker associated with the other strand to utter special information;

identifying at least one of the elements of the prompt list in the one strand using automatic speech recognition;

identifying via the automatic speech recognition a special information candidate in the other strand directly after a location in time where the element was found in the voice recording of the one strand;

assigning a confidence score to the element located in the one strand and to the special information candidate in the other strand;

combining the confidence scores and applying a threshold to the combined confidence scores; and

rendering utterances corresponding with the special information candidate unintelligible on a special purpose digital recorder when the combined confidence scores satisfy the threshold.

2. A method according to claim 1, further comprising:

identifying endpoints within the voice recording;

partitioning the voice recording into segments based on the identified endpoints; and

translating each of the segments to text using speech recognition, wherein the prompt list is applied to the text of the segments.

3. A method according to claim 2, wherein the endpoints are determined based on pauses in the voice recording.

4. A method according to claim 1, further comprising:

providing a user interface;

displaying a translation of the utterances associated with the special information candidates within the user interface;

providing to a user, processing controls within the user interface; and

receiving instructions for rendering the utterances of the special information candidate unintelligible using the processing controls.

5. A method according to claim 1, further comprising:

identifying a further special information candidate, comprising;

identifying an estimated time period for an occurrence of the utterances in response to the element identified in the one strand;

applying the estimated time period to the voice recording of the other strand immediately after the identified element; and

identifying the utterances that occur within the estimated time period as corresponding to the special information candidate.

6. A method according to claim 1, further comprising:

applying markers to the voice recording as an indication that the voice recording includes the special information.

7. A method according to claim 1, further comprising:

determining the confidence score for the special information candidate, comprising;

determining an expected value for the special information candidate;

determining a transcribed value of the utterances associated with the special information candidate;

comparing the expected value with the transcribed value; and

providing a high confidence score to the special information candidate when the expected value matches the transcribed value and providing a low confidence score when the expected value does not match the transcribed value.

8. A method according to claim 1, further comprising:

reprocessing the voice recording, comprising;

identifying a problem with one or more utterances that were rendered unintelligible;

applying different processing parameters to the identified problematic utterances;

identifying a different special information candidate in the other strand directly after a location in time where the element was found in the voice recording of the one strand, wherein the different special information candidate comprises the utterances and further utterances or a portion of the utterances; and

rendering the different special information candidate unintellibigle.

9. A method according to claim 8, wherein the different processing parameters comprise at least one of different threshold values for detecting the element and the special information candidate, adjusted parameters for determining the confidence scores for the identification of special information, application of speaker identification, and endpoint analysis to the voice recording.

10. A method according to claim 1, wherein the rendering of the special information comprises at least one of:

recording a masking signal over at least a portion of the utterances associated with the special information candidate;

encrypting at least a portion of the utterances associated with the special information candidate; and

deleting at least a portion of the utterances associated with the special information candidate.

11. A computer-implemented system for masking special data comprising:

a processor; and

a memory embodying instructions that when executed by the processor cause the processor to perform the steps, comprising;

identifying speakers of a voice recording;

separating the voice recording into strands corresponding to each of the speakers, wherein one strand is associated with one of the speakers and another strand is associated with a different speaker;

applying to one of the strands, a prompt list comprising elements that prompt the speaker associated with the other strand to utter special information;

identifying at least one of the elements of the prompt list in the one strand using automatic speech recognition;

identifying via the automatic speech recognition a special information candidate in the other strand directly after a location in time where the element was found in the voice recording of the one strand;

assigning a confidence score to the element located in the one strand and to the special information candidate in the other strand;

combining the confidence scores and applying a threshold to the combined confidence scores; and

rendering on a special purpose digital recorder, utterances corresponding with the special information candidate unintelligible when the combined confidence scores satisfy the threshold.

12. A system according to claim 11, wherein the memory when executed by the processor causes the processor to further identify endpoints within the voice recording and to partition the voice recording into segments based on the identified endpoints and to translate each of the segments to text using speech recognition, wherein the prompt list is applied to the text of the segments.

13. A system according to claim 12, wherein the endpoints are determined based on pauses in the voice recording.

14. A system according to claim 11, wherein the memory when executed by the processor causes the processor to further display a translation of the utterances associated with the special information candidate within the user interface, to provide to a user, processing controls within the user interface, and to receive instructions for rendering the utterances of the special information candidate unintelligible using the processing controls.

15. A system according to claim 11, wherein the memory when executed by the processor causes the processor to identify a further special information candidate by identifying an estimated time period for an occurrence of the utterances in response to the element identified in the one strand, applying the estimated time period to the voice recording of the other strand immediately after the identified element, and identifying the utterances that occur within the estimated time period as corresponding to the special information candidate.

16. A system according to claim 11, wherein the memory when executed by the processor causes the processor to further apply markers to the voice recording as an indication that the voice recording includes the special information.

17. A system according to claim 11, wherein the memory when executed by the processor causes the processor to further determine the confidence score for the special information candidate by determining an expected value for the special information candidate and a transcribed value of the utterances associated with the special information candidate, comparing the expected value with the transcribed value, and providing a high confidence score to the special information candidate when the expected value matches the transcribed value and providing a low confidence score when the expected value does not match the transcribed value.

18. A system according to claim 11, wherein the memory when executed by the processor causes the processor to further reprocess the voice recording by identifying a problem with one or more utterances that were rendered unintelligible, applying different processing parameters to the identified problematic utterances, identifying a different special information candidate in the other strand directly after a location in time where the element was found in the voice recording of the one strand, wherein the different special information candidate comprises the utterances and further utterances or a portion of the utterances, and rendering the different special information candidate unintellibigle.

19. A system according to claim 18, wherein the different processing parameters comprise at least one of different threshold values for detecting the element and the special information candidate, adjusted parameters for determining the confidence scores for the identification of special information, application of speaker identification, and endpoint analysis to the voice recording.

20. A system according to claim 11, wherein the rendering of the special information comprises at least one of recording a masking signal over at least a portion of the utterances associated with the special information candidate, encrypting at least a portion of the utterances associated with the special information candidate, and deleting at least a portion of the utterances associated with the special information candidate.