Selective security masking within recorded speech utilizing speech recognition techniques

• US 8,577,684 B2
• Filed: 07/13/2005
• Issued: 11/05/2013
• Est. Priority Date: 07/13/2005
• Status: Active Grant

• Current Assignee: Avaya LLC (Avaya Incorporated), Avaya Management L.P. (Avaya Incorporated)
• Original Assignee: Intellisist, Inc. dba Spoken Communications (Avaya Incorporated)
• Inventors: Lee, Howard M., Lutz, Steven, Odinak, Gilad
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Serrou, Abdelali
• Application Number:
  US11/181,572
• Publication Number:
  US 20070016419A1
• Time in Patent Office: 3,037 Days
• Field of Search: 704/273, 704/233, 704/270, 704/270.1, 704/251, 704/253
• US Class Current: 704/273
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: H04L 2209/04 Masking or blinding
• CPC Class Codes: H04L 2209/56 Financial cryptography, e.g. electronic payment or e-cash
• CPC Class Codes: H04L 9/3231 Biological data, e.g. fingerprint, voice or retina network architectures or network communication protocols for supporting authentication of entities using biometrical features in a packet data network H04L63/0861

Abstract

Apparatuses and methods are described to secure information contained within voice recordings. A voice recording is loaded into a data processing system, wherein the voice recording results from an utterance of a human during an interaction between the human and an interface of a business entity. The voice recording is processed to recognize at least one element of text in the voice recording. The data processing system determines if the at least one element of text represents special information that pertains to the human. A segment in the voice recording is rendered unintelligible if the at least one element of text represents special information that pertains to the human.

Claims

1. A method comprising:

loading a voice recording into a data processing system, wherein the voice recording comprises utterances from an interaction between a user and an interface of a business entity;

separating the voice recording into a business entity strand associated with the interface and a user strand associated with the user;

identifying prompt list candidates in the business entity strand, wherein each prompt list candidate comprises an utterance by the interface;

obtaining a prompt list comprising elements that represent requests for special information from the user and comparing the prompt list with the prompt list candidates in the business entity strand;

assigning a score to one or more of the prompt list candidates based on an element in the prompt list and applying a threshold to the scores of the one or more prompt list candidates;

establishing a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

searching the voice recording of the user strand for an utterance by the user that represents the special information, directly after a location in time where the element of the prompt list was found in the voice recording of the business entity strand;

assigning a score to the utterance by the user;

verifying that the utterance by the user corresponds with the special information, comprising;

determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the utterance by the user in the user strand; and

identifying the utterance by the user as the special information when the combined score exceeds a further threshold; and

rendering the special information of the voice recording unintelligible.

2. The method of claim 1, wherein the special information is selected from the group consisting of confidential information, sensitive information, and private information.

3. The method of claim 1, wherein the special information is selected from the group consisting of a social security number, a personal identification number (PIN), a financial account number, a name, and an address.

4. The method of claim 1, wherein the special information is defined by the user.

5. The method of claim 1, wherein the rendering encrypts the special information.

6. The method of claim 1, wherein the rendering masks the special information.

7. The method of claim 1, wherein the rendering deletes the special information.

8. The method of claim 1, wherein the interface of the business entity is a live person.

9. The method of claim 1, wherein the interface of the business entity is an automated process.

10. The method of claim 1, further comprising:

performing under a contractual relationship, wherein the contractual relationship has terms wherein a data custodian receives value from the business entity in exchange for at least one of the loading, the processing, or the rendering.

11. The method of claim 1, wherein the voice recording is stored in an analog form.

12. The method of claim 1, wherein the voice recording is stored in a digital form.

13. The method of claim 1, wherein the rendering is performed on a textual representation of the voice recording.

14. The method of claim 1, further comprising:

associating an identifier with the voice recording, wherein the identifier is based on the processing and the identifier communicates information pertaining to the special information that is associated with the user.

47. The method of claim 1, wherein a length of the unintelligible voice recording is chosen based on an expected length of time for the user to speak an item of the special information.

48. The method of claim 47, wherein the length is ten seconds when the meaning of the element of the prompt list is a credit card number.

15. A method comprising:

loading a voice recording into a data processing system, wherein the voice recording has been processed to render unintelligible at least one segment that contains special information that pertains to a user and the voice recording results from utterances during an interaction between the user and an interface of a business entity; and

reprocessing the voice recording to adjust a location of the at least one segment that is rendered unintelligible, comprising;

rendering the at least one segment of the voice recording intelligible;

separating the voice recording into a business entity strand associated with the interface and a user strand associated with the user;

identifying prompt list candidates in the business entity strand, wherein each prompt list candidate comprises an utterance by the interface;

obtaining a prompt list comprising elements that represent requests for special information from the user and comparing the prompt list with the prompt list candidates in the business entity strand;

assigning a score to one or more of the prompt list candidates based on an element in the prompt list and applying a threshold to the scores of the one or more prompt list candidates;

establishing a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

searching the voice recording of the user strand for an utterance by the user that represents the special information, directly after a location in time where the element of the prompt list was found in the voice recording of the business entity strand;

assigning a score to the utterance by the user;

verifying that the utterance by the user corresponds with the special information, comprising;

determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the utterance by the user in the user strand; and

identifying the utterance by the user as the special information when the combined score exceeds a further threshold; and

rendering the special information of the voice recording unintelligible.

16. The method of claim 15, wherein a key is used to perform decryption during the rendering of the voice recording as intelligible.

17. The method of claim 15, wherein a key is used to perform encryption during the rendering of the special information of the voice recording as unintelligible.

18. The method of claim 15, wherein the interface of the business entity is a live person.

19. The method of claim 15, wherein the interface of the business entity is an automated process.

20. An apparatus comprising:

an information display;

a storage device; and

a processor programmed to;

read a voice recording from the storage device, wherein the voice recording results from utterances during an interaction between a user and an interface of a business entity;

separate the voice recording into a business entity strand associated with the interface and a user strand associated with the user;

identify prompt list candidates in the business entity strand;

obtain a prompt list comprising elements that represent requests for special information from the user and compare the prompt list with the prompt list candidates in the business entity strand;

assign a score to one or more of the prompt list candidates based on an element in the prompt list and apply a threshold to the scores of the one or more prompt list candidates;

establish a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

search the voice recording of the user strand for one or more utterances made by the user, directly after a location in time where the element of the prompt list was found in the voice recording of the business entity strand;

determine if the one or more utterances represent special information that pertains to the user by assigning a score to at least one of the one or more utterances by the user;

verify that the utterance by the user corresponds with the special information comprising determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the at least one utterance by the user in the user strand and identifying the at least one utterance by the user as the special information when the combined score exceeds a further threshold;

display the at least one utterance on the information display; and

accept a command from the user, wherein the command initiates a process on the at least one utterance.

21. The apparatus of claim 20, wherein the command deletes a segment in the voice recording that corresponds to the at least one utterance.

22. The apparatus of claim 20, wherein the command masks a segment in the voice recording corresponding to the at least one utterance.

23. The apparatus of claim 20, wherein the command encrypts a segment in the voice recording corresponding to the at least one utterance.

24. The apparatus of claim 23, wherein the command decrypts a segment in the voice recording corresponding to the at least one utterance.

25. The apparatus of claim 20, further comprising:

a data custodian, the data custodian having a first contractual relationship with the business entity, the first contractual relationship having terms wherein the business entity is to deliver value to the data custodian in response to operation of the processor programmed as in claim 20.

26. A method comprising:

receiving a voice signal from a user;

receiving a voice signal from an interface of a business entity, wherein the voice signal from the user is responsive to the voice signal from the business entity;

overlaying markers with one of the voice signal from the user or the voice signal from the interface of the business entity;

identifying prompt list candidates in the voice signal from the business entity interface, wherein each prompt list candidate comprises an utterance by the interface;

obtaining a prompt list comprising elements that represent requests for special information from the user based on the markers and comparing the prompt list with the prompt list candidates in the voice signal from the business entity interface;

assigning a score to one or more of the prompt list candidates based on an element in the prompt list and applying a threshold to the scores of the one or more prompt list candidates;

establishing a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

searching the voice signal from the user for an utterance by the user that represents the special information located directly after a location in time where the element of the prompt list was found in the voice signal of the business entity interface;

assigning a score to the utterance by the user;

verifying that the utterance by the user corresponds with the special information, comprising;

determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the utterance by the user in the user strand; and

identifying the utterance by the user as the special information when the combined score exceeds a further threshold; and

rendering the special information of the voice signal from the user unintelligible.

27. The method of claim 26, wherein a tone is used for the markers.

28. The method of claim 26, wherein the special information is selected from the group consisting of confidential information, sensitive information, and private information.

29. The method of claim 26, wherein the special information is selected from the group consisting of a social security number, a personal identification number (PIN), a financial account number, a name, and an address.

30. The method of claim 26, wherein the special information is defined by the user.

31. The method of claim 26, further comprising:

performing under a contractual relationship, wherein the contractual relationship has terms wherein a data custodian receives value from the business entity in exchange for at least one of the receiving the voice signal from the user, the receiving the voice signal from an interface of the business entity, the overlaying, or the rendering.

32. A non-transitory computer readable medium containing executable computer program instructions, which when executed by a data processing system, cause the data processing system to perform a method comprising:

loading a voice recording into a data processing system, wherein the voice recording comprises utterances from an interaction between a user and an interface of a business entity;

separating the voice recording into a business entity strand associated with the interface and a user strand associated with the user;

identifying prompt list candidates in the business entity strand, wherein each prompt list candidate comprises an utterance by the interface;

obtaining a prompt list comprising elements that represent requests for special information from the user and comparing the prompt list with the prompt list candidates in the business entity strand;

assigning a score to one or more of the prompt list candidates based on an element in the prompt list and applying a threshold to the scores of the one or more prompt list candidates;

establishing a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

searching the voice recording of the user strand for an utterance by the user that represents the special information, directly after a location in time where the element of the prompt list was found in the voice recording of the business entity strand;

assigning a score to the utterance by the user;

verifying that the utterance by the user corresponds with the special information, comprising;

determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the at least one utterance by the user in the user strand; and

identifying the utterance by the user as the special information when the combined score exceeds a further threshold; and

rendering the special information of the voice recording unintelligible.

33. The non-transitory computer readable medium, as set forth in claim 32, wherein the special information is selected from the group consisting of confidential information, sensitive information, and private information.

34. The non-transitory computer readable medium, as set forth in claim 32, wherein the special information is selected from the group consisting of a social security number, a personal identification number (PIN), a financial account number, a name, and an address.

35. The non-transitory computer readable medium, as set forth in claim 32, wherein the special information is defined by the user.

36. The non-transitory computer readable medium, as set forth in claim 32, wherein the rendering encrypts the special information.

37. The non-transitory computer readable medium, as set forth in claim 32, wherein the rendering masks the special information.

38. The non-transitory computer readable medium, as set forth in claim 32, wherein the interface of the business entity is a live person.

39. The non-transitory computer readable medium, as set forth in claim 32, wherein the interface of the business entity is an automated process.

40. The non-transitory computer readable medium, as set forth in claim 32, wherein subsequent to the rendering, the voice recording is stored in a form selected from the group consisting of analog, digital, and text.

41. An apparatus comprising:

means for storing a voice recording, wherein the voice recording comprises utterances from an interaction between a user and an interface of a business entity;

means for separating the voice recording into a business entity strand associated with the interface and a user strand associated with the user;

means for identifying prompt list candidates in the business entity strand, wherein each prompt list candidate comprises an utterance by the interface;

means for obtaining a prompt list comprising elements that represent requests for special information from the user and means for comparing the prompt list with the prompt list candidates in the business entity strand;

means for assigning a score to one or more of the prompt list candidates based on an element in the prompt list and for applying a threshold to the scores of the one or more prompt list candidates;

means for establishing a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

means for searching the voice recording of the user strand for one or more utterances made by the user, directly after a location in time where the element of the prompt list was found in the voice recording business entity strand;

means for determining if the one or more utterances correspond with the special information associated with the user by assigning a score to at least one of the one or more utterances by the user, by determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the at least one utterance by the user in the user strand, and by identifying the at least one utterance by the user as the special information when the combined score exceeds a further threshold; and

means for rendering the special information in the voice recording, corresponding to the at least one utterance by the user, unintelligible.

42. The apparatus of claim 41, further comprising:

associating an identifier with the voice recording.

43. A method comprising:

loading a voice recording into a data processing system, wherein the voice recording results from utterances during an interaction between a user and an interface of a business entity;

separating the voice recording into a business entity strand associated with the interface and a user strand associated with the user;

identifying prompt list candidates in the business entity strand, wherein each prompt list candidate comprises an utterance by the interface;

obtaining a prompt list comprising elements that represent requests for special information from the user and comparing the prompt list with the prompt list candidates in the business entity strand;

assigning a score to one or more of the prompt list candidates based on an element in the prompt list and applying a threshold to the scores of the one or more prompt list candidates;

establishing a match between at least one of the prompt list candidates and the element of the prompt list when the value for the one prompt list candidate exceeds the threshold;

searching the voice recording of the user strand for an utterance by the user that represents the special information, directly after a location in time where the element of the prompt list was found in the voice recording of the business entity strand;

assigning a score to the utterance by the user;

verifying that the utterance by the user corresponds with the special information, comprising;

determining a combined score based on the utterance of the user and the utterance of the business entity by combining the score for the prompt list candidate in the business entity strand that matches the element of the prompt list and the score for the utterance by the user in the user strand;

identifying the utterance by the user as the special information when the combined score exceeds a further threshold; and

associating an identifier with the voice recording to indicate a presence of the special information.

44. The method of claim 43, wherein the voice recording is in a form selected from the group consisting of analog, digital, and text.

45. The method of claim 44, wherein the identifier further indicates at least one of an absence of the special information, encryption, masking, data owner, data custodian, and data analyst.

46. The method of claim 43, wherein the associating associates the identifier with the voice recording and the voice recording is in a form selected from the group consisting of analog and digital.