Dynamic protection of personal information in audio recordings

• US 10,468,026 B1
• Filed: 08/17/2018
• Issued: 11/05/2019
• Est. Priority Date: 08/17/2018
• Status: Active Grant

• Current Assignee: Century Interactive Company, LLC
• Original Assignee: Century Interactive Company, LLC
• Inventors: Newman, Zachary Ward, Wakefield, Reid Samuel, Brunelli, Mitchell John
• Primary Examiner(s): Ortiz-Sanchez, Michael
• Application Number:
  US16/104,702
• Time in Patent Office: 445 Days
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04M 2203/301 Management of recordings
• CPC Class Codes: H04M 2203/303 Marking
• CPC Class Codes: H04M 2203/6009 Personal information, e.g. profiles or personal directories being only provided to authorised persons
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656

Abstract

In a general aspect, audio recordings are managed to protect personal identifiable information (PII) and or sensitive personal information (SPI). In an aspect, a first audio file associated with a voice call and a second audio file associated with the voice call are obtained. The first audio file is partitioned into one or more segments. Which of the one or more segments of the first audio file the number of spoken digits exceeds a certain number are identified. The second audio file is partitioned into segments corresponding in time to the identified segments of the first audio file. The voice call is tagged as containing PII or SPI in response to determining that trigger words are spoken in at least one of the segments of the second audio file.

Claims

1. A method to protect personal identifiable information (PII) or sensitive personal information (SPI) in audio recordings, comprising:

obtaining a first audio file associated with a first channel of a voice call;

obtaining a second audio file associated with a second channel of the voice call;

partitioning, by one or more processors, the first audio file into one or more segments comprising spoken digits, each segment comprising a start time and an end time, by;

detecting a spoken digit in the first audio file at a timepoint;

determining whether a subsequent spoken digit occurred within a certain amount of time of a previous spoken digit; and

setting the end time of the segment when the certain amount of time between subsequent digits is exceeded with no digits being detected;

identifying in which of the one or more segments of the first audio file the number of spoken digits exceeds a certain number;

partitioning, by the one or more processors, the second audio file into segments corresponding in time to the identified segments of the first audio file;

determining whether to tag the voice call as containing PII or SPI in response to determining whether one or more trigger words are spoken in at least one of the corresponding segments of the second audio file;

partitioning, by one or more processors, the second audio file into one or more segments comprising spoken digits, each segment comprising a start time and an end time, by;

detecting a spoken digit in the second audio file at a timepoint;

determining whether a subsequent spoken digit occurred within a certain amount of time of a previous spoken digit; and

setting the end time of the segment when the certain amount of time between subsequent digits is exceeded with no digits being detected;

identifying one or more segments of the second audio file in which the number of spoken digits exceeds a certain number;

partitioning the first audio file into one or more segments corresponding in time to the one or more identified segments in the second audio file;

determining whether to tag the voice call as containing PII or SPI in response to determining whether one or more trigger words are spoken in at least one of the one or more corresponding segments of the first audio file.

2. The method of claim 1, further comprising:

restricting access to the first and second audio files associated with the tagged voice call.

3. The method of claim 2, wherein restricting access to the first and second audio files associated with the tagged voice call comprises:

rejecting requests to obtain the first or second audio file associated with the voice call from the server.

4. The method of claim 1, further comprising:

deleting one or more portions of the second audio file corresponding in time to the at least one of the segments of the second audio file in which one or more trigger words are determined to be spoken and storing the second audio file as an edited second audio file; and

deleting one or more portions of the first audio file corresponding in time to the deleted one or more portions of the second audio and storing the first audio file as an edited first audio file.

5. The method of claim 1, wherein the first and second audio files associated with the voice call are obtained in response to receiving a notification that a voice call is available for analysis.

6. The method of claim 1, wherein the start time is a number of seconds prior to the timepoint.

7. The method of claim 6, wherein the number of seconds is twenty seconds.

8. The method of claim 1, wherein the certain number of identified digits exceeded in a segment is four.

9. The node device of claim 1, wherein the start time is a number of seconds prior to the timepoint.

10. The node device of claim 9, wherein the number of seconds is twenty seconds.

11. A node device, for protecting personal identifiable information (PII) or sensitive personal information (SPI) in audio recordings, comprising one or more processors and memory storing instructions that when executed by the one or more processors cause the node device to:

obtain a first audio file associated with a first channel of a voice call;

obtain a second audio file associated with a second channel of the voice call;

partition the first audio file into one or more segments comprising spoken digits, each segment comprising a start time and an end time, by;

detecting a spoken digit in the first audio file at a timepoint;

determining whether a subsequent spoken digit occurred within a certain amount of time of a previous spoken digit;

and setting the end time of the segment when the certain amount of time between subsequent digits is exceeded with no digits being detected;

identify in which of the one or more segments of the first audio file the number of spoken digits exceeds a certain number;

partition the second audio file into segments corresponding in time to the identified segments of the first audio file;

tag the voice call as containing PII or SPI in response to determining that one or more trigger words are spoken in at least one of the corresponding segments of the second audio file;

partition the second audio file into one or more segments comprising spoken digits, each segment comprising a start time and an end time, by;

detecting a spoken digit in the second audio file at a timepoint;

determining whether a subsequent spoken digit occurred within a certain amount of time of a previous spoken digit; and

setting the end time of the segment when the certain amount of time between subsequent digits is exceeded with no digits being detected;

identify one or more segments of the second audio file in which the number of spoken digits exceeds a certain number;

partition the first audio file into one or more segments corresponding in time to the one or more identified segments in the second audio file;

tag the voice call as containing PII or SPI in response to determining that one or more trigger words are spoken in at least one of the one or more corresponding segments of the first audio file.

12. The node device of claim 11, the memory comprising instructions that when executed by the one or more processors cause the node device to:

restrict access to the first and second audio files associated with the tagged voice call.

13. The node device of claim 12, wherein restricting access to the first and second audio files associated with the tagged voice call comprises:

rejecting requests to obtain the first or second audio file associated with the voice call.

14. The node device of claim 11, the memory comprising instructions that when executed by the one or more processors cause the node device to:

delete one or more portions of the second audio file corresponding in time to the at least one of the segments of the second audio file in which one or more trigger words are determined to be spoken and storing the second audio file as an edited second audio file; and

delete one or more portions of the first audio file corresponding in time to the deleted one or more portions of the second audio and storing the first audio file as an edited first audio file.

15. The node device of claim 11, wherein the first and second audio files associated with the voice call are obtained in response to receiving a notification that a voice call is available for analysis.

16. The node device of claim 11, wherein the certain number of identified digits exceeded in a segment is four.

17. A non-transitory computer-readable medium for protecting personal identifiable information (PII) or sensitive personal information (SPI) in audio recordings,

comprising instructions that when executed by one or more processors of a node device are operable to cause the node device to:

obtain a first audio file associated with a first channel of a voice call;

obtain a second audio file associated with a second channel of the voice call;

partition the first audio file into one or more segments comprising spoken digits, each segment comprising a start time and an end time, by;

detecting a spoken digit in the first audio file at a timepoint;

determining whether a subsequent spoken digit occurred within a certain amount of time of a previous spoken digit;

and setting the end time of the segment when the certain amount of time between subsequent digits is exceeded with no digits being detected;

identify in which of the one or more segments of the first audio file the number of spoken digits exceeds a certain number;

partition the second audio file into segments corresponding in time to the identified segments of the first audio file;

and tag the voice call as containing PII or SPI in response to determining that one or more trigger words are spoken in at least one of the corresponding segments of the second audio file;

partition the second audio file into one or more segments comprising spoken digits, each segment comprising a start time and an end time, by;

detecting a spoken digit in the second audio file at a timepoint;

determining whether a subsequent spoken digit occurred within a certain amount of time of a previous spoken digit;

and setting the end time of the segment when the certain amount of time between subsequent digits is exceeded with no digits being detected;

identify one or more segments of the second audio file in which the number of spoken digits exceeds a certain number;

partition the first audio file into one or more segments corresponding in time to the one or more identified segments in the second audio file;

tag the voice call as containing PII or SPI in response to determining that one or more trigger words are spoken in at least one of the one or more corresponding segments of the first audio file.

18. The non-transitory computer readable medium of claim 17, further comprising instructions to:

restrict access to the first and second audio files associated with the tagged voice call.

19. The non-transitory computer readable medium of claim 18, wherein restricting access to the first and second audio files associated with the tagged voice call comprises:

rejecting requests to obtain the first or second audio file associated with the voice call.

20. The non-transitory computer readable medium of claim 17, further comprising instructions to:

delete one or more portions of the second audio file corresponding in time to the at least one of the segments of the second audio file in which one or more trigger words are determined to be spoken and storing the second audio file as an edited second audio file; and

delete one or more portions of the first audio file corresponding in time to the deleted one or more portions of the second audio and storing the first audio file as an edited first audio file.

21. The non-transitory computer readable medium of claim 17,

wherein the first and second audio files associated with the voice call are obtained in response to receiving a notification that a voice call is available for analysis.

22. The non-transitory computer readable medium of claim 17, wherein the start time is a number of seconds prior to the timepoint.

23. The non-transitory computer readable medium of claim 22, wherein the number of seconds is twenty seconds.