Identical conversation detection method and apparatus

• US 10,033,857 B2
• Filed: 01/26/2015
• Issued: 07/24/2018
• Est. Priority Date: 04/01/2014
• Status: Active Grant

• Current Assignee: Securus Technologies Incorporated (Securus Special Investments LLC)
• Original Assignee: Securus Technologies Incorporated (Securus Special Investments LLC)
• Inventors: Gainsboro, Jay, Weinstein, Lee
• Primary Examiner(s): BEZUAYEHU, SOLOMON G
• Application Number:
  US14/605,889
• Publication Number:
  US 20150281433A1
• Time in Patent Office: 1,275 Days
• Field of Search: 379 8801- 8819, 4554122, 4554561
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04M 3/2281 Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656

Abstract

An automated system is disclosed for detecting situations in which identical segments of conversation appear within two different recordings. The system automatically detects where within each of two audio recordings an identical conversation segment begins and ends, thus enabling detection of conversations where multiple prison inmates participated in a conference call.

Claims

1. A method of detecting a disallowed communication session between a first person using a first communication device and a second person using a second communication device, comprising:

recording first audio captured at the first communication device to produce a first recording;

recording second audio captured at the second communication device to produce a second recording;

generating a first signature of the first recording, wherein the first signature identifies one or more speech formants in the first audio;

generating a second signature of the second recording, wherein the second signature identifies one or more speech formants in the in the second audio;

identifying an overlapping timespan in the first recording and the second recording;

correlating a portion of the first signature generated from a portion of the first audio recorded during the overlapping timespan and a portion of the second signature generated from a portion of the second audio recorded during the overlapping timespan to determine whether a portion of the overlapping timespan of the first recording and the second recording are correlated greater than a predetermined threshold;

signaling the detection of the disallowed communication session between the first person using the first communication device and the second person using the second communication device, wherein the correlated portion of the overlapping timespan corresponds to a disallowed conversation between the first person and the second person; and

displaying, on a computer display, a notification of the detected disallowed communication session in response to the signaled detection.

2. The method of claim 1, wherein the first signature comprises spectral time information that includes one or more spectral peaks corresponding to the one or more speech formants in the first recording and the second signature comprises spectral time information that includes one or more spectral peaks corresponding to the one or more speech formants in the second recording.

3. The method of claim 2, wherein the amplitudes of the spectral peaks of the first signature are greater than the amplitudes of the spectral peaks of the second signature.

4. The method of claim 2, wherein correlating the first signature and the second signature comprises comparing the one or more spectral peaks in the first signature against the one or more spectral peaks in the second signature.

5. The method of claim 2, further comprising:

determining the overlapping timespan portion of the first audio and the second audio based on the correlation of the spectral peaks in the first signature and the spectral peaks in the second signature.

11. The method of claim 1, wherein the first signature is generated by calculating a first Fourier transform of one or more segments of the first recording, and wherein the second signature is generated by calculating a second Fourier transform of one or more segments of the second recording.

12. The method of claim 1, wherein the first signature is generated by calculating an autocorrelation of one or more segments of the first recording.

15. The method of claim 11, wherein the first Fourier transform has a first frequency resolution and the second Fourier transform has a second frequency resolution.

16. The method of claim 15, further comprising adjusting the first frequency resolution of the first Fourier transform;

generating an updated first signature by calculating the first Fourier transform of the one or more segments of the first recording using the adjusted first frequency resolution; and

correlating a portion of the updated first signature generated from a portion of the first audio recorded during the overlapping timespan and a portion of the second signature generated from a portion of the second audio recorded during the overlapping timespan, wherein the spectral peaks in the updated first signature are compared against the spectral peaks in the second signature.

17. The method of claim 16, wherein the first frequency resolution is adjusted to identify, within the updated first signature, a portion of the one or more formants that have been identified in the second signature.

6. A system for detecting a disallowed communication session between a first person using a first communication device and a second person using a second communication device, the comprising:

one or more processors; and

a memory device coupled to the one or more processors, the memory device storing computer-readable instructions that, upon execution by the one or more processors, cause the system to;

record first audio captured at the first communication device to produce a first recording;

record second audio captured at the second communication device to produce a second recording;

generate a first signature of the first recording, wherein the first signature identifies one or more speech formants in the in the first audio

generate a second signature of the second recording, wherein the second signature identifies one or more speech formants in the second audio;

identify an overlapping timespan in the first recording and the second recording;

correlate a portion of the first signature generated from a portion of the first audio recorded during the overlapping timespan and a portion of the second signature generated from a portion of the second audio recorded during the overlapping timespan to determine whether a portion of the overlapping timespan of the first recording and the second recording are correlated greater than a predetermined threshold;

signal the detection of the disallowed communication session between the first person using the first communication device and the second person using the second communication device, wherein the correlated portion of the overlapping timespan corresponds to a disallowed conversation between the first person and the second person; and

display, on a computer display, a notification of the detected disallowed communication session in response to the signaled detection.

7. The system of claim 6, wherein the first signature comprises spectral time information that includes one or more spectral peaks corresponding to the one or more speech formants in the first recording and the second signature comprises spectral time information that includes one or more spectral peaks corresponding to the one or more speech formants in the second recording.

8. The system of claim 7, wherein the amplitudes of the spectral peaks of the first signature are greater than the amplitudes of the spectral peaks of the second signature.

9. The system of claim 7, wherein correlating the first signature and the second signature comprises comparing the one or more spectral peaks in the first signature against the one or more spectral peaks in the second signature.

10. The system of claim 7, the memory device storing additional computer-readable instructions that, upon execution by the one or more processors, further cause the system to:

determine the overlapping timespan of the first audio and the second audio based on the correlation of the spectral peaks in the first signature and the spectral peaks in the second signature.

13. The system of claim 6, wherein the first signature is generated by calculating a first Fourier transform of one or more segments of the first recording, and wherein the second signature is generated by calculating a second Fourier transform of one or more segments of the second recording.

14. The system of claim 6, wherein the first signature is generated by calculating an autocorrelation of one or more segments of the first recording.

18. The system of claim 13, wherein the first Fourier transform has a first frequency resolution and the second Fourier transform has a second frequency resolution.

19. The system of claim 18, the memory device storing additional computer-readable instructions that, upon execution by the one or more processors, further cause the system to:

adjust the first frequency resolution of the first Fourier transform;

generate an updated first signature by calculating the first Fourier transform of the one or more segments of the first recording using the adjusted first frequency resolution; and

correlate a portion of the updated first signature generated from a portion of the first audio recorded during the overlapping timespan and a portion of the second signature generated from a portion of the second audio recorded during the overlapping timespan, wherein the spectral peaks in the updated first signature are compared against the spectral peaks in the second signature.

20. The system of claim 19, wherein the first frequency resolution is adjusted to identify within the updated first signature a portion of the one or more formants that have been identified in the second signature.