Systems and methods for detecting call provenance from call audio

• US 9,037,113 B2
• Filed: 06/29/2011
• Issued: 05/19/2015
• Est. Priority Date: 06/29/2010
• Status: Active Grant

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Balasubramaniyan, Vijay, Ahamad, Mustaque, Traynor, Patrick Gerard, Hunter, Michael Thomas, Poonawalla, Aamir
• Primary Examiner(s): Gelin, Jean
• Application Number:
  US13/807,837
• Publication Number:
  US 20130109358A1
• Time in Patent Office: 1,420 Days
• Field of Search: 455/403, 455/410, 455/411, 455/412.1, 455/450, 455/451
• US Class Current: 455/411
• CPC Class Codes: H04L 43/0829 Packet loss
• CPC Class Codes: H04L 65/1076 Screening of IP real time communications, e.g. spam over Internet telephony [SPIT]
• CPC Class Codes: H04M 1/68 Circuit arrangements for preventing eavesdropping
• CPC Class Codes: H04M 2203/558 Databases
• CPC Class Codes: H04M 2203/6027 Fraud preventions
• CPC Class Codes: H04M 2203/6045 Identity confirmation
• CPC Class Codes: H04M 3/2281 Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls
• CPC Class Codes: H04M 7/0078 Security; Fraud detection; Fraud prevention
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
• CPC Class Codes: H04W 12/12 Detection or prevention of fraud
• CPC Class Codes: H04W 12/63 Location-dependent; Proximity-dependent
• CPC Class Codes: H04W 12/65 Environment-dependent, e.g. using captured environmental data
• CPC Class Codes: H04W 12/79 Radio fingerprint
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic

Abstract

Various embodiments of the invention are detection systems and methods for detecting call provenance based on call audio. An exemplary embodiment of the detection system can comprise a characterization unit, a labeling unit, and an identification unit. The characterization unit can extract various characteristics of networks through which a call traversed, based on call audio. The labeling unit can be trained on prior call data and can identify one or more codecs used to encode the call, based on the call audio. The identification unit can utilize the characteristics of traversed networks and the identified codecs, and based on this information, the identification unit can provide a provenance fingerprint for the call. Based on the call provenance fingerprint, the detection system can identify, verify, or provide forensic information about a call audio source.

Claims

1. A method comprising:

receiving, for a first call between a plurality of parties at a corresponding plurality of devices, data, received at a first device of the plurality of devices during the first call, representing audio from one or more other devices of the plurality of devices;

determining, by a computing device, based on the data representing the audio received at the first device, at least one of a noise profile, packet loss characteristics, and speaker-independent artifacts associated with the audio received at the first device; and

at least one of identifying and verifying uniquely, based on the at least one of the noise profile, packet loss characteristics, and speaker-independent artifacts associated with the audio received at the first device, at least a portion of a path of transmission of the first call between the plurality of devices, wherein the at least one of identifying and verifying uniquely is not dependent on metadata associated with the first call.

2. The method of claim 1, the plurality of parties including a calling party and a called party, the path of transmission of the first call between the plurality of devices comprising a path from a device of the calling party to a device of the called party.

3. The method of claim 1, the plurality of parties including a calling party and a called party, the path of transmission of the first call between the plurality of devices comprising a path from a device of the called party to a device of the calling party.

4. The method of claim 1, wherein the at least one of identifying and verifying uniquely the at least a portion of a path of transmission of the first call comprises identifying a unique telecommunications device involved in the first call.

5. The method of claim 4 wherein the telecommunications device is one of the corresponding plurality of devices.

6. The method of claim 1, further comprising verifying at least a portion of the metadata associated with the first call based on the at least one of identifying and verifying uniquely.

7. The method of claim 1, wherein the at least one of identifying and verifying uniquely the at least a portion of a path of transmission of the first call comprises identifying a geography associated with a device from the one or more other devices.

8. The method of claim 1, wherein the at least one of identifying and verifying uniquely the at least a portion of a path of transmission of the first call comprises determining that the first call and a second call have in common a same portion of the path of transmission of the first call.

9. The method of claim 1, wherein the at least one of identifying and verifying uniquely the at least a portion of a path of transmission of the first call comprises:

determining a feature profile of the first call based on the data representing the audio received at the first device; and

classifying the feature profile of the first call based at least in part on a plurality of feature profiles of previous calls.

10. The method of claim 1, further comprising:

determining a plurality of provenance fingerprints for a plurality of calls, each provenance fingerprint being based on one or more predetermined characteristics detected in audio from an associated call, and each provenance fingerprint uniquely identifying at least a portion of a path of the associated call; and

compiling the plurality of provenance fingerprints into an exclusion list.

11. The method of claim 10, the exclusion list being a blacklist.

12. The method of claim 10, the exclusion list being a whitelist.

13. The method of claim 1, further comprising:

determining a plurality of provenance fingerprints for a plurality of calls, each provenance fingerprint being based on characteristics detected in audio from an associated call, and each provenance fingerprint uniquely identifying an associated call source; and

compiling the plurality of provenance fingerprints into a fraud database.

14. The method of claim 13, further comprising:

determining a first provenance fingerprint for the first audio from the first call, wherein the provenance fingerprint is based on one or more characteristics of the audio from the first call and uniquely identifies the at least a portion of the path of the first call;

matching the first provenance fingerprint to at least one of the plurality of provenance fingerprints in the fraud database; and

automatically providing a notification based on the matching.

15. The method of claim 1, further comprising determining a provenance fingerprint for the first call, wherein the provenance fingerprint is based on one or more predetermined characteristics detected in the audio from the first call.

16. The method of claim 15, wherein the provenance fingerprint uniquely identifies the at least a portion of the path of transmission.

17. The method of claim 16, wherein the provenance fingerprint identifies a unique telecommunications device involved in the first call.

18. The method of claim 16, wherein the provenance fingerprint uniquely identifies a geography of an origin of the first call.

19. The method of claim 15, wherein determining the provenance fingerprint of the first call occurs absent consideration of metadata associated with the call.

20. The method of claim 19, further comprising calculating a risk score indicating a measure of similarity between the provenance fingerprint and the metadata.

21. The method of claim 15, further comprising adding the provenance fingerprint to a blacklist.

22. The method of claim 21, further comprising notifying a caller when a call recipient is included in the blacklist.

23. The method of claim 15, further comprising adding the provenance fingerprint to a whitelist.

24. The method of claim 23, further comprising notifying a caller when a call recipient is included in the whitelist.

25. A system comprising:

a processor;

a memory operatively coupled to the processor;

a characterization unit for extracting, independent of metadata associated with a first call, one or more characteristics from call audio received at only a single end of the first call, wherein the one or more characteristics include at least one of a noise profile, packet loss characteristics, and speaker-independent artifacts associated with the audio received at only the single end of the first call and storing the extracted characteristics in the memory;

a training unit for learning application of one or more of labels to a plurality of prior call audios, based at least partially on characteristics detected from audio from prior calls; and

a prediction unit for predicting, with the processor, labels for the first call based on the learned application of the one or more labels from the training unit to the call audio of the first call.

26. The system of claim 25, the prediction unit being configured to at least one of identify and verify uniquely at least a portion of a transmission path of the first call, based solely on characteristics extracted from the audio from the first call.

27. The system of claim 26, the prediction unit being configured to at least one of identify and verify a unique telecommunications device involved in the first call, based on characteristics extracted from the audio from the first call.

28. The system of claim 26, the prediction unit being configured to at least one of identify and verify a geography of an origin of the first call, based on the characteristics extracted from the audio from the first call.

29. The system of claim 25, the prediction unit being further configured to determine a first provenance fingerprint for the first call, based on characteristics detected in the audio from the first call, the first provenance fingerprint indicating uniquely at least a portion of the path of the audio from the first call.

30. The system of claim 29, the prediction unit being further configured to identify at least a partial match between the first provenance fingerprint for the first call and a second provenance fingerprint for a second call.

31. The system of claim 30, the prediction unit being further configured to identify that a unique telecommunications device involved in the first call is the same as a unique telecommunications device involved in the second call, based on comparing the first provenance fingerprint to the second provenance fingerprint.

32. The system of claim 30, the prediction unit being further configured to identify a common geography between an origin of the first call and an origin of the second call, based on comparing the first provenance fingerprint to the second provenance fingerprint.

33. The system of claim 29, the characterization unit being further configured to identify the noise profile in the audio from the first call, wherein the determination of the first provenance fingerprint is based at least partially on the identification of the noise profile.

34. The system of claim 29, the characterization unit being further configured to measure quality of the audio from the first call, wherein the determination of the first provenance fingerprint is based at least partially on the quality of the audio from the first call.

35. The system of claim 25, the training unit comprising a multi-label classifier.

36. The system of claim 35, the multi-label classifier being configured to identify which of a predetermined plurality of codecs were used to encode the first call.

37. The system of claim 25, the training unit being further configured to detect one or more unique networks traversed by the first call.

38. The system of claim 25, wherein the characterization unit, the training unit, and the prediction unit are co-located at the single end of the first call.

39. A method comprising:

detecting, independent of voice analysis, one or more characteristics associated with a first call solely from data representing audio received at only a single end of the first call, wherein the one or more characteristics include at least one of a noise profile, packet loss characteristics, and speaker-independent artifacts associated with the audio received at only the single end of the first call;

determining, by a processor, a first provenance fingerprint for the first call based on the detected call characteristics; and

identifying and verifying uniquely one or more devices or networks spanning at least a portion of a transmission path of the first call.

40. The method of claim 39, further comprising identifying an error in at least a portion of metadata associated with the first call, based on the characteristics detected in the audio from the first call.

41. The method of claim 40, further comprising calculating a risk score based on a comparison of the provenance fingerprint to the metadata.

42. The method of claim 39, further comprising verifying at least a portion of metadata associated with the first call, based on the characteristics detected in the audio from the first call.

43. The method of claim 39, further comprising identifying at least one type of network traversed by the first call, based on the audio from the first call, wherein the determination of the first provenance fingerprint is based at least partially on the identification of the at least one type of network traversed by the first call.

44. The method of claim 39, further comprising identifying at least one codec used to encode the first call, based on the audio from the first call, wherein the determination of the first provenance fingerprint is based at least partially on the identification of the at least one codec.

45. The method of claim 39, further comprising detecting a packet-loss concealment algorithm used during transmission of the first call, based on the audio from the first call, wherein the determination of the first provenance fingerprint is based at least partially on the detection of the packet-loss concealment algorithm.

46. The method of claim 39, further comprising identifying a geographical origin of the first call, based on the characteristics detected in the audio from the first call.

47. The method of claim 39, further comprising verifying a unique telecommunications device involved in the first call, based on the audio from the first call, wherein the verification comprises:

receiving metadata associated with the first call, wherein the metadata includes a phone number of a calling party;

comparing the metadata to the provenance fingerprint for the first call; and

providing notification whether the provenance fingerprint indicates that the phone number in the metadata is incorrect.