System and method for link analysis based on image processing

• US 9,740,915 B2
• Filed: 10/29/2012
• Issued: 08/22/2017
• Est. Priority Date: 10/31/2011
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Wiegenfeld, Eran, Svyatets, Alexander, Shmidov, Gennady
• Primary Examiner(s): CESE, KENNY A
• Application Number:
  US13/663,401
• Publication Number:
  US 20130148858A1
• Time in Patent Office: 1,758 Days
• Field of Search: 382105, 382115
• CPC Class Codes: G06F 2209/542 Intercept
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: H04L 63/306 intercepting packet switched data communications, e.g. Web, Internet or IMS communications
• CPC Class Codes: H04W 4/21 for social networking applications

Abstract

Methods and systems to identify relationships between individuals by analyzing digital images and automatically detecting individuals who appear together in the images. A link analysis system accepts one or more digital images, and automatically recognizes individuals who appear together in the images. The system may recognize the individuals, for example, by applying a suitable face recognition process to the images. Upon identifying individuals who appear together, the system defines a relationship between them and acts upon the relationship.

Claims

1. A method, comprising:

monitoring, by an interface coupled to a processor, communication conducted in a communication network;

extracting, by the interface, a plurality of digital images from a monitored communication session;

automatically recognizing, by the processor, first and second individuals who appear in first and second ones of the extracted digital images;

defining, by the processor, a relationship between the recognized first and second individuals; and

maintaining, by the processor, a relationship map as a data structure in a database, wherein the relationship map includes the defined relationship between the recognized first and second individuals;

wherein monitoring communication conducted in a communication network includes;

intercepting communication traffic conducted in the communication network such that communication traffic is diverted to pass through the interface before reaching the communication traffic'"'"'s intended destination;

detecting whether the communication traffic is encrypted in accordance with a cryptographic transport-layer protocol,

upon detecting that the communication traffic is encrypted in accordance with a cryptographic transport-layer protocol, decrypting the transport-layer encryption before extracting the plurality of digital images; and

after decrypting the communication traffic, re-encrypting the communication traffic with the transport-layer encryption and sending the re-encrypted traffic to the intended destination; and

comprising assigning a confidence score to the relationship,

wherein assigning the confidence score comprises setting the confidence score depending on a source from which the first images are obtained, wherein the confidence score is set higher for images that are obtained from seized equipment as compared to images that are obtained from public websites.

2. The method of claim 1, wherein extracting the plurality of digital images from a monitored communication session comprises extracting a plurality of digital images attached to a same email message.

3. The method according to claim 1, wherein extracting the images comprises reconstructing a communication session in which the images are exchanged, and extracting the images from the reconstructed session.

4. The method of claim 1, wherein extracting the plurality of digital images from a monitored communication session comprises extracting a plurality of digital images uploaded to a web-site in a same upload session.

5. The method according to claim 1, wherein recognizing the first and second individuals comprises recognizing in the images a car license plate that is associated with one of the first and second individuals.

6. The method according to claim 1, wherein assigning the confidence score comprises setting the confidence score depending on a number of times the first and second individuals appear in the images in a same monitored communication session.

7. Apparatus, comprising:

an interface that monitors communication conducted in a communication network and extracts a plurality of digital images from a monitored communication session; and

a processor that;

automatically recognizes first and second individuals who appear in first and second ones of the extracted digital images;

defines a relationship between the recognized first and second individuals; and

maintains a relationship map which includes the defined relationship between the recognized first and second individuals;

wherein the interface performs the monitoring by at least;

intercepting communication traffic conducted in the communication network such that communication traffic is diverted to pass through the apparatus before reaching the communication traffic'"'"'s intended destination,

detecting whether the communication traffic is encrypted in accordance with a cryptographic transport-layer protocol;

upon detecting that the communication traffic is encrypted in accordance with a cryptographic transport-layer protocol, decrypting the transport-layer encryption before extracting the plurality of digital images; and

after decrypting the communication traffic, re-encrypting the communication traffic with the transport-layer encryption and sending the re-encrypted traffic to the intended destination;

wherein the processor is configured to assign a confidence score to the relationship,

wherein the processor is configured to set the confidence score depending on a source from which the images are obtained, wherein the confidence score is set higher for images that are obtained from seized equipment as compared to images that are obtained from public websites.

8. The apparatus according to claim 7, wherein extracting the plurality of digital images from a monitored communication session comprises extracting a plurality of digital images attached to a same email message.

9. The apparatus according to claim 7, wherein the processor is configured to reconstruct a communication session in which the images are exchanged, and to extract the images from the reconstructed session.

10. The apparatus according to claim 7, wherein the processor is configured to recognize the first and second individuals by recognizing in the images a car license plate that is associated with one of the first and second individuals.

11. The apparatus according to claim 7, wherein the processor is configured to set the confidence score depending on a number of times the first and second individuals appear in the images in a same communication session.

12. The apparatus according to claim 7, wherein the interface is configured to accept the digital images from at least two image sources selected from a group of sources consisting of public Internet content, surveillance cameras and seized digital equipment.

13. The apparatus according to claim 7, wherein extracting the plurality of digital images from a monitored communication session comprises extracting a plurality of digital images uploaded to a web-site in a same upload session.