System, method and program product for camera-based discovery of social networks

• US 8,320,617 B2
• Filed: 03/27/2009
• Issued: 11/27/2012
• Est. Priority Date: 03/27/2009
• Status: Active Grant

• Current Assignee: UTC Fire & Security Americas Corporation, Inc. (Carrier Global Corporation)
• Original Assignee: UTC Fire & Security Americas Corporation, Inc. (Carrier Global Corporation)
• Inventors: Krahnstoever, Nils Oliver, Lim, Ser Nam, Yu, Ting, Patwardhan, Kedar Anil
• Primary Examiner(s): Werner, Brian P
• Assistant Examiner(s): Goradia, Shefali
• Application Number:
  US12/412,486
• Publication Number:
  US 20100245567A1
• Time in Patent Office: 1,341 Days
• Field of Search: 382/100, 382/103, 382/118, 348/143
• US Class Current: 382/103
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A system, method and program product for camera-based discovery of social networks. The computer implemented method for identifying individuals and associating tracks with individuals in camera-generated images from a face capture camera(s) and a tracking camera(s), wherein the computer implemented method includes: receiving images of an individual from the face capture camera(s) on a computer; receiving images of a track(s) of an individual from the tracking camera(s) on a computer; automatically determining with the computer the track(s) from the images from the tracking camera(s); and associating with the computer the track(s) with the individual(s) and a unique identifier. The present invention has been described in terms of specific embodiment(s), and it is recognized that equivalents, alternatives, and modifications, aside from those expressly stated, are possible and within the scope of the appending claims.

Claims

1. A computer implemented method for identifying individuals and associating tracks with individuals in camera-generated images, wherein the camera-generated images are from a camera system comprising at least one face capture camera and at least one tracking camera, said computer implemented method comprising:

receiving a plurality of images of a plurality of individuals from the at least one face capture camera on a computer;

receiving a plurality of images of at least one track along which the plurality of individuals moved from the at least one tracking camera on a computer;

automatically determining with the computer the at least one track along which the plurality of individuals moved from the plurality of images from the at least one tracking camera;

associating with the computer the at least one track along which the plurality of individuals moved with at least one of the plurality of individuals and a unique identifier; and

identifying a social network with the plurality of individuals based upon a frequency that the plurality of individuals are co-located.

2. The method of claim 1, wherein the identifying further comprises assigning individuals having similar tracks to a particular social group.

3. The method of claim 1, further comprising identifying an individual by associating the individual with a predetermined list.

4. The method of claim 3, wherein the plurality of images comprises a plurality of face images and the identifying the individual further comprises:

performing a facial recognition algorithm of the plurality of face images of the individual; and

computing motion similarities between a plurality of images of at least one track along which the individual moved.

5. The method of claim 1, further comprising constructing a social network graph based on the social network.

6. The method of claim 5, further comprising deriving a plurality of social groups from the social network graph.

7. The method of claim 6 the deriving comprises applying a modularity-cut algorithm to the social network.

8. The method of claim 6 the deriving comprises applying an eigen-based algorithm to the social network.

9. The method of claim 6, further comprising estimating at least one group leader of at least one of the plurality of social groups.

10. The method of claim 9, the estimating comprises applying an eigen-based algorithm to estimate the at least one group leader.

11. The method of claim 1, wherein the camera system is located at a venue selected from a group consisting of:

a prison, a shopping mall, a school, a public venue, a mass transit site, a public gathering, a financial company, and a sports venue.

12. The method of claim 1, the identifying comprising calculating motion similarities between two tracks on a ground plane.

13. The method of claim 12, the calculating comprising applying a graph-cut based algorithm.

14. The method of claim 1, comprising acquiring a plurality of high, resolution face images of the individual with the at least one face capture camera.

15. The method of claim 1, wherein the at least one face capture camera comprises one of a pan tilt zoom (PTZ) camera or high-resolution mega-pixel camera.

16. The method of claim 1, wherein the at least one tracking camera comprises a fixed surveillance camera.

17. A system for identifying a plurality of individuals and associating tracks with individuals comprising:

at least one camera configured to obtain a plurality of images of the plurality of individuals;

a computer configured to process the plurality of images, wherein the computer;

receives a plurality of images of the plurality of individuals from the at least one camera;

automatically determines at least two tracks along which the plurality of individuals moved from the plurality of images of the plurality of individuals;

associates each of the at least two tracks along which the plurality of individuals moved with unique identifiers; and

automatically identifies a social network based on a frequency that the plurality of individuals are co-located.

18. The system of claim 17, wherein the at least one camera comprises a high-resolution mega-pixel camera with a wide field of view.

19. The system of claim 17, wherein the at least one camera comprises a high-resolution catadioptric camera.

20. The system of claim 17 wherein the plurality of images comprises a plurality of facial images and wherein the computer performs operation comprising:

performing a facial recognition algorithm of the plurality of facial images of an individual; and

computing motion similarities between a plurality of images of at least one track along which the individual moved.

21. A computer implemented method for identifying a plurality of individuals and associating tracks with the plurality of individuals in camera-generated images, wherein the camera-generated images are from a camera system comprising at least one face capture camera and at least one tracking camera, said computer implemented method comprising:

receiving a plurality of images of the plurality of individuals from the at least one camera;

automatically determining at least two tracks along which the plurality of individuals moved from the plurality of images of the plurality of individuals;

associating the at least two tracks along which the plurality of individuals moved with unique identifiers; and

automatically identifying a social network for the plurality of individuals based upon a frequency that the plurality of individuals are co-located.

22. The computer implemented method of claim 21 wherein the plurality of images comprises a plurality of facial images and wherein the computer implemented method further comprises:

performing a facial recognition algorithm of the plurality of facial images of an individual; and

computing motion similarities between a plurality of images of at least one track along which the individual moved.

23. A non-transitory computer-readable media having computer readable instructions recorded thereon for identifying a plurality of individuals and associating tracks, with the plurality of individuals in camera-generated images, wherein the camera-generated images are from a camera system comprising at least one face capture camera and at least one tracking camera, the instructions comprising:

instructions for receiving a plurality of images of the plurality of individuals from the at least one face capture camera;

instructions for receiving a plurality of images of at least one track along which the plurality of individuals moved from the at least one tracking camera;

instructions for automatically determining the at least one track along which the plurality of individuals moved from the plurality of images from the at least one tracking camera;

instructions for associating the at least one track along which the plurality of individuals moved with at least one of an individual and a unique identifier; and

identifying a social network with the plurality of individuals based upon a frequency that the plurality of individuals are co-located.