INTERACTIVE SYSTEM FOR RECOGNITION ANALYSIS OF MULTIPLE STREAMS OF VIDEO

• US 20090074261A1
• Filed: 10/30/2008
• Published: 03/19/2009
• Est. Priority Date: 03/16/2004
• Status: Active Grant

• Current Assignee: Freeman J. Andrew, Gordon T. Haupt, Robert P. Vallone, Stephen D. Fleischer, Stephen G. Russell, Timothy B. Frederick
• Original Assignee: Freeman J. Andrew, Gordon T. Haupt, Robert P. Vallone, Stephen D. Fleischer, Stephen G. Russell, Timothy B. Frederick
• Inventors: Haupt, Gordon T., Russell, Stephen G., Vallone, Robert P., Fleischer, Stephen D., Frederick, Timothy B., Freeman, J. Andrew
• Granted Patent: US 8,803,975 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method of identifying an object captured in a video image in a multi-camera video surveillance system is disclosed. Sets of identifying information are stored in profiles, each profile being associated with one object. The disclosed method of identifying an object includes comparing identifying information extracted from images captured by the video surveillance system to one or more stored profiles. A confidence score is calculated for each comparison and used to determine a best match between the extracted set of identifying information and an object. In one embodiment, the method is used as part of a facial recognition system incorporated into a video surveillance system.

Claims

1. A method for maintaining associations between profiles and objects in an object recognition system, wherein a profile comprises a set of identifying information extracted from an image set associated with the profile, comprising the computer-implemented steps of:

automatically creating a first association between a first stored profile and a first object;

automatically creating a second association between a second stored profile and the first object;

providing a view of the image sets associated with the first and second stored profiles to a user;

receiving feedback from the user about the second association; and

modifying the second association in accordance with the received feedback.

2. The method of claim 1, wherein an object comprises a person, and a set of identifying information comprises a facial feature set.

3. The method of claim 1, wherein the step of providing a view of the image sets associated with the first and second stored profiles additionally comprises providing a view of the first and second associations to the user.

4. The method of claim 3, wherein:

the step of receiving feedback includes receiving feedback that the second association is correct; and

the step of modifying includes storing confirmation information with the second association.

5. The method of claim 3, wherein the step of receiving feedback includes receiving feedback that the second association is incorrect, and the step of modifying includes removing the second stored profile from the object recognition system.

6. The method of claim 3, wherein the step of receiving feedback includes receiving feedback indicating that the second stored profile should be associated with a second object;

and the step of modifying includes creating an association between the second stored profile and the second object.

7. The method of claim 1, further comprising the computer-implemented steps of:

receiving a new image set;

extracting a set of identifying information from the new image set;

creating a new profile that includes the extracted set of identifying information;

providing a view of the new image set to a user;

receiving feedback from the user indicating an object that should be associated with the new profile; and

creating a new association between the new profile and the object indicated by the user.

8. The method of claim 7, wherein:

the step of receiving feedback from the user indicating an object comprises receiving feedback from the user indicating that the new profile should be associated with the first object; and

the step of creating an association comprises creating an association between the new profile and the first object, such that the first stored profile, the second stored profile and the new profile are all associated with the first object.

9. The method of claim 7, wherein:

the step of providing a view of the new image set additionally comprises providing a list of potential object identifications for the new profile; and

the step of receiving feedback from the user indicating an object comprises receiving a choice of one object from the list.

10. The method of claim 7, wherein:

the step of receiving feedback from the user indicating an object comprises receiving feedback that a new object should be created; and

the step of creating a new association comprises creating a new object and creating an association between the new profile and the new object.

11. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

12. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

13. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

14. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

15. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

16. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

17. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

18. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

19. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

20. A computer-readable storage medium storing instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.