Grouping items in video stream images into events

• US 7,646,895 B2
• Filed: 04/04/2006
• Issued: 01/12/2010
• Est. Priority Date: 04/05/2005
• Status: Expired due to Fees

• Current Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Original Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Inventors: Haupt, Gordon T., Russell, Stephen G., Vallone, Robert P., Fleischer, Stephen D., Frederick, Timothy B., Freeman, J. Andrew
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): DRENNAN, BARRY T
• Application Number:
  US11/398,158
• Publication Number:
  US 20060222244A1
• Time in Patent Office: 1,379 Days
• Field of Search: 382/103, 382115-118, 382/224, 382/305, 348143-159, 348/161
• US Class Current: 382/115
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A technique is disclosed for determining a group in which to add a new image set. The new image set is captured by one or more cameras in a video surveillance system. Similarity scores are generated between the new image set and one or more image sets of a plurality of groups, wherein each group includes one or more image sets of at least one object. The new image set is added to a group based on one or more factors. Also, a technique is disclosed for determining when to close a group, wherein a closed group is one to which an image set may not be added.

Claims

1. A method comprising the steps of:

receiving a new image set;

selecting a matching group from a plurality of groups; and

adding the new image set to the matching group;

wherein each group of the plurality of groups includes one or more image sets;

wherein at least one group of the plurality of groups includes a plurality of image sets;

wherein at least one image set in the plurality of groups includes a plurality of images;

wherein each image set comprises one or more images of at least one object;

wherein selecting a matching group from a plurality of groups includes;

generating a similarity score between the new image set and one or more image sets in each group of the plurality of groups; and

selecting the matching group based, at least in part, on one or more of the following factors;

a highest weighted similarity score of each group, wherein the highest weighted similarity score of a particular group of the plurality of groups is weighted based on a comparison between (a) time or spatial information about the new image set and (b) expected time or spatial information about the object associated with the particular group, wherein the expected time or spatial information is not derived from any image;

an average similarity score of each group, wherein the average similarity score for said at least one group is the average of the similarity scores of the plurality of image sets of said at least one group;

or

the number of image sets in each group that have corresponding similarity scores above a certain threshold;

wherein the steps are performed on one or more computing devices.

2. The method of claim 1, wherein selecting the matching group includes selecting the matching group based, at least in part, on the number of image sets in each group that have corresponding similarity scores above a certain threshold.

3. The method of claim 1, wherein selecting the matching group includes selecting the matching group based, at least in part, on the average similarity score of each group, wherein the average similarity score for said at least one group is the average of the similarity scores of the plurality of image sets of said at least one group.

4. The method of claim 3, wherein the average similarity score is weighted based on external information.

5. The method of claim 1, wherein:

selecting the matching group includes selecting the matching group based, at least in part, on a highest weighted similarity score; and

the highest weighted similarity score of a particular group of the plurality of groups is weighted based on a comparison between (a) time or spatial information about the new image set and (b) expected time or spatial information about the object associated with the particular group, wherein the expected time or spatial information is not derived from any image.

6. The method of claim 4, wherein the external information includes at least one of the following:

time proximity between when the new image set was received and when the one or more image sets of said each group were received;

or

spatial proximity between the object depicted in the image set and the object depicted in each of the one or more image sets of said each group.

7. The method of claim 6, wherein the external information includes time proximity between when the new image set was received and when the one or more image sets of said each group were received.

8. The method of claim 6, wherein:

the external information includes spatial proximity; and

the spatial proximity is determined by comparing the physical location of the camera that captured the new image set with the physical location of the one or more cameras that captured the one or more image sets of said each group.

9. The method of claim 1, wherein generating a similarity score between the new image set and one or more image sets in each group of the plurality of groups includes:

generating a feature set for each image set, wherein the feature set is a set of characteristic parameters of the at least one object depicted in said each image set; and

generating a similarity score for each of the one or more image sets based on the feature sets of the one or more image sets.

10. The method of claim 1, wherein the number of image sets in the selected group has reached a threshold number, wherein adding the new image set to the matching group includes determining, based on one or more second factors, whether to drop a particular image set from the matching group.

11. The method of claim 1, wherein each group of the plurality of groups corresponds to an object that is common among images in the one or more image sets that belong to said each group;

and

the method further comprises;

determining when to close a particular group of the plurality of groups based, at least in part, on one or more of;

how much time has elapsed since the latest image set was added to the particular group;

how many image sets are in the particular group;

the length of time the particular group has been open;

or

data about the one or more image sets in the particular group;

before said particular group is closed, treating the particular group as a candidate to which new image sets may be added; and

after the particular group is closed, ceasing to treat the particular group as a candidate to which new image sets may be added.

12. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on a lapse of a particular time between when the latest image set was added to the particular group and a current time.

13. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on how many image sets are in the particular group.

14. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on the amount of time the particular group has been open.

15. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on data about the one or more image sets in the particular group.

16. The method of claim 15, wherein data about the one or more image sets in the particular group includes at least one of the following:

similarity scores of at least one image set that is in the particular group;

quality measurements associated with the at least one image set;

or

other parameters of the at least one image set.

17. The method of claim 16, wherein data about the one or more image sets in the particular group includes at least similarity scores of at least one image set that is in the particular group.

18. The method of claim 16, wherein data about the one or more image sets in the particular group includes at least quality measurements associated with the at least one image set.

19. The method of claim 16, wherein data about the one or more image sets in the particular group includes at least other parameters of the at least one image set, wherein other parameters include at least one of the following:

a similarity score between the at least one image set and a stored event is considered a positive identification, or

the object is not within a certain portion of the at least one image set.

20. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 1.

21. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 2.

22. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 3.

23. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 4.

24. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 5.

25. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 6.

26. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 7.

27. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 8.

28. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 9.

29. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 10.

30. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 11.

31. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 12.

32. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 13.

33. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 14.

34. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 15.

35. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 16.

36. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 17.

37. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 18.

38. One or more storage media storing instructions which, when processed by one or more processors, cause the performance of the steps recited in claim 19.