Grouping items in video stream images into events

• US 20060222244A1
• Filed: 04/04/2006
• Published: 10/05/2006
• Est. Priority Date: 04/05/2005
• Status: Active Grant

• Current Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Original Assignee: 3VR Security Incorporated (Identiv, Inc.)
• Inventors: Haupt, Gordon T., Russell, Stephen G., Vallone, Robert P., Fleischer, Stephen D., Frederick, Timothy B., Freeman, J. Andrew
• Granted Patent: US 7,646,895 B2
• Time in Patent Office: Days
• US Class Current: 382/220
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A technique is disclosed for determining a group in which to add a new image set. The new image set is captured by one or more cameras in a video surveillance system. Similarity scores are generated between the new image set and one or more image sets of a plurality of groups, wherein each group includes one or more image sets of at least one object. The new image set is added to a group based on one or more factors. Also, a technique is disclosed for determining when to close a group, wherein a closed group is one to which an image set may not be added.

Claims

1. A method comprising performing a machine-executed operation involving instructions, wherein the machine-executed operation is at least one of:

A) sending said instructions over transmission media;

B) receiving said instructions over transmission media;

C) storing said instructions onto a machine-readable storage medium; and

D) executing the instructions;

wherein said instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform;

receiving a new image set;

selecting a matching group from a plurality of groups; and

adding the new image set to the matching group;

wherein each group of the plurality of groups includes one or more image sets;

wherein each image set comprises one or more images of at least one object;

wherein selecting a matching group from a plurality of groups includes;

generating a similarity score between the new image set and one or more image sets in each group of the plurality of groups; and

selecting the matching group based, at least in part, on one or more of the following factors;

a highest weighted similarity score of each group, wherein the highest weighted similarity score is weighted based on external information;

an average similarity score, of each group, between the new image set and one or more image sets of each group; and

the number of image sets in each group that have corresponding similarity scores above a certain threshold.

2. The method of claim 1, wherein selecting the matching group includes selecting the matching group based, at least in part, on the number of images sets in each group that have corresponding similarity scores above a certain threshold.

3. The method of claim 1, wherein selecting the matching group includes selecting the matching group based, at least in part, on the average similarity score, of each group, between the new image set and one or more image sets of each group.

4. The method of claim 3, wherein the average similarity score is weighted based on external information.

5. The method of claim 1, wherein selecting the matching group includes selecting the matching group based, at least in part, on a highest weighted similarity score, wherein the highest weighted similarity score is weighted based on external information.

6. The method of claim 5, wherein the external information includes at least one of the following:

time proximity between when the new image set was received and when the one or more image sets of said each group were received; and

spatial proximity between the object depicted in the image set and the object depicted in each of the one or more image sets of said each group.

7. The method of claim 6, wherein the external information includes time proximity between when the new image set was received and when the one or more image sets of said each group were received.

8. The method of claim 6, wherein:

the external information includes spatial proximity; and

the spatial proximity is determined by comparing the physical location of the camera that captured the new image set with the physical location of the one or more cameras that captured the one or more image sets of said each group.

9. The method of claim 1, wherein generating a similarity score between the new image set and one or more image sets in each group of the plurality of groups includes:

generating a feature set for each image set, wherein the feature set is a set of characteristic parameters of the at least one object depicted in said each image set; and

generating a similarity score for each of the one or more image sets based on the feature sets of the one or more image sets.

10. The method of claim 1, wherein the number of image sets in the selected group has reached a threshold number, wherein adding the new image set to the matching group includes determining, based on one or more second factors, whether to drop a particular image set from the matching group.

11. A method of determining whether to close a group, comprising performing a machine-executed operation involving instructions, wherein the machine-executed operation is at least one of:

A) sending said instructions over transmission media;

B) receiving said instructions over transmission media;

C) storing said instructions onto a machine-readable storage medium; and

D) executing the instructions;

wherein said instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform;

establishing a plurality of groups;

wherein each group includes one or more image sets;

wherein each image set includes one or more images of at least one object;

wherein each group corresponds to an object that is common among images in the one or more image sets that belong to said each group;

determining when to close a particular group of the plurality of groups based, at least in part, on one or more of;

how much time has elapsed since the latest image set was added to the particular group;

how many image sets are in the particular group;

the length of time the particular group has been open; and

data about the one or more image sets in the particular group;

before said particular group is closed, treating the particular group as a candidate to which new image sets may be added; and

after the particular group is closed, ceasing to treat the particular group as a candidate to which new image sets may be added.

12. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on a lapse of a particular time between when the latest image set was added to the particular group and a current time.

13. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on how many image sets are in the particular group.

14. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on the amount of time the particular group has been open.

15. The method of claim 11, wherein determining when to close a particular group of the plurality of groups is performed based, at least in part, on data about the one or more image sets in the particular group.

16. The method of claim 15, wherein data about the one or more image sets in the particular group includes at least one of the following:

similarity scores of at least one image set that is in the particular group;

quality measurements associated with the at least one image set; and

other parameters of the at least one image set.

17. The method of claim 16, wherein data about the one or more image sets in the particular group includes at least similarity scores of at least one image set that is in the particular group.

18. The method of claim 16, wherein data about the one or more image sets in the particular group includes at least quality measurements associated with the at least one image set.

19. The method of claim 16, wherein data about the one or more image sets in the particular group includes at least other parameters of the at least one image set, wherein other parameters include at least one of the following:

a similarity score between the at least one image set and a stored event is considered a positive identification, and the object is not within a certain portion of the at least one image set.