Method and system for recognizing employees in a physical space based on automatic behavior analysis

• US 7,957,565 B1
• Filed: 03/28/2008
• Issued: 06/07/2011
• Est. Priority Date: 04/05/2007
• Status: Active Grant

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Hershey, Jeff, Sharma, Rajeev, Mummareddy, Satish, Jung, Namsoon
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US12/079,901
• Time in Patent Office: 1,166 Days
• Field of Search: None
• US Class Current: 382/115
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

The present invention is a method and system for recognizing employees among the people in a physical space based on automatic behavior analysis of the people in a preferred embodiment. The present invention captures a plurality of input images of the people in the physical space by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the behavioral characteristics of the people for the employee recognition purpose. The behavior analysis can comprise a path analysis as one of the characterization methods. The path analysis collects a plurality of trip information for each tracked person during a predefined window of time. The trip information can comprise spatial and temporal attributes, such as coordinates of the person'"'"'s position, trip time, trip length, and average velocity for each of the plurality of trips. Based on the employee recognition criteria applied to the trip information, the present invention distinguishes employees from non-employees during a predefined window of time. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the behavior of the people from the plurality of input images.

Claims

1. A method for recognizing a plurality of employees in a physical space based on behavior analysis of the people in said physical space, comprising the following steps of:

a) capturing a plurality of input images of a plurality of persons by a plurality of means for capturing images in said physical space,

b) processing said plurality of input images in order to track each person in said plurality of persons,

c) processing behavior analysis of each person in said plurality of persons based on the tracking, and

d) recognizing said plurality of employees by applying employee recognition criteria to the output of the behavior analysis for said plurality of persons,

whereby said output of the behavior analysis comprises a sequence of visits or a combination of visits to a predefined category in said physical space by said plurality of persons.

2. The method according to claim 1, wherein the method further comprises a step of repeating the steps from a) to d) in claim 1 for a plurality of physical spaces in a network of the physical spaces.

3. The method according to claim 1, wherein the method further comprises a step of differentiating the levels of employee recognition,

wherein first employee recognition criteria are applied to a group of employees and second employee recognition criteria are applied to another group of employees.

4. The method according to claim 1, wherein the method further comprises a step of processing the path analysis of each person in said plurality of persons for said behavior analysis, to distinguish employees from non-employees,

whereby attributes of said path analysis comprise information for initial point and destination, global trip, time, and average velocity.

5. The method according to claim 1, wherein the method further comprises a step of constructing the employee recognition criteria based on a set of predefined rules for employee recognition,

whereby a rule application logic module is utilized for applying the employee recognition criteria to the behavior analysis data.

6. The method according to claim 1, wherein the method further comprises a step of utilizing spatial rules as the employee recognition criteria,

whereby the spatial rules comprise at least one threshold for the frequency of tracking in a predefined place of said physical space.

7. The method according to claim 1, wherein the method further comprises a step of utilizing temporal rules as the employee recognition criteria,

whereby the temporal rules comprise at least one threshold for dwell time and average trip time in a specific place of said physical space.

8. The method according to claim 1, wherein the method further comprises a step of utilizing repetitive pattern of behavior as the employee recognition criteria,

whereby the repetitive pattern comprises repetitive trips between two points in said physical space.

9. The method according to claim 1, wherein the method further comprises a step of utilizing specific events in the physical space as the employee recognition criteria,

whereby behaviors driven by said specific event comprise a responsive behavior to an announcement for employees in said physical space.

10. The method according to claim 1, wherein the method further comprises a step of utilizing the relationship among the tracks of said plurality of persons as the employee recognition criteria,

whereby the relationship comprises one-to-many relationship over a period of time, in which one track interacts with multiple tracks.

11. The method according to claim 1, wherein the method further comprises a step of constructing said employee recognition criteria based on a combination of a set of predefined rules,

whereby population in said physical space can be divided into subpopulations by the combination of rules.

12. The method according to claim 1, wherein the method further comprises a step of constructing said employee recognition criteria based on application of a learning algorithm-based behavior analysis to the training population.

13. The method according to claim 12, wherein the method further comprises a step of utilizing distance measure and clustering for said learning algorithm-based behavior analysis.

14. The method according to claim 12, wherein the method further comprises a step of utilizing a Support Vector Machine (SVM) as said learning algorithm-based behavior analysis,

whereby learned features of the behavior comprise duration of dwell time, start, end, number of stops, location of stops, average length of stops, velocity change, repetition of location, displacement of stops, non-uniform pattern, percentage of time spent in a specific employee region, and color signature of tracks of said plurality of persons.

15. The method according to claim 1, wherein the method further comprises a step of constructing said employee recognition criteria based on example-based behavior analysis algorithm.

16. The method according to claim 1, wherein the method further comprises a step of applying different employee recognition criteria per each employee group,

whereby the application of different employee recognition criteria per each employee group makes deeper understanding of said each employee group possible.

17. The method according to claim 1, wherein the method further comprises a step of defining domain-specific criteria for said employee recognition criteria,

whereby a subset of the domain-specific criteria are constructed based on retail-specific rules.

19. The apparatus according to claim 18, wherein the apparatus further comprises means for repeatedly using the means from a) to d) in claim 1 for a plurality of physical spaces in a network of the physical spaces.

18. An apparatus for recognizing a plurality of employees in a physical space based on behavior analysis of the people in said physical space, comprising:

a) means for capturing a plurality of input images of a plurality of persons by a plurality of means for capturing images in said physical space,

b) means for processing said plurality of input images in order to track each person in said plurality of persons,

c) means for processing behavior analysis of each person in said plurality of persons based on the tracking, and

d) means for recognizing said plurality of employees by applying employee recognition criteria to the output of the behavior analysis for said plurality of persons,

whereby said output of the behavior analysis comprises a sequence of visits or a combination of visits to a predefined category in said physical space by said plurality of persons.

20. The apparatus according to claim 18, wherein the apparatus further comprises means for differentiating the levels of employee recognition,

wherein first employee recognition criteria are applied to a group of employees and second employee recognition criteria are applied to another group of employees.

21. The apparatus according to claim 18, wherein the apparatus further comprises means for processing the path analysis of each person in said plurality of persons for said behavior analysis, to distinguish employees from non-employees,

whereby attributes of said path analysis comprise information for initial point and destination, global trip, time, and average velocity.

22. The apparatus according to claim 18, wherein the apparatus further comprises means for constructing the employee recognition criteria based on a set of predefined rules for employee recognition,

whereby a rule application logic module is utilized for applying the employee recognition criteria to the behavior analysis data.

23. The apparatus according to claim 18, wherein the apparatus further comprises means for utilizing spatial rules as the employee recognition criteria,

whereby the spatial rules comprise at least one threshold for the frequency of tracking in a predefined place of said physical space.

24. The apparatus according to claim 18, wherein the apparatus further comprises means for utilizing temporal rules as the employee recognition criteria,

whereby the temporal rules comprise at least one threshold for dwell time and average trip time in a specific place of said physical space.

25. The apparatus according to claim 18, wherein the apparatus further comprises means for utilizing repetitive pattern of behavior as the employee recognition criteria,

whereby the repetitive pattern comprises repetitive trips between two points in said physical space.

26. The apparatus according to claim 18, wherein the apparatus further comprises means for utilizing specific events in the physical space as the employee recognition criteria,

whereby behaviors driven by said specific event comprise a responsive behavior to an announcement for employees in said physical space.

27. The apparatus according to claim 18, wherein the apparatus further comprises means for utilizing the relationship among the tracks of said plurality of persons as the employee recognition criteria,

whereby the relationship comprises one-to-many relationship over a period of time, in which one track interacts with multiple tracks.

28. The apparatus according to claim 18, wherein the apparatus further comprises means for constructing said employee recognition criteria based on a combination of a set of predefined rules,

whereby population in said physical space can be divided into subpopulations by the combination of rules.

29. The apparatus according to claim 18, wherein the apparatus further comprises means for constructing said employee recognition criteria based on application of a learning algorithm-based behavior analysis to the training population.

30. The apparatus according to claim 29, wherein the apparatus further comprises means for utilizing distance measure and clustering for said learning algorithm-based behavior analysis.

31. The apparatus according to claim 29, wherein the apparatus further comprises means for utilizing a Support Vector Machine (SVM) as said learning algorithm-based behavior analysis,

whereby learned features of the behavior comprise duration of dwell time, start, end, number of stops, location of stops, average length of stops, velocity change, repetition of location, displacement of stops, non-uniform pattern, percentage of time spent in a specific employee region, and color signature of tracks of said plurality of persons.

32. The apparatus according to claim 18, wherein the apparatus further comprises means for constructing said employee recognition criteria based on example-based behavior analysis algorithm.

33. The apparatus according to claim 18, wherein the apparatus further comprises means for applying different employee recognition criteria per each employee group,

whereby the application of different employee recognition criteria per each employee group makes deeper understanding of said each employee group possible.

34. The apparatus according to claim 18, wherein the apparatus further comprises means for defining domain-specific criteria for said employee recognition criteria,

whereby a subset of the domain-specific criteria are constructed based on retail-specific rules.