Method and system for automatic analysis of the trip of people in a retail space using multiple cameras

• US 8,098,888 B1
• Filed: 01/28/2008
• Issued: 01/17/2012
• Est. Priority Date: 01/28/2008
• Status: Active Grant

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Mummareddy, Satish, Sharma, Rajeev, Jung, Namsoon
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US12/011,650
• Time in Patent Office: 1,450 Days
• Field of Search: 382/103, 348/169
• US Class Current: 382/103
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion

Abstract

The present invention is a method and system for automatically determining the trip of people in a physical space, such as retail space, by capturing a plurality of input images of the people by a plurality of means for capturing images, processing the plurality of input images in order to track the people in each field of view of the plurality of means for capturing images, mapping the trip on to the coordinates of the physical space, joining the plurality of tracks across the multiple fields of view of the plurality of means for capturing images, and finding information for the trip of the people based on the processed results from the plurality of tracks. The trip information can comprise coordinates of the people'"'"'s position and temporal attributes, such as trip time and trip length, for the plurality of tracks. The physical space may be a retail space, and the people may be customers in the retail space. The trip information can provide key measurements along the entire shopping trip, from entrance to checkout, that deliver deeper insights about the trip as a whole.

Claims

1. A method for automatically determining the trip of a person in a physical space, comprising the following steps of:

a) capturing a plurality of input images of said person by a plurality of means for capturing images,

b) performing background subtraction applying a two-cluster model to a grayscale intensity histogram per pixel,

c) processing said plurality of input images in order to track said person in each field of view of said plurality of means for capturing images,

d) mapping the trip on to the coordinates of the physical space,

e) joining a plurality of tracks across the multiple fields of view, and

f) finding information for said trip of said person based on the processed results from the plurality of tracks,

wherein the two-cluster model consists of two pairs of thresholds that are represented in one 4-byte word.

2. The method according to claim 1, wherein the method further comprises steps of:

a) tracking said person in a first field of view of a first means for capturing images among the plurality of means for capturing images,

b) tracking said person in a second field of view of a second means for capturing images among the plurality of means for capturing images when said person moves from the first field of view to the second field of view, and

c) aggregating the plurality of tracks in the first field of view and the second field of view,

wherein the results of the aggregated tracks provide a series of coordinates and temporal attribute for said trip of said person.

3. The method according to claim 1, wherein the method further comprises a step of utilizing the trip information to provide key measurements along the entire trip, from entrance to exit, that deliver deeper insights about the trip as a whole.

4. The method according to claim 1, wherein the method further comprises a step of extracting analytical and statistical data from said trip information.

5. The method according to claim 1, wherein the method further comprises a step of detecting incomplete and broken tracks and filtering them out to get a subset of the more reliable complete tracks.

6. The method according to claim 1, wherein the method further comprises a step of differentiating said person from the background by analyzing past motion information to generate a background model for each pixel in each of said plurality of means for capturing images,

wherein said background model is a non-Gaussian cluster of pixel intensities.

7. The method according to claim 1, wherein the method further comprises a step of computing changes in mean intensity and amount of foreground segmentation of the input images, and then resetting a background model based on the changes.

8. The method according to claim 1, wherein the method further comprises a step of applying calibration information of said plurality of means for capturing images to transform each local track in said plurality of means for capturing images into a global coordinate space.

9. The method according to claim 1, wherein the method further comprises a step of detecting the existence of said person at each pixel level in the images for each video stream.

10. The method according to claim 1, wherein the method further comprises a step of detecting the existence of said person using a model-based approach.

11. The method according to claim 10, wherein the method further comprises a step of modeling both the means for capturing images and the dimensions of an average-sized person.

12. The method according to claim 1, wherein the method further comprises a step of utilizing a rule application module for processing the group behavior analysis and joining the tracks,

whereby the rule application module comprises information unit verification technologies.

13. An apparatus for automatically determining the trip of a person in a physical space, comprising:

a) a plurality of means for capturing images that captures a plurality of input images of said person,

b) means for control and processing that performs the following steps of;

performing background subtraction applying a two-cluster model to a grayscale intensity histogram per pixel,

processing said plurality of input images in order to track said person in each field of view of said plurality of means for capturing images,

mapping the trip on to the coordinates of the physical space,

joining a plurality of tracks across the multiple fields of view, and

finding information for said trip of said person based on the processed results from the plurality of tracks,

wherein the two-cluster model consists of two pairs of thresholds that are represented in one 4-byte word.

14. The apparatus according to claim 13, wherein the apparatus further comprises a computer that performs the following steps of:

a) tracking said person in a first field of view of a first means for capturing images among the plurality of means for capturing images,

b) tracking said person in a second field of view of a second means for capturing images among the plurality of means for capturing images when said person moves from the first field of view to the second field of view, and

c) aggregating the plurality of tracks in the first field of view and the second field of view,

wherein the results of the aggregated tracks provide a series of coordinates and temporal attributes for said trip of said person.

15. The apparatus according to claim 13, wherein the apparatus further comprises a computer for utilizing the trip information to provide key measurements along the entire trip, from entrance to exit, that deliver deeper insights about the trip as a whole.

16. The apparatus according to claim 13, wherein the apparatus further comprises a computer for extracting analytical and statistical data from said trip information.

17. The apparatus according to claim 13, wherein the apparatus further comprises a computer for detecting incomplete and broken tracks and filtering them out to get a subset of the more reliable complete tracks.

18. The apparatus according to claim 13, wherein the apparatus further comprises a computer for differentiating said person from the background by analyzing past motion information to generate a background model for each pixel in each of said plurality of means for capturing images,

wherein said background model is a non-Gaussian cluster of pixel intensities.

19. The apparatus according to claim 13, wherein the apparatus further comprises a computer for computing changes in mean intensity and amount of foreground segmentation of the input images, and then resetting a background model based on the changes.

20. The apparatus according to claim 13, wherein the apparatus further comprises a computer for applying calibration information of said plurality of means for capturing images to transform each local track in said plurality of means for capturing images into a global coordinate space.

21. The apparatus according to claim 13, wherein the apparatus further comprises a computer for detecting the existence of said person at each pixel level in the images for each video stream.

22. The apparatus according to claim 13, wherein the apparatus further comprises a computer for detecting the existence of said person using a model-based approach.

23. The apparatus according to claim 22, wherein the apparatus further comprises a computer for modeling both the means for capturing images and the dimensions of an average-sized person.

24. The apparatus according to claim 13, wherein the apparatus further comprises a computer for utilizing a rule application module for processing the group behavior analysis and joining the tracks,

whereby the rule application module comprises information unit verification technologies.