System and method for detecting, tracking and counting human objects of interest using a counting system and a data capture device

• US 9,177,195 B2
• Filed: 09/18/2012
• Issued: 11/03/2015
• Est. Priority Date: 09/23/2011
• Status: Active Grant

• Current Assignee: Shoppertrak RCT LLC
• Original Assignee: Shoppertrak RCT Corporation (Johnson Controls International plc)
• Inventors: Marcheselli, Edward A., Dispensa, Gary, Stemen, Thomas D., Kastilahn, William C.
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US13/622,083
• Publication Number:
  US 20140079282A1
• Time in Patent Office: 1,141 Days
• Field of Search: 382/100, 382/103, 382/107, 348/143, 348/150, 348/156, 340/573.1
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: G07C 9/28 the pass enabling tracking or indicating presence

Abstract

A system for counting and tracking objects of interest within a predefined area with a sensor that captures object data and a data capturing device that receives subset data to produce reports that provide information related to a time, geographic, behavioral, or demographic dimension.

Claims

1. A method for counting and tracking defined objects and anonymous objects, comprising the steps of:

capturing object data with a sensor, wherein the object data is associated with both defined objects and anonymous objects that pass an entrance;

receiving subset data with a data capturing device, wherein the subset data is associated with defined objects and includes a unique identifier, an entry time, an exit time, and location data for each defined object;

receiving object data and subset data at a counting system;

identifying matching object and subset data, wherein matches are representative of the defined objects;

counting the defined objects and the anonymous objects;

tracking the defined objects and the anonymous objects;

associating a location of a defined object with a predefined area;

generating path data by plotting X and Y coordinates for the defined object within the predefined area at sequential time periods; and

receiving data from another independent system regarding characteristics of the object.

2. The method of claim 1, wherein the object data includes at least a starting frame, an ending frame, and a direction.

3. The method of claim 1, wherein the other independent system is selected from the group consisting of point of sale systems, loyalty rewards systems, point of sale trigger information, and mechanical turks.

4. The method of claim 1, further including the steps of converting the subset data into sequence records and converting the object data into track records.

5. The method of claim 4, further including the step of creating a sequence array of all of the sequence records, wherein a sequence record comprises:

(a) a unique ID, which is an unsigned integer associated with a mobile handset, a telephone number associated with the mobile handset or any other unique number, character, or combination thereof, associated with object;

(b) a start time, which may consist of information indicative of a time when the object was first detected within a coverage area;

(c) an end time, which may consist of information indicative of a time when the object was last detected within the coverage area of the data capturing device; and

(d) an array of references to all tracks that overlap a particular sequence record.

6. The method of claim 5, further including the step of creating a track array of all of the track records, wherein a track record comprises:

(a) a counter, which is a unique ID representative of a sensor associated with an object;

(b) a direction, which may consist of information that is representative of the direction of movement for the object;

(c) a start time, which may consist of information indicative of a time when the object was first detected within a field of view of the sensor;

(d) an end time, which may consist of information indicative of a time when the object left the field of view of the sensor; and

(e) a handset ID, which identifies a mobile handset associated with a particular object.

7. The method of claim 1, further including the step of using Z coordinates to further track objects within a predefined area defined by multiple floors.

8. The method of claim 1, further including the step of using the object data and subset data to generate reports showing at least one of the number of objects within a predefined area, a number of objects within the predefined area during a specific time period, a number of predefined areas that were visited by an object, or dwell times for one or more predefined areas.

9. The method of claim 1, further including the step of using path data, object data, and subset data to aggregate the most common paths taken by objects and correlate path data information with dwell times.

10. The method of claim 9, further including the step of generating a report that shows at least one of the following:

(a) the most common paths that objects take in a store, including corresponding dwell times;

(b) changes in shopping patterns by time period or season; and

(c) traffic patterns for use by store security or HVAC systems in increasing or decreasing resources at particular times.

11. A method for counting and tracking defined objects and anonymous objects, comprising the steps of:

capturing object data with a sensor, wherein the object data is associated with both defined objects and anonymous objects that pass an entrance;

receiving subset data with a data capturing device, wherein the subset data is associated with defined objects and includes a unique identifier, an entry time, an exit time, and location data for each defined object;

receiving object data and subset data at a counting system;

identifying matching object and subset data, wherein matches are representative of the defined objects;

counting the defined objects and the anonymous objects;

tracking the defined objects and the anonymous objects;

associating a location of a defined object with a predefined area;

generating path data by plotting X and Y coordinates for the defined object within the predefined area at sequential time periods; and

receiving location data at the capturing device, wherein the location data is received from tracking technology that detects signals emitted from one of the group consisting of mobile handsets, membership cards, employee badges, rail tickets, air tickets, rental car keys, hotel keys, store-sponsored credit cards, debit cards, or loyalty reward cards and wherein the signals include T-IMSI, CDMA, or Wi-Fi signals.

12. The method of claim 11, further including the step of receiving data from another independent system regarding characteristics of the object, wherein the other independent system is selected from the group consisting of point of sale systems, loyalty rewards systems, point of sale trigger information, and mechanical turks.

13. The method of claim 11, further including the step of using Z coordinates to further track objects within a predefined area defined by multiple floors.

14. A method for counting and tracking defined objects and anonymous objects, comprising the steps of:

capturing object data with a sensor, wherein the object data is associated with both defined objects and anonymous objects that pass an entrance;

receiving subset data with a data capturing device, wherein the subset data is associated with defined objects and includes a unique identifier, an entry time, an exit time, and location data for each defined object;

receiving object data and subset data at a counting system;

identifying matching object and subset data, wherein matches are representative of the defined objects;

counting the defined objects and the anonymous objects;

tracking the defined objects and the anonymous objects;

associating a location of a defined object with a predefined area;

generating path data by plotting X and Y coordinates for the defined object within the predefined area at sequential time periods; and

generating a conversion rate by (a) loading transaction data related to transactions performed, (b) loading traffic data including a traffic count and sorting by time periods, and (c) dividing the transactions by the traffic counts for the time periods.

15. The method of claim 14, wherein the transaction data includes a sales amount, the number of items purchases, the specific items purchased, the data and time of the transaction, the register used for the transaction, and the sales associate that completed the transaction.

16. The method of claim 14, further including the step of generating a report showing comparisons between purchasers and non-purchasers based on at least one of the dwell times, the predefined areas, or the time periods.

17. A method for counting and tracking defined objects and anonymous objects, comprising the steps of:

capturing object data with a sensor, wherein the object data is associated with both defined objects and anonymous objects that pass an entrance;

receiving subset data with a data capturing device, wherein the subset data is associated with defined objects and includes a unique identifier, an entry time, an exit time, and location data for each defined object;

receiving object data and subset data at a counting system;

identifying matching object and subset data, wherein matches are representative of the defined objects;

counting the defined objects and the anonymous objects;

tracking the defined objects and the anonymous objects;

associating a location of a defined object with a predefined area;

generating path data by plotting X and Y coordinates for the defined object within the predefined area at sequential time periods;

defining predefined areas and generating track records for an object including an enter time and an exit time for each predefined area; and

determining a dwell time for an object within a predefined area by subtracting the exit time for that predetermined area from the enter time for that predetermined area.

18. The method of claim 17, further including the step of using Z coordinates to further track objects within a predefined area defined by multiple floors.

19. The method of claim 17, further including the step of using the object data and subset data to generate reports showing at least one of the number of objects within a predefined area, a number of objects within the predefined area during a specific time period, a number of predefined areas that were visited by an object, or dwell times for one or more predefined areas.

20. The method of claim 17, further including the step of using path data, object data, and subset data to aggregate the most common paths taken by objects and correlate path data information with dwell times.