Cross-channel in-store shopper behavior analysis

• US 10,262,331 B1
• Filed: 01/29/2016
• Issued: 04/16/2019
• Est. Priority Date: 01/29/2016
• Status: Active Grant

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Sharma, Rajeev, Shin, Joonhwa, Yoon, Youngrock R, Kim, Donghun
• Primary Examiner(s): Stamber, Eric W
• Assistant Examiner(s): Waesco, Joseph M
• Application Number:
  US15/011,442
• Time in Patent Office: 1,173 Days
• Field of Search: None
• CPC Class Codes: G06Q 30/0205 Location or geographical consideration
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/33 for indoor environments, e.g. buildings

Abstract

A method and system for cross-channel shopper behavior analysis. Tracking individual shopper behavior across many retail locations (i.e., across multiple channels) can be extremely valuable for product manufacturers and retails. A configuration of vision sensors and mobile access points can be used to detect and track shoppers as they travel a retail environment, forming a trajectory. The trajectory data can then be associated with Point of Sale data to form a full set of shopper behavior data. The shopper'"'"'s data can then be aggregated for multiple visits to many retail locations in a geographic area. The aggregated shopper data can be filtered using application-specific criteria for further analysis.

Claims

1. A method for tracking of shopper behavior across a plurality of locations using at least a processor to perform the steps of:

a. tracking in-store shopper behavior at a plurality of retail locations using a Shopper Behavior Tracker module, wherein the Shopper Behavior Tracker comprises

i. detecting at least one person in at least one of the plurality of retail locations using an At-Door Shopper Detector module, wherein the At-Door Shopper Detector module further comprises extracting a set of features and estimating demographics information for the at least one person detected,

ii. tracking the movements of said at least one person at said at least one of the plurality of retail locations location, forming at least one trajectory using a Multi-modal Shopper Tracker module,

iii. associating the at least one trajectory with shopper segment data and the at least one trajectory with Point-of-Sale (PoS) data to generate shopper profile data using a Multi-modal Shopper Data Associator module, wherein the Multi-modal Shopper Data Associator module comprises either;

1. detecting the completion of at least one mobile trajectory,

2. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data contains at least one vision trajectory,

3. performing matching between the at least one vision trajectory and the at least one mobile trajectory,

4. fusing vision trajectories that are associated with the same target at a given time frame using measurement fusion,

5. combining the fused vision trajectories with the mobile trajectory to complete missing segments in the vision trajectories,

or;

1. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data was created at a time matching a time when the tracked shopper entered or exited the store,

2. computing a similarity of visual features and shopper data, wherein the visual features comprise face and body features, and the shopper data comprise estimated demographics data,

3. determining the best match of the shopper profile data with the estimated demographics data, and

4. combining the best-matched data to create a new shopper profile data,

or,

1. retrieving a set of shopper profile data from the in-store shopper database,

2. retrieving a set of PoS data from a PoS database, wherein the PoS data is created at a time matching a time when the tracked shopper exited the store,

3. identifying categories of items in the retrieved PoS data using a pre-learned item-category mapping table,

4. mapping the location of the categories of items in the store using store layout and planogram information,

5. computing a probabilistic or deterministic measure between the shopper profile data and the PoS data based on the likelihood that the PoS data corresponds to the shopper profile data,

6. determining the best match between the shopper profile data and the PoS data, and

7. associating the instances of the best matching shopper profile data with the PoS data,

iv. populating an in-store shopper database with the shopper profile data, and

v. identifying a revisiting or cross-visiting shopper using shopper profile data from the Multi-modal Shopper Data Associator module and populating the cross-channel shopper database with the result, using a Revisiting and Cross-visiting Shopper Identifier module,

b. populating a cross-channel shopper database with a set of shopper profile data obtained from the Shopper Behavior Tracker module,

c. aggregating shopper profile data for each tracked shopper across multiple locations using the cross-channel shopper database, and

d. profiling shopper behavior using a Shopper Behavior Profiler module.

2. The method of claim 1, wherein the Shopper Behavior Profiler further comprises estimating derivable metrics for characterizing shopper behavior across multiple locations.

3. The method of claim 1, wherein the Shopper Behavior Profiler further comprises:

a. deriving metrics for each shopper profile in the cross-channel shopper database, wherein the derivable metrics comprise brand loyalty, shopping style, store loyalty, shopper channel distribution, shopper channel preference, household income, household size, healthy food preference, and organic food preference, and

b. updating a behavioral segment data section of the cross-channel shopper database.

4. The method of claim 1, further comprising using a Revisiting and Cross-visiting Shopper Identifier module for:

a. determining whether a first set of shopper profile data already exists in the cross-channel shopper database,

b. if a match is found,

i. retrieving the existing shopper profile data from the cross-channel shopper database, and

ii. merging the first set of shopper profile data with the existing shopper profile data,

c. if a match is not found,

i. creating a new shopper profile data, and

d. updating the cross-channel shopper database.

5. The method of claim 1, further comprising using the cross-channel shopper database, the Shopper Behavior Profiler module, and an Anonymous Panel Creator module for creating an anonymous shopper panel for at least one application using a set of application-specific criteria.

6. The method of claim 1, wherein the Multi-modal Shopper Tracker module further comprises:

a. capturing a set of images using at least a vision sensor,

b. detecting an object of interest in the set of images and determining the time and location of said object,

c. receiving a data stream from at least a mobile signal sensor,

d. sorting received data from mobile signal sensors from the same source device to bundle them into a single set,

e. calculating a location of the source device using a trilateration method, and

f. tracking the source device using shopper profile data.

7. A system for tracking of shopper behavior across a plurality of locations using at least a processor to perform the steps of:

a. tracking in-store shopper behavior at a plurality of retail locations using a Shopper Behavior Tracker module, wherein the Shopper Behavior Tracker comprises

i. detecting at least one person in at least one of the plurality of retail locations using an At-Door Shopper Detector module, wherein the At-Door Shopper Detector module further comprises extracting a set of features and estimating demographics information for the at least one person detected,

ii. tracking the movements of said at least one person at said at least one of the plurality of retail locations location, forming at least one trajectory using a Multi-modal Shopper Tracker module,

iii. associating the at least one trajectory with shopper segment data and the at least one trajectory with Point-of-Sale (PoS) data to generate shopper profile data using a Multi-modal Shopper Data Associator module, wherein the Multi-modal Shopper Data Associator module comprises either;

1. detecting the completion of at least one mobile trajectory,

2. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data contains at least one vision trajectory,

3. performing matching between the at least one vision trajectory and the at least one mobile trajectory,

4. fusing vision trajectories that are associated with the same target at a given time frame using measurement fusion,

5. combining the fused vision trajectories with the mobile trajectory to complete missing segments in the vision trajectories,

or;

5. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data was created at a time matching a time when the tracked shopper entered or exited the store,

6. computing a similarity of visual features and shopper data, wherein the visual features comprise face and body features, and the shopper data comprise estimated demographics data,

7. determining the best match of the shopper profile data with the estimated demographics data, and

8. combining the best-matched data to create a new shopper profile data,

or,

8. retrieving a set of shopper profile data from the in-store shopper database,

9. retrieving a set of PoS data from a PoS database, wherein the PoS data is created at a time matching a time when the tracked shopper exited the store,

10. identifying categories of items in the retrieved PoS data using a pre-learned item-category mapping table,

11. mapping the location of the categories of items in the store using store layout and planogram information,

12. computing a probabilistic or deterministic measure between the shopper profile data and the PoS data based on the likelihood that the PoS data corresponds to the shopper profile data,

13. determining the best match between the shopper profile data and the PoS data, and

14. associating the instances of the best matching shopper profile data with the PoS data,

iv. populating an in-store shopper database with the shopper profile data, and

v. identifying a revisiting or cross-visiting shopper using shopper profile data from the Multi-modal Shopper Data Associator module and populating the cross-channel shopper database with the result, using a Revisiting and Cross-visiting Shopper Identifier module,

b. populating a cross-channel shopper database with a set of shopper profile data obtained from the Shopper Behavior Tracker module,

c. aggregating shopper profile data for each tracked shopper across multiple locations using the cross-channel shopper database, and

d. profiling shopper behavior using a Shopper Behavior Profiler module.

8. The system of claim 7, wherein the Shopper Behavior Profiler further comprises estimating derivable metrics for characterizing shopper behavior across multiple locations.

9. The system of claim 7, wherein the Shopper Behavior Profiler further comprises:

a. deriving metrics for each shopper profile in the cross-channel shopper database, wherein the derivable metrics comprise brand loyalty, shopping style, store loyalty, shopper channel distribution, shopper channel preference, household income, household size, healthy food preference, and organic food preference, and

b. updating a behavioral segment data section of the cross-channel shopper database.

10. The system of claim 7, further comprising using a Revisiting and Cross-visiting Shopper Identifier module for:

a. determining whether a first set of shopper profile data already exists in the cross-channel shopper database,

b. if a match is found,

i. retrieving the existing shopper profile data from the cross-channel shopper database, and

ii. merging the first set of shopper profile data with the existing shopper profile data,

c. if a match is not found,

i. creating a new shopper profile data, and

d. updating the cross-channel shopper database.

11. The system of claim 7, further comprising using the cross-channel shopper database, the Shopper Behavior Profiler module, and an Anonymous Panel Creator module for creating an anonymous shopper panel for at least one application using a set of application-specific criteria.

12. The system of claim 7, wherein the Multi-modal Shopper Tracker module further comprises:

a. capturing a set of images using at least a vision sensor,

b. detecting an object of interest in the set of images and determining the time and location of said object,

c. receiving a data stream from at least a mobile signal sensor,

d. sorting received data from mobile signal sensors from the same source device to bundle them into a single set,

e. calculating a location of the source device using a trilateration method, and

f. tracking the source device using shopper profile data.