Method and system for narrowcasting based on automatic analysis of customer behavior in a retail store

• US 7,930,204 B1
• Filed: 07/20/2007
• Issued: 04/19/2011
• Est. Priority Date: 07/25/2006
• Status: Active Grant

• Current Assignee: Nytell Software LLC (Intellectual Ventures LLC)
• Original Assignee: Videomining Corporation
• Inventors: Sharma, Rajeev, Mummareddy, Satish, Hershey, Jeff, Kamat, Akhil, Ahlers, Chris
• Primary Examiner(s): Jarrett, Scott L
• Application Number:
  US11/880,421
• Time in Patent Office: 1,369 Days
• Field of Search: 705/10
• US Class Current: 705/7.32
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0203 Market surveys; Market polls

Abstract

The present invention is a method and system for automatically changing the advertisement contents on a means for displaying the contents based on a customer'"'"'s behavior or a group of customers'"'"' behavior in a retail store. The analysis of behavioral pattern is performed automatically and in real-time based on the visual information from the shopping and walkthrough history of a customer or a group of customers, using arrays of sensing devices, such as a plurality of means for capturing images, and a plurality of computer vision technologies on the visual information.

Claims

1. A method of narrowcasting advertisement contents on a means for displaying output based on an automatic and real-time behavior analysis of a customer or a group of customers in a retail store, comprising the following steps of:

a) capturing a plurality of input video images of the customer or the group of customers in the retail store using a plurality of sensing devices connected in an array of sensing devices to capture video images wherein the array of sensing devices are connected to a first means for control and processing,

b) generating individual tracks from the captured plurality of input video images in the first means for control and processing, the tracks representing a shopping and walkthrough history of where the customer or the group of customers have been in the retail store,

c) joining the individual tracks from the first means for control and processing in a second means for control and processing, wherein the tracks are joined using a Bayesian Belief Network,

d) automatically analyzing, in real-time, behavior pattern of the customer or the group of customers based on visual information from the joined tracks in the second means for control and processing,

e) customizing the advertisement contents according to the joined tracks and the behavior pattern from the behavior analysis, and

f) narrowcasting the customized advertisement contents on means for displaying output to the customer or the group of customers in the retail store.

2. The method according to claim 1, wherein the method further comprises a step of extracting the interest of the customer or the group of customers by measuring the time spent in a predefined area within the store, wherein the time is calculated based on an analysis of the joined tracks for temporal pattern attribute.

3. The method according to claim 1, wherein the method further comprises a step of analyzing and extracting the behavioral pattern of interest either at an individual customer level or at a group of customers level, by tracking each customer in the retail store through the array of sensing devices, wherein the tracking of each customer is processed based on joined tracks from videos of the plurality of means for capturing images, and wherein the interest at a group of customers level is decided by the majority of the plurality of behavior analysis results after aggregating the tracking for each customer.

4. The method according to claim 1, wherein the method further comprises a step of utilizing a temporal pattern attribute for the behavior analysis, wherein the temporal pattern attribute is calculated based on an analysis of the joined tracks, and wherein the temporal pattern attribute comprises time spent in each section of the store or the time of the day.

5. The method according to claim 1, wherein the method further comprises a step of utilizing a spatial preference pattern attribute for the behavior analysis, wherein the spatial preference pattern attribute is calculated based on an analysis of the joined tracks, and wherein the spatial preference pattern attribute comprises a path history and a preference in a path vs. other paths.

6. The method according to claim 1, wherein the method further comprises a step of utilizing a frequency pattern attribute for the behavior analysis, wherein the frequency pattern attribute is calculated based on an analysis of the joined tracks, and wherein the frequency pattern attribute comprises frequency of visiting certain sections multiple times or more times than other sections.

7. The method according to claim 1, wherein the method further comprises a step of utilizing a relational pattern attribute for the behavior analysis, wherein the relational pattern attribute is calculated based on an analysis of the joined tracks, and wherein the relational pattern attribute comprises relative effect of one path vs. another, relative effect of a path when there is interior decoration modification or floor-layout change, or relationship between the path and amenities in the store, including a bathroom, diaper changing room, water fountain, telephone booth, and customer service.

8. The method according to claim 1, wherein the method further comprises a step of utilizing a special event pattern attribute for the behavior analysis, wherein the special event pattern attribute is calculated based on an analysis of the joined tracks, and wherein the special event pattern attribute comprises change in the path due to the special event of the day.

9. The method according to claim 1, wherein the method further comprises a step of providing promotional messages to the customer in addition to the targeted advertisement contents based on the analysis of the behavioral pattern, whereby the promotional messages further encourage the customer to engage in certain transactional activity.

10. The method according to claim 1, wherein the method further comprises a step of playing the targeted advertisement contents on the means for displaying output installed in a remote location that is separated from the location of the plurality of means for capturing images or products that are promoted by the targeted advertisement, wherein the behavior analysis of the customer, based on the joined tracks, is used for the targeted advertisement, and whereby the targeted advertisement delivery looks more coincidental than deliberately arranged when the customer arrives at the location of the means for displaying output for the targeted advertisement contents along the shopping path.

11. An apparatus of narrowcasting advertisement contents on a means for displaying output based on an automatic and real-time behavior analysis of a customer or a group of customers in a retail store, comprising:

a) means for capturing a plurality of input video images of the customer or the group of customers in the retail store using a plurality of sensing devices connected in an array of sensing devices to capture video images wherein the array of sensing devices are connected to a first means for control and processing,

b) the first means for control and processing to generate individual tracks from the captured plurality of input video images, the tracks representing a shopping and walkthrough history of where the customer or the group of customers have been in the retail store,

c) second means for control and processing to join the individual tracks from the first means for control and processing, wherein the tracks are joined using a Bayesian Belief Network, and to automatically analyze, in real-time, behavior pattern of the customer or the group of customers based on visual information from the joined tracks in the second means for control and processing,

d) means for customizing the advertisement contents according to the joined tracks and the behavior pattern from the behavior analysis, and

e) means for narrowcasting the customized advertisement contents on means for displaying output to the customer or the group of customers in the retail store.

12. The apparatus according to claim 11, wherein the apparatus further comprises means for extracting the interest of the customer or the group of customers by measuring the time spent in a predefined area within the store, wherein the time is calculated based on an analysis of the joined tracks for temporal pattern attribute.

13. The apparatus according to claim 11, wherein the apparatus further comprises means for analyzing and extracting the behavioral pattern of interest either at an individual customer level or at a group of customers level, by tracking each customer in the retail store through the array of sensing devices, wherein the tracking of each customer is processed based on joined tracks from videos of the plurality of means for capturing images, and wherein the interest at a group of customers level is decided by the majority of the plurality of behavior analysis results after aggregating the tracking for each customer.

14. The apparatus according to claim 11, wherein the apparatus further comprises means for utilizing a temporal pattern attribute for the behavior analysis, wherein the temporal pattern attribute is calculated based on an analysis of the joined tracks, and wherein the temporal pattern attribute comprises time spent in each section of the store or the time of the day.

15. The apparatus according to claim 11, wherein the apparatus further comprises means for utilizing a spatial preference pattern attribute for the behavior analysis, wherein the spatial preference pattern attribute is calculated based on an analysis of the joined tracks, and wherein the spatial preference pattern attribute comprises a path history and a preference in a path vs. other paths.

16. The apparatus according to claim 11, wherein the apparatus further comprises means for utilizing a frequency pattern attribute for the behavior analysis, wherein the frequency pattern attribute is calculated based on an analysis of the joined tracks, and wherein the frequency pattern attribute comprises frequency of visiting certain sections multiple times or more times than other sections.

17. The apparatus according to claim 11, wherein the apparatus further comprises means for utilizing a relational pattern attribute for the behavior analysis, wherein the relational pattern attribute is calculated based on an analysis of the joined tracks, and wherein the relational pattern attribute comprises relative effect of one path vs. another, relative effect of a path when there is interior decoration modification or floor-layout change, or relationship between the path and amenities in the store, including a bathroom, diaper changing room, water fountain, telephone booth, and customer service.

18. The apparatus according to claim 11, wherein the apparatus further comprises means for utilizing a special event pattern attribute for the behavior analysis, wherein the special event pattern attribute is calculated based on an analysis of the joined tracks, and wherein the special event pattern attribute comprises change in the path due to the special event of the day.

19. The apparatus according to claim 11, wherein the apparatus further comprises means for providing promotional messages to the customer in addition to the targeted advertisement contents based on the analysis of the behavioral pattern, whereby the promotional messages further encourage the customer to engage in certain transactional activity.

20. The apparatus according to claim 11, wherein the apparatus further comprises means for playing the targeted advertisement contents on the means for displaying output installed in a remote location that is separated from the location of the plurality of means for capturing images or products that are promoted by the targeted advertisement, wherein the behavior analysis of the customer, based on the joined tracks, is used for the targeted advertisement, and whereby the targeted advertisement delivery looks more coincidental than deliberately arranged when the customer arrives at the location of the means for displaying output for the targeted advertisement contents along the shopping path.