Method and system for analyzing shopping behavior in a store by associating RFID data with video-based behavior and segmentation data

• US 8,380,558 B1
• Filed: 12/06/2007
• Issued: 02/19/2013
• Est. Priority Date: 12/21/2006
• Status: Active Grant

• Current Assignee: Videomining Corporation
• Original Assignee: Videomining Corporation
• Inventors: Mummareddy, Satish, Hershey, Jeff, Jung, Namsoon, Sharma, Rajeev
• Primary Examiner(s): Backer, Firmin
• Assistant Examiner(s): Chambers, Michael A
• Application Number:
  US11/999,717
• Time in Patent Office: 1,902 Days
• Field of Search: 705/10, 705/14, 705/7.29, 348/169
• US Class Current: 705/7.29
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

The present invention is a method and system for analyzing shopping behavior by associating RFID data, such as tracking data by the RFID tag identifications, with video-based behavior and segmentation data, such as behavior analysis and demographic composition analysis of the customers, utilizing a plurality of means for sensing and using RFID tags, a plurality of means for capturing images, and a plurality of computer vision technologies. In the present invention, the association can further comprise the association of the RFID with the transaction data or any time-based measurement in the retail space. The analyzed shopping behavior in the present invention helps people to better understand business elements in a retail space. It is one of the objectives of the present invention to provide an automatic video-based segmentation of customers in the association with the RFID based tracking of the customers, based on a novel usage of a plurality of means for capturing images and a plurality of computer vision technologies on the captured visual information of the people in the retail space. The plurality of computer vision technologies can comprise face detection, person tracking, body parts detection, and demographic classification of the people, on the captured visual information of the people in the retail space.

Claims

1. A method for analyzing shopping behavior of a plurality of persons in a retail space by associating RFID data with video-based behavior analysis data using a computer vision based behavior analysis and segmentation technology performed on a computer, comprising the following steps of:

a) capturing a plurality of input images of the plurality of persons by a plurality of means for capturing images,

b) processing the plurality of input images in order to analyze the behavior of the plurality of persons in each field of view of the plurality of means for capturing images, by applying a plurality of computer vision technologies on the plurality of input images, wherein the plurality of computer vision technologies comprise face detection, person tracking, body parts detection, and demographic classification,

c) sensing RFID tag data utilizing a plurality of means for sensing and using RFID tag,

d) associating the video-based behavior analysis data of the plurality of persons with the RFID tag data, and

e) correlating a plurality of associations for each person, in the plurality of persons, in different locations and different windows of time to have an entire tracking of the shopping path for the person,

wherein the correlation integrates fragmented video tracking segments into the entire tracking of the shopping path from one location to another location with a help of proximity detection using the associated RFID tag data, and

wherein the RFID tag data comprise time, location, and identification of the RFID tag at the time they were sensed by the plurality of means for sensing and using RFID tag.

2. The method according to claim 1, wherein the method further comprises a step of associating segmentation data of the plurality of persons with the RFID tag data and the video-based behavior analysis data,

whereby the segmentation data comprises demographic classification of the plurality of persons.

3. The method according to claim 1, wherein the method further comprises a step of associating a transaction data in the retail space with the RFID tag data, the video-based behavior analysis data, and a segmentation data of the plurality of persons,

whereby the segmentation data comprises demographic classification of the plurality of persons.

4. The method according to claim 1, wherein the method further comprises a step of associating a temporal data with the RFID tag data, the video-based behavior analysis data, and a segmentation data of the plurality of persons, whereby the temporal data comprises loyalty card data or survey data.

5. The method according to claim 1, wherein the method further comprises a step of analyzing a plurality of sections in the retail space based on the association of the video-based behavior analysis data with the RFID tag data, which produces quantitative measurement per section, dominant path measurement among the sections, section correlation measurement for optimal distance between the sections, and section sequence measurement,

wherein a section is defined as a logical entity, including a group of products, a group of product types, space, areas in a store, display of a group of products, or department with similar relevance.

6. The method according to claim 5, wherein the method further comprises a step of analyzing demographic characteristics of the plurality of persons with regard to the sections in the retail space,

whereby the demographic characteristics comprise age, gender, and ethnicity information.

7. The method according to claim 1, wherein the method further comprises a step of analyzing details of aisle dynamics within area covered by vision-based behavior analysis,

whereby the aisle dynamics comprise traffic pass-through and u-turn breakout, and aisle penetration.

8. An apparatus for analyzing shopping behavior of a plurality of persons in a retail space by associating RFID data with video-based behavior analysis data using a computer vision based behavior analysis and segmentation technology performed on a computer, comprising:

a) means for capturing a plurality of input images of the plurality of persons by a plurality of means for capturing images,

b) means for processing the plurality of input images in order to analyze the behavior of the plurality of persons in each field of view of the plurality of means for capturing images, by applying a plurality of computer vision technologies on the plurality of input images, wherein the plurality of computer vision technologies comprise face detection, person tracking, body parts detection, and demographic classification,

c) means for sensing RFID tag data utilizing a plurality of means for sensing and using RFID tag,

d) means for associating the video-based behavior analysis data of the plurality of persons with the RFID tag data, and

e) means for correlating a plurality of associations for each person, in the plurality of persons, in different locations and different windows of time to have an entire tracking of the shopping path for the person,

wherein the correlation integrates fragmented video tracking segments into the entire tracking of the shopping path from one location to another location with a help of proximity detection using the associated RFID tag data, and

wherein the RFID tag data comprise time, location, and identification of the RFID tag at the time they were sensed by the plurality of means for sensing and using RFID tag.

9. The apparatus according to claim 8, wherein the apparatus further comprises means for associating segmentation data of the plurality of persons with the RFID tag data and the video-based behavior analysis data, whereby the segmentation data comprises demographic classification of the plurality of persons.

10. The apparatus according to claim 8, wherein the apparatus further comprises means for associating a transaction data in the retail space with the RFID tag data, the video-based behavior analysis data, and a segmentation data of the plurality of persons,

whereby the segmentation data comprises demographic classification of the plurality of persons.

11. The apparatus according to claim 8, wherein the apparatus further comprises means for associating a temporal data with the RFID tag data, the video-based behavior analysis data, and a segmentation data of the plurality of persons, whereby the temporal data comprises loyalty card data or survey data.

12. The apparatus according to claim 8, wherein the apparatus further comprises means for analyzing a plurality of sections in the retail space based on the association of the video-based behavior analysis data with the RFID tag data, which produces quantitative measurement per section, dominant path measurement among the sections, section correlation measurement for optimal distance between the sections, and section sequence measurement,

wherein a section is defined as a logical entity, including a group of products, a group of product types, space, areas in a store, display of a group of products, or department with similar relevance.

13. The apparatus according to claim 12, wherein the apparatus further comprises means for analyzing demographic characteristics of the plurality of persons with regard to the sections in the retail space,

whereby the demographic characteristics comprise age, gender, and ethnicity information.

14. The apparatus according to claim 8, wherein the apparatus further comprises means for analyzing details of aisle dynamics within area covered by vision-based behavior analysis,

whereby the aisle dynamics comprise traffic pass-through and u-turn breakout, and aisle penetration.