Method and system for recognizing the intentions of shoppers in retail aisles based on their trajectories
First Claim
1. A method for determining a shopper'"'"'s intention in an aisle, based on the shopper'"'"'s trajectory, comprising the following steps of:
- a) providing a hardware comprising at least a camera, a plurality of machine learning-based human body detectors or a multi-hypothesis tracker, a computer processor and a set of computer vision algorithms wherein the camera captures a plurality of images, and the plurality of machine learning-based human body detectors or a multi-hypothesis tracker detects a set of people across the plurality of images, and wherein the computer processor tracks the shopper trajectories of the set of people, extracts shopper trajectory features, and analyzes the shopper trajectory features for shopper behaviors, wherein the hardware determines the shopper'"'"'s intention in the aisle by the steps of;
b) setting up a shopper behavior model,wherein the shopper behavior model consists of a relationship between shopper trajectory dynamics and atomic behaviors,wherein the shopper trajectory dynamics comprise position, speed, and orientation,c) detecting and tracking the shopper from input images captured by the camera to generate a trajectory of the shopper,d) converting the shopper trajectory into a lateral trajectory wherein the lateral trajectory into y and t coordinates, wherein y coordinates are in the direction perpendicular to an aisle and t coordinates are instances of time,e) extracting shopper trajectory features from the trajectory, wherein the shopper trajectory features include spatiotemporal dimensions, andf) analyzing the shopper trajectory features of the shopper'"'"'s trajectory based on the shopper behavior model to determine the shopper'"'"'s behavior state,wherein the lateral direction is a direction perpendicular to the aisle direction, andwherein the shopper trajectory features comprise lateral position, lateral position histogram, motion orientation, and motion speed.
17 Assignments
0 Petitions
Accused Products
Abstract
The present invention is a method and system for automatically recognizing which products a shopper intends to find or purchase based on the shopper'"'"'s trajectory in a retail aisle. First, the system detects and tracks the person to generate the trajectory of the shopper. Then some of the dynamic features are extracted from the shopper trajectory. The shopper trajectory features of a given trajectory are typically the positions, the motion orientations, and speeds at each point of the trajectory. A shopper behavior model is designed based on some of the primitive actions of shoppers. The last step of the method is to analyze a given shopper trajectory to estimate the shopper'"'"'s intention. The step either utilizes decision rules based on the extracted shopper trajectory features, or utilizes a trained Hidden Markov Model, to estimate the progression of the primitive actions from the trajectory. The decoded progression of the shopper behavior states is then interpreted to finally determine the shopper'"'"'s intention.
-
Citations
16 Claims
-
1. A method for determining a shopper'"'"'s intention in an aisle, based on the shopper'"'"'s trajectory, comprising the following steps of:
-
a) providing a hardware comprising at least a camera, a plurality of machine learning-based human body detectors or a multi-hypothesis tracker, a computer processor and a set of computer vision algorithms wherein the camera captures a plurality of images, and the plurality of machine learning-based human body detectors or a multi-hypothesis tracker detects a set of people across the plurality of images, and wherein the computer processor tracks the shopper trajectories of the set of people, extracts shopper trajectory features, and analyzes the shopper trajectory features for shopper behaviors, wherein the hardware determines the shopper'"'"'s intention in the aisle by the steps of; b) setting up a shopper behavior model, wherein the shopper behavior model consists of a relationship between shopper trajectory dynamics and atomic behaviors, wherein the shopper trajectory dynamics comprise position, speed, and orientation, c) detecting and tracking the shopper from input images captured by the camera to generate a trajectory of the shopper, d) converting the shopper trajectory into a lateral trajectory wherein the lateral trajectory into y and t coordinates, wherein y coordinates are in the direction perpendicular to an aisle and t coordinates are instances of time, e) extracting shopper trajectory features from the trajectory, wherein the shopper trajectory features include spatiotemporal dimensions, and f) analyzing the shopper trajectory features of the shopper'"'"'s trajectory based on the shopper behavior model to determine the shopper'"'"'s behavior state, wherein the lateral direction is a direction perpendicular to the aisle direction, and wherein the shopper trajectory features comprise lateral position, lateral position histogram, motion orientation, and motion speed. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. An apparatus for determining a shopper'"'"'s intention in an aisle, based on the shopper'"'"'s trajectory, comprising:
-
a) providing a hardware comprising at least a camera, a plurality of machine learning-based human body detectors or a multi-hypothesis tracker, a computer processor and a set of computer vision algorithms wherein the camera captures a plurality of images, and the plurality of machine learning-based human body detectors or a multi-hypothesis tracker detects a set of people across the plurality of images, and wherein the computer processor tracks the shopper trajectories of the set of people, extracts shopper trajectory features, and analyzes the shopper trajectory features for shopper behaviors, wherein the hardware determines the shopper'"'"'s intention in the aisle by the steps of; b) setting up a shopper behavior model, wherein the shopper behavior model consists of a relationship between shopper trajectory dynamics and atomic behaviors, wherein the shopper trajectory dynamics comprise position, speed, and orientation, c) detecting and tracking the shopper from input images captured by the camera to generate a trajectory of the shopper, d) converting the shopper trajectory into a lateral trajectory wherein the lateral trajectory into y and t coordinates, wherein y coordinates are in the direction perpendicular to an aisle and t coordinates are instances of time, e) extracting shopper trajectory features from the trajectory, wherein the shopper trajectory features include spatiotemporal dimensions, and f) analyzing the shopper trajectory features of the shopper'"'"'s trajectory based on the shopper behavior model to determine the shopper'"'"'s behavior state, wherein the lateral direction is a direction perpendicular to the aisle direction, and wherein the shopper trajectory features comprise lateral position, lateral position histogram, motion orientation, and motion speed. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
Specification