Method and system for measuring shopper response to products based on behavior and facial expression
First Claim
1. A method for determining a person'"'"'s response to a retail element, based on the person'"'"'s facial expression and shopping behavior, comprising the following steps of:
- a) detecting and tracking a face from first input images captured by at least a first means for capturing images, estimating two-dimensional and three-dimensional poses of the face, and localizing facial features, using at least a control and processing system,b) estimating gaze direction of the person using the two-dimensional and three-dimensional poses and positions of the facial features and changes in affective state of the person by extracting emotion-sensitive features, and recognizing a demographic category of the person,c) detecting and tracking the person from second input images captured by at least a second means for capturing images, producing a trajectory of the person, and estimating body orientation, using the control and processing system,d) identifying the shopping behaviors of the person toward the retail element, utilizing position and the body orientation of the person relative to the retail element, ande) determining intermediate responses and end response of the person to the retail element by analyzing the changes in affective states and interest, in the context of the shopping behavior and the demographics category of the person,wherein the first means for capturing images and the second means for capturing images are connected to the control and processing system via at least a means for video interface, and wherein the shopping behaviors include showing interest, engagement, interaction, or purchasing.
16 Assignments
0 Petitions
Accused Products
Abstract
The present invention is a method and system for measuring human response to retail elements, based on the shopper'"'"'s facial expressions and behaviors. From a facial image sequence, the facial geometry—facial pose and facial feature positions—is estimated to facilitate the recognition of facial expressions, gaze, and demographic categories. The recognized facial expression is translated into an affective state of the shopper and the gaze is translated into the target and the level of interest of the shopper. The body image sequence is processed to identify the shopper'"'"'s interaction with a given retail element—such as a product, a brand, or a category. The dynamic changes of the affective state and the interest toward the retail element measured from facial image sequence is analyzed in the context of the recognized shopper'"'"'s interaction with the retail element and the demographic categories, to estimate both the shopper'"'"'s changes in attitude toward the retail element and the end response—such as a purchase decision or a product rating.
295 Citations
42 Claims
-
1. A method for determining a person'"'"'s response to a retail element, based on the person'"'"'s facial expression and shopping behavior, comprising the following steps of:
-
a) detecting and tracking a face from first input images captured by at least a first means for capturing images, estimating two-dimensional and three-dimensional poses of the face, and localizing facial features, using at least a control and processing system, b) estimating gaze direction of the person using the two-dimensional and three-dimensional poses and positions of the facial features and changes in affective state of the person by extracting emotion-sensitive features, and recognizing a demographic category of the person, c) detecting and tracking the person from second input images captured by at least a second means for capturing images, producing a trajectory of the person, and estimating body orientation, using the control and processing system, d) identifying the shopping behaviors of the person toward the retail element, utilizing position and the body orientation of the person relative to the retail element, and e) determining intermediate responses and end response of the person to the retail element by analyzing the changes in affective states and interest, in the context of the shopping behavior and the demographics category of the person, wherein the first means for capturing images and the second means for capturing images are connected to the control and processing system via at least a means for video interface, and wherein the shopping behaviors include showing interest, engagement, interaction, or purchasing. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. An apparatus for determining a person'"'"'s response to a retail element, based on the person'"'"'s facial expression and shopping behavior, comprising:
-
a) at least a first means for capturing images that captures first input images, b) at least a control and processing system that performs the following steps of; detecting and tracking a face from the first input images, estimating two-dimensional and three-dimensional pose of the face, localizing facial features, estimating gaze direction of the person using the two-dimensional and three-dimensional poses and the positions of the facial features and changes in affective state of the person by extracting emotion-sensitive features, c) at least a second means for capturing images that captures second input images, d) the control and processing system that performs the following steps of; detecting and tracking the person from the second input images producing a trajectory of the person, and estimating body orientation, identifying the shopping behaviors of the person toward the retail element, utilizing position and the body orientation of the person relative to the retail element, and determining intermediate responses and end response of the person to the retail element by analyzing the changes in affective states and interest, in the context of the shopping behavior, and e) at least a means for video interface that connects the first means for capturing images and the second means for capturing images to the control and processing system, wherein the shopping behaviors include showing interest, engagement, interaction, or purchasing. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42)
-
Specification