Automatic detection and aggregation of demographics and behavior of people
First Claim
1. A method for detecting and aggregating demographics and behavior of people, comprising the following steps of:
- a) detecting the people from a plurality of input images in the vicinity of a fixture with at least a visual sensing device using a control and processing system,wherein the visual sensing device is connected to a means for video interface,b) detecting faces in the plurality of input images,c) detecting locations of facial features on the detected faces and estimating facial geometry including position, size, and orientation of the detected faces,d) aligning the detected faces using the facial geometry estimation,e) building appearance models of the people across image frames containing the detected faces, by averaging pixel values of the detected faces, aligned using estimates for the position, size, and orientation of the detected faces,wherein an estimation of the position, size, and orientation is carried out by employing a parallel array of multiple learning machine regressors,f) tracking the detected faces by keeping identities assigned to the detected faces,g) obtaining demographics information and behavior features of the people automatically,h) computing timestamps and location markers for the obtained demographics information and behavior features, andi) combining the obtained demographics information and behavior features with the timestamps and the location markers into a feature vector,wherein the behavior is detected based on an analysis of measurements for the behavior features,wherein the behavior features comprise timestamps, start time and end time of tracks for appearance and disappearance of the detected faces, duration of watching fixtures, counts of the detected faces, counts of attentive faces, viewership, location markers, and facial pose estimates, andwherein the demographics information comprises estimates of gender, age, and ethnicity of the people.
17 Assignments
0 Petitions
Accused Products
Abstract
The present invention is a system and framework for automatically measuring and correlating visual characteristics of people and accumulating the data for the purpose of demographic and behavior analysis. The demographic and behavior characteristics of people are extracted from a sequence of images using techniques from computer vision. The demographic and behavior characteristics are combined with a timestamp and a location marker to provide a feature vector of a person at a particular time at a particular location. These feature vectors are then accumulated and aggregated automatically in order to generate a data set that can be statistically analyzed, data mined and/or queried.
-
Citations
8 Claims
-
1. A method for detecting and aggregating demographics and behavior of people, comprising the following steps of:
-
a) detecting the people from a plurality of input images in the vicinity of a fixture with at least a visual sensing device using a control and processing system, wherein the visual sensing device is connected to a means for video interface, b) detecting faces in the plurality of input images, c) detecting locations of facial features on the detected faces and estimating facial geometry including position, size, and orientation of the detected faces, d) aligning the detected faces using the facial geometry estimation, e) building appearance models of the people across image frames containing the detected faces, by averaging pixel values of the detected faces, aligned using estimates for the position, size, and orientation of the detected faces, wherein an estimation of the position, size, and orientation is carried out by employing a parallel array of multiple learning machine regressors, f) tracking the detected faces by keeping identities assigned to the detected faces, g) obtaining demographics information and behavior features of the people automatically, h) computing timestamps and location markers for the obtained demographics information and behavior features, and i) combining the obtained demographics information and behavior features with the timestamps and the location markers into a feature vector, wherein the behavior is detected based on an analysis of measurements for the behavior features, wherein the behavior features comprise timestamps, start time and end time of tracks for appearance and disappearance of the detected faces, duration of watching fixtures, counts of the detected faces, counts of attentive faces, viewership, location markers, and facial pose estimates, and wherein the demographics information comprises estimates of gender, age, and ethnicity of the people. - View Dependent Claims (2, 3, 4)
-
-
5. An apparatus for detecting and aggregating demographics and behavior of people, comprising:
-
a) at least a control and processing system programmed to perform the following steps of; detecting the people from a plurality of input images in the vicinity of a fixture with at least a visual sensing device, wherein the visual sensing device is connected to a means for video interface, detecting faces in the plurality of input images, detecting locations of facial features on the detected faces and estimating facial geometry including position, size, and orientation of the detected faces, aligning the detected faces using the facial geometry estimation, building appearance models of the people across image frames containing the detected faces, by averaging pixel values of the detected faces, aligned using estimates for the position, size, and orientation of the detected faces, wherein an estimation of the position, size, and orientation is carried out by employing a parallel array of multiple learning machine regressors, tracking the detected faces by keeping identities assigned to the detected faces, obtaining demographics information and behavior features of the people automatically, computing timestamps and location markers for the obtained demographics information and behavior features, and combining the obtained demographics information and behavior features with the timestamps and the location markers into a feature vector, and b) at least a means for storing data, wherein the behavior is detected based on an analysis of measurements for the behavior features, wherein the behavior features comprise timestamps, start time and end time of tracks for appearance and disappearance of the detected faces, duration of watching fixtures, counts of the detected faces, counts of attentive faces, viewership, location markers, and facial pose estimates, and wherein the demographics information comprises estimates of gender, age, and ethnicity of the people. - View Dependent Claims (6, 7, 8)
-
Specification