Method and system for dynamically targeting content based on automatic demographics and behavior analysis

• US 7,921,036 B1
• Filed: 06/29/2009
• Issued: 04/05/2011
• Est. Priority Date: 04/30/2002
• Status: Expired due to Fees

• Current Assignee: Accessify, LLC (Ascend Innovation Management, LLC)
• Original Assignee: Videomining Corporation
• Inventors: Sharma, Rajeev, Jung, Namsoon, Moon, Hankyu, Saurabh, Varij
• Primary Examiner(s): Weiss, John G
• Assistant Examiner(s): GOLDMAN, MICHAEL H
• Application Number:
  US12/459,282
• Time in Patent Office: 645 Days
• Field of Search: 705/14.49, 705/14.52, 705/14.56, 705/14.58, 705/14.66, 705/14.67
• US Class Current: 705/14.66
• CPC Class Codes: G06Q 20/3674 involving authentication
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute

Abstract

The present invention is a method and system for selectively executing content on a display based on the automatic recognition of predefined characteristics, including visually perceptible attributes, such as the demographic profile of people identified automatically using a sequence of image frames from a video stream. The present invention detects the images of the individual or the people from captured images. The present invention automatically extracts visually perceptible attributes, including demographic information, local behavior analysis, and emotional status, of the individual or the people from the images in real time. The visually perceptible attributes further comprise height, skin color, hair color, the number of people in the scene, time spent by the people, and whether a person looked at the display. A targeted media is selected from a set of media pools, according to the automatically-extracted, visually perceptible attributes and the feedback from the people.

Claims

1. A method for selectively executing targeted media on a means for playing content, comprising the following steps of:

a) capturing a plurality of images for an individual or people with a single or a plurality of means for capturing images,

b) detecting a single or a plurality of face images of said individual or said people from said plurality of images using a person detector,

c) automatically extracting visually perceptible attributes of said individual or said people from said single or plurality of face images using the person detector and a person classifier,

d) selecting a targeted media from a set of media pools according to the automatically extracted, visually perceptible attributes using a media player,

e) customizing the targeted media in real-time, based on the automatically-extracted, visually perceptible attributes, including demographic information,

f) processing a demographic classification in real-time and dynamically narrowcasting the targeted media,

wherein the method of extracting the demographic information comprises a usage of at least a support vector machine (SVM) that is trained for age and gender, and

g) displaying the targeted media on the means for playing content,

wherein the person detector and the person classifier are embodied in a computing device and connected to the means for capturing images and the media player, and wherein the visually perceptible attributes comprise demographic information, local behavior analysis and emotional status.

2. The method according to claim 1, wherein the method further comprises a step of automatically determining the number of people in the scene, computing time spent by the people, determining whether a person looked at the display, extracting visually perceptible attributes of each person,

wherein the visually perceptible attributes comprise gender, age, height, skin color, and hair color.

3. The method according to claim 1, wherein the method further comprises a step of applying a business rule from a set of media selection rules to a new event that is generated by a person recognizer for the detected person.

4. The method according to claim 1, wherein the method further comprises a step of selecting digital media to be displayed on a network of digital devices, based on the automatically-extracted, visually perceptible attributes, including the demographic information.

5. The method according to claim 1, wherein the method further comprises the steps of:

a) playing random content on the means for playing content,

wherein more than one suitable content for a given audience profile class is available, and a random selection of the content is performed among the available contents based on a probability distribution,

b) automatically analyzing the response of the people to the randomly played content,

c) scoring the content based on the response analysis according to demographic, behavioral, and emotional attributes of the people, and

d) playing a matching content for the people based on the score, according to the demographic, behavioral, and emotional attributes,

wherein a randomization scheme controls the degree of randomness based on the measured response analysis.

6. The method according to claim 1, wherein the method further comprises a step of processing the content search and selection on the Internet, and retrieving the content from the Internet, based on the automatically-extracted, visually perceptible attributes.

7. The method according to claim 1, wherein the method further comprises a step of sending the selected content to a mobile device,

whereby the mobile device comprises a mobile phone and a Bluetooth-based device.

8. The method according to claim 1, wherein the method further comprises a step of processing the images captured by the means for capturing images that are installed in a mobile device,

whereby the mobile device comprises a mobile phone, a cart, and a Bluetooth-based device.

9. The method according to claim 1, wherein the method further comprises a step of installing a second means for capturing images in places where the means for playing content are not located,

wherein the visually perceptible attributes are automatically extracted from the images captured from the second means for capturing images, and

wherein the narrowcasting in the means for playing content is based on the automatically-extracted, visually perceptible attributes.

10. The method according to claim 1, wherein the method further comprises a step of extracting the visually perceptible attributes before the actual appearance of the people in the vicinity of the means for playing content and playing the content that is selected based on the automatically-extracted, visually perceptible attributes while the people approach the means for playing content.

11. The method according to claim 1, wherein the method further comprises a step of classifying the demographics for multiple people and selecting the content based on the analysis of the composition of the multiple people.

12. The method according to claim 1, wherein the method further comprises a step of analyzing the behavior of people in the vicinity of the means for playing content.

13. The method according to claim 1, wherein the method further comprises a step of the measuring emotional state of the people in response to the content, wherein the face images are tracked and fed to a facial expression recognition step to recognize changes in facial features, from which the emotional state of the individual is estimated.

14. The method according to claim 1, wherein the method further comprises a step of randomly selecting media content according to probability densities determined by the match between the audience profile and the targeted media,

wherein more than one suitable content for a given audience profile class is available,

and the random selection is performed among the available contents,

wherein a set of media selection rules is implemented using a table of probability distribution, and

whereby more closely-matched content receives higher probability to be selected.

15. The method according to claim 1, wherein the method further comprises a step of modifying the media selection scheme based on response feedback from the audience utilizing a reinforcement learning method to maximize the effectiveness of the media, wherein the audience takes a role of environment, a media play takes a role of action, and a media response takes a role of a reward, and a media control takes a role of an agent,

wherein the media response is interpreted differently based on a measured media-independent audience profile, and

whereby the response feedback comprises changes in emotional state, viewing duration, and degree of attention toward the media.

16. The method according to claim 15, wherein the method further comprises a step of utilizing a temporal difference learning as a reinforcement learning method to modify the media selection scheme,

wherein the temporal difference learning iteratively updates a value function of each state, which captures the cumulative future effectiveness of the media starting from the state, and

whereby the goal of utilizing the temporal difference learning is to update an optimal media selection scheme.

17. An apparatus, comprising:

a) means for capturing a plurality of images for an individual or people with a single or a plurality of means for capturing images,

b) a person detector for detecting a single or a plurality of face images of said individual or said people from said plurality of images,

c) a person classifier for automatically extracting visually perceptible attributes of said individual or said people from said single or plurality of face images,

d) a media control for customizing a targeted media in a set of media pools in real-time, based on the automatically-extracted, visually perceptible attributes, including demographic information,

e) said person classifier for processing a demographic classification in real-time,

wherein the method of extracting the demographic information comprises a usage of at least a support vector machine (SVM) that is trained for age and gender,

f) a media player for selecting a targeted media from the set of media pools, according to the automatically-extracted, visually perceptible attributes, and

g) means for playing content that selectively displays the targeted media,

wherein the person detector and the person classifier are embodied in a computing device and connected to the means for capturing images and the media player, and

wherein the visually perceptible attributes comprise demographic information, local behavior analysis, and emotional status.