Method and system for image and video analysis, enhancement and display for communication

• US 8,208,694 B2
• Filed: 06/06/2007
• Issued: 06/26/2012
• Est. Priority Date: 06/06/2006
• Status: Active Grant

• Current Assignee: Moreideas Inc.
• Original Assignee: Peter Whaite, Ren Saint-Pierre, Thomas Jelonek
• Inventors: Jelonek, Thomas, Whaite, Peter, Saint-Pierre, Ren
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US11/759,067
• Publication Number:
  US 20080002892A1
• Time in Patent Office: 1,847 Days
• Field of Search: 382117-118, 382/190, 382/305, 382/209, 382/215, 382/217, 382/224, 382/228
• US Class Current: 382/118
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 2201/3278 Transmission

Abstract

There is described a system and a method for generating a metatag indicative of an attribute of a person represented in an image, the method comprising: performing a computer-based analysis of the image; identifying a feature related to the attribute of the person in the image using results from the computer-based analysis; generating the metatag based on the identified feature; and associating the metatag to the image, the metatag being thereby related to the attribute of the person.

Claims

1. A method for generating metatags indicative of an attribute of a person represented in an image, said method comprising:

performing a computer-based analysis of the image;

identifying individual facial features of the person in the image using facial feature detection analysis to determine structural characteristics of individual facial features, said identifying including estimating a position of individual facial features;

applying objective visual appearance metrics to at least one of the individual facial features to determine a qualifier indicating a specific characteristic of the at least one of the individual facial features;

associating the qualifier to the at least one of the individual facial features;

generating the metatags based on the associated qualifier;

creating a consumer profile using the metatags, wherein the creating a consumer profile comprises;

retrieving pre-set profile indicators;

matching the metatags with the pre-set profile indicators;

creating the consumer profile using the pre-set profile indicators which match the metatags; and

retrieving advertising data, comparing the consumer profile with the advertising data to generated correspondences with advertisements thereby generating targeted advertisement using the consumer profile, and outputting the targeted advertisement to a user interface for display;

wherein the metatags comprise at least one of;

an eye color, a shape of eye, a shape of nose, a shape of face, a hair color, baldness, a hair style, hair quality, a type of glasses, a make-up color, a make-up application, a make-up characteristic, an eyebrow color, an eyebrow shape, an eye bag, facial hair, wrinkles, blemishes, a lipstick color, a mouth shape, a lip shape, a lip color, pimples, a skin color, a skin tone, a skin texture, a cheekbone contour shape, a teeth shape, a teeth condition, smile characteristics, an eye-shadow type, an eye-shadow color and a level of tan.

2. The method as in claim 1, further comprising at least one of outputting the metatags on a communication network and displaying the metatags on the user-interface of a user.

3. The method as in claim 1, wherein said individual facial features comprises one of:

hair, eye, nose, ear, eyebrows, eye glasses, skin, cheekbone contour, mouth, lip, teeth, makeup, jewelry and facial hair.

4. The method as in claim 1, wherein the image comprises at least one of a still image and a video image.

5. The method as in claim 1, wherein the step of identifying comprises at least one of:

using a specific feature detector;

using appearance-based information; and

using statistical training and statistical models.

6. The method as in claim 1, further comprising modifying the image to generate modified images using at least one of:

the metatags, the consumer profile and a user input.

7. The method as in claim 6, wherein the step of modifying the image comprises at least one of:

applying a template, applying an avatar, and morphing.

8. The method as in claim 7, further comprising ranking the modified images in an order determined by at least one of:

the consumer profile and the user input.

9. The method as in claim 1, wherein said identifying further comprises:

associating the metatags to the image, the metatags being thereby related to the attribute of the person.

10. The method as in claim 1, further comprising classifying the at least at least one of the individual facial features in a category, said category being one of:

eye, nose, mouth, ear, hair, glasses, and jewelry.

11. The method as in claim 1, wherein the qualifier comprises at least one of:

a tone, an evenness, a color, a shape, an aspect, a type.

22. The method of claim 1, wherein the pre-set profile indicators include at least one of:

age group, ethnicity, gender, emotional expression, lifestyle, affluence, social values, geo-location, personality, behavior and social group.

12. A system for generating metatags indicative of an attribute of a person represented in an image, said system comprising:

an image analysis device for performing a computer-based analysis of the image;

a feature identifier for identifying individual facial features of the person in the image using structural characteristics of individual features, said identifying including estimating a position of individual features;

a qualification module for applying objective visual appearance metrics to at least one of the individual facial features to determine a qualifier indicating a specific characteristic of the at least one of the individual facial features and associating a qualifier to the at least one of the individual facial features,

a metatag generator for generating the metatags based on the associated qualifier;

a metatag matcher for associating the metatags to the image, the metatags being thereby related to the attribute of the person;

an image inference module for creating a consumer profile using the metatags, wherein the creating a consumer profile comprises;

retrieving pre-set profile indicators;

matching the metatags with the pre-set profile indicators;

creating the consumer profile using the pre-set profile indicators which match the metatags; and

an advertisement matcher for matching targeted advertisement to the matched metatags;

wherein the metatags comprise at least one of;

an eye color, a shape of eye, a shape of nose, a shape of face, a hair color, baldness, a hair style, hair quality, a type of glasses, a make-up color, a make-up application, a make-up characteristic, an eyebrow color, an eyebrow shape, an eye bag, facial hair, wrinkles, blemishes, a lipstick color, a mouth shape, a lip shape, a lip color, pimples, a skin color, a skin tone, a skin texture, a cheekbone contour shape, a teeth shape, a teeth condition, smile characteristics, an eye-shadow type, an eye-shadow color and a level of tan.

13. The system as in claim 12, further comprising an output device for outputting the associated metatags on a communication network.

14. The system as in claim 12, wherein the at least one of the individual facial features is one of:

hair, eye, nose, ear, eyebrows, eye glasses, skin, cheekbone contour, mouth, lip, teeth, jewelry and facial hair.

15. The system as in claim 12, wherein the image comprises at least one of a still image and a video image.

16. The system as in claim 12, further comprising a display unit for displaying the matched targeted advertisement to a user.

17. The system as in claim 12, wherein the feature identifier further comprises a feature identifier for identifying the at least one of the individual facial features in the image using at least one of:

known structural and appearance information about the at least one of the individual facial features, statistical training and pre-established statistical models of the at least one of the individual facial features.

18. The system as in claim 12, further comprising a profile generator for creating a consumer profile using the metatags.

19. The system as in claim 18, further comprising an image modification module for modifying the image according to at least one of:

the metatags, the consumer profile and a user input.

20. The system as in claim 19, further comprising an advertising matcher in communication with the metatag matcher for matching targeted advertisement to the image by comparing the consumer profile and the metatags with advertising and statistical data, the targeted advertisement being based on the user input.

21. A method for generating metatags indicative of an attribute of a person represented in an image, said method comprising:

sending the image to a processing device over a communication network;

performing a computer-based analysis of the image;

identifying individual facial features of the person in the image using structural characteristics of each individual facial features, said identifying including estimating a position of each individual facial features;

applying objective visual appearance metrics to at least one of the individual facial features to determine a qualifier indicating a specific characteristic of the at least one of the individual facial features;

associating the qualifier to the at least one of the individual facial features,

generating the metatags based on the associated qualifier;

associating the metatags to the image, the metatags being thereby related to the attribute of the person,

storing the metatags associated to the image in a database, the database being accessible over the communication network;

modifying the image comprising;

retrieving at least one of the metatags, the consumer profile and a user input;

retrieving appearance templates;

matching any one of the metatags, the consumer profile and a user input with the appearance templates;

generate a modified image using the matched appearance templates; and

accessing the database to retrieve the metatags and using the metatags to perform displaying the metatags on a display unit of a user, using the metatags to create a consumer profile, using the metatags to modify the image, and matching the metatags to a set of advertisements for display to the user;

wherein the metatags comprise at least one of;

an eye color, a shape of eye, a shape of nose, a shape of face, a hair color, baldness, a hair style, hair quality, a type of glasses, a make-up color, a make-up application, a make-up characteristic, an eyebrow color, an eyebrow shape, an eye bag, facial hair, wrinkles, blemishes, a lipstick color, a mouth shape, a lip shape, a lip color, pimples, a skin color, a skin tone, a skin texture, a cheekbone contour shape, a teeth shape, a teeth condition, smile characteristics, an eye-shadow type, an eye-shadow color and a level of tan.