Method and apparatus for generating meta data of content

• US 7,953,254 B2
• Filed: 03/30/2007
• Issued: 05/31/2011
• Est. Priority Date: 10/27/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Seo, Joon-kyu
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US11/730,227
• Publication Number:
  US 20080101660A1
• Time in Patent Office: 1,523 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

A method and apparatus are provided for generating emotional information including a user'"'"'s impressions in relation to multimedia content or meta data regarding the emotional information, and a computer readable recording medium storing the method. The meta data generating method includes receiving emotional information in relation to the content from at least one client system which receives and reproduces the content; generating meta data for an emotion using the emotional information; and coupling the meta data for the emotion to the content. Accordingly, it is possible to automatically acquire emotional information by using the facial expression of a user who is appreciating multimedia content, and use the emotional information as meta data.

Claims

1. A method of generating emotional information in relation to content, the method comprising:

acquiring a user image while the content is reproduced;

extracting facial information from the user image;

generating the emotional information in relation to the content, by comparing the facial information with facial expression template information, wherein the emotional information comprises a quantitative measure of a strength of an emotion corresponding to an emotion type; and

transmitting the emotional information to a server which provides the content, wherein

multiple user images of a same user in each scene are acquired, and

the generating the emotional information comprises;

extracting a plurality of pieces of the facial information from the user images;

generating pieces of the emotional information corresponding to each piece of the facial information;

identifying the emotion type in each piece of the generated emotional information; and

generating a representative type of the emotion in each scene based on a number of times the emotion type is identified in corresponding pieces of the generated emotional information.

2. The method of claim 1, wherein the acquiring the user image comprises acquiring the user image whenever each scene of the content is reproduced.

3. The method of claim 1, wherein the acquiring the user image comprises receiving at least two user images from an image input unit while the content is reproduced.

4. The method of claim 3, wherein the generating the emotional information comprises generating representative emotional information representing emotional information generated from the at least two user images.

5. The method of claim 1, wherein the acquiring the user image comprises acquiring the user image at a time corresponding to an emotional information acquiring time included in the meta data of the content.

6. The method of claim 1, further comprising, before generating the emotional information, normalizing a size of the extracted facial information in correspondence to a size of the facial expression template information.

7. A non-transitory computer readable recording medium storing a computer program for performing a method of generating emotional information in relation to content, the method comprising:

acquiring a user image while the content is reproduced;

extracting facial information from the user image;

generating the emotional information in relation to the content by comparing the facial information with facial expression template information, wherein the emotional information comprises a quantitative measure of a strength of an emotion corresponding to an emotion type; and

transmitting the emotional information to a server which provides the content, wherein multiple user images of a same user in each scene are acquired, and

the generating the emotional information comprises;

extracting a plurality of pieces of the facial information from the user images;

generating pieces of the emotional information corresponding to each piece of the facial information;

identifying the emotion type in each piece of the generated emotional information; and

generating a representative type of the emotion in each scene based on a number of times the emotion type is identified in corresponding pieces of the generated emotional information.

8. An apparatus for generating emotional information in relation to content, the apparatus comprising:

an image input unit which acquires a user image while the content is reproduced;

a facial image extraction unit which extracts facial information from the user image;

an emotional information generating unit which compares the facial information with facial expression template information, and generates the emotional information in relation to the content based on a result of the comparison, wherein the emotional information comprises a quantitative measure of a strength of an emotion corresponding to an emotion type; and

a communication unit which transmits the emotional information to a server which provides the content, wherein multiple user images of a same user in each scene are acquired, and

the emotional information generating unit;

extracts a plurality of pieces of the facial information from the user images;

generates pieces of the emotional information corresponding to each piece of the facial information;

identifies the emotion type in each piece of the generated emotional information; and

generates a representative type of the emotion in each scene based on a number of times the emotion type is identified in corresponding pieces of the generated emotional information.

9. The apparatus of claim 8, wherein the image input unit acquires the user image whenever each scene of the content is reproduced.

10. The apparatus of claim 8, wherein the image input unit acquires at least two user images while the content is reproduced.

11. The apparatus of claim 10, wherein the emotional information generating unit generates representative emotional information which represents the emotional information generated from the at least two user images.

12. The apparatus of claim 8, wherein the image input unit acquires the user image at a time corresponding to an emotional information acquiring time included in the meta data of the content.

13. The apparatus of claim 8, further comprising a facial image normalization unit which normalizes a size of the extracted facial information in correspondence to a size of the facial expression template information, wherein the emotional information generating unit compares the normalized facial information with the facial expression template information.