Information processing apparatus and method for recognizing a user's emotion

• US 8,407,055 B2
• Filed: 08/03/2006
• Issued: 03/26/2013
• Est. Priority Date: 08/05/2005
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Asano, Yasuharu, Yamamoto, Noriyuki
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Serrou, Abdelali
• Application Number:
  US11/498,348
• Publication Number:
  US 20070033050A1
• Time in Patent Office: 2,427 Days
• Field of Search: 704/275, 704/251, 704/231, 704/270, 704/E17.002
• US Class Current: 704/270
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

An information processing apparatus includes an obtaining unit that obtains meta-information concerning content; a predicting unit that predicts an emotion of a user who is viewing the content from the meta-information obtained by the obtaining unit; and a recognizing unit that recognizes an emotion of the user using the emotion predicted by the predicting unit and user information acquired from the user.

Claims

1. An information processing apparatus for recognizing a user'"'"'s emotion, said apparatus comprising:

obtaining means for obtaining meta-information concerning content;

predicting means for predicting an emotion of a user who is viewing the content from the meta-information obtained by the obtaining means; and

recognizing means for recognizing an emotion of the user using the emotion predicted by the predicting means and user information acquired from the user, wherein said recognizing means comprises a prediction information obtaining means, a recognition-model selection means, a recognition-model holding means, and a matching means, wherein a method for recognizing a user'"'"'s emotion is based on the Bayes decision rule,

wherein said prediction information obtaining means obtains prediction information from said predicting means and supplies said prediction information to said recognition-model selection means to select appropriate recognition model(s) from said recognition-model holding means based on said prediction information, and supplies the selected recognition model(s) to said matching means,

wherein an emotion recognition result output from said emotion recognizing means is fed back to said emotion predicting means so that said emotion predicting means can be adapted to said user, and wherein the emotion recognition result is used as information indicating the user'"'"'s preference to obtain information suited to the preference of the user when obtaining information via a network, wherein the emotion is an emotion of a user who is viewing the content.

2. The information processing apparatus according to claim 1, wherein the recognizing means performs weighting for the emotion of the user according to the emotion predicted by the predicting means.

3. The information processing apparatus according to claim 1, further comprising

sound-feature-value extracting means for collecting sound emitted from the user and extracting a feature value from the collected sound, wherein the recognizing means uses the feature value extracted by the sound-feature-value extracting means as the user information.

4. The information processing apparatus according to claim 1, further comprising

image-feature-value extracting means for capturing an image of the user and extracting a feature value from the captured image, wherein the recognizing means uses the feature value extracted by the image-feature-value extracting means as the user information.

5. The information processing apparatus according to claim 1, wherein the meta-information obtained by the obtaining means comprises text information concerning the content.

6. The information processing apparatus according to claim 1, wherein when the content includes audio information, the obtaining means extracts a feature value from the audio information, and obtains the extracted feature value as the meta-information.

7. The information processing apparatus according to claim 1, wherein when the content includes speech information, the obtaining means recognizes speech based on the speech information, and obtains text extracted from the recognized speech as the meta-information.

8. The information processing apparatus according to claim 1, wherein when the content includes video information, the obtaining means obtains information concerning performers appearing in video based on the video information as the meta-information.

9. The information processing apparatus according to claim 1, wherein when the content has time information attached thereto, the obtaining means extracts the meta-information in accordance with the progress of the content according to the time information.

10. The information processing apparatus according to claim 1, wherein:

the predicting means includes a table used to predict the emotion of the user;

the emotion of the user recognized by the recognizing means is supplied to the predicting means; and

the predicting means updates the table in response to the supplied recognized emotion of the user.

11. An information processing method carried out on an information processing apparatus for recognizing a user'"'"'s emotion, the method comprising the steps of:

obtaining, via a processor, meta-information concerning content from an obtaining unit;

predicting an emotion of a user who is viewing the content from the obtained meta-information using a predicting unit; and

recognizing using the processor, an emotion of the user using the predicted emotion and information acquired from the user by a recognizing unit, wherein said recognizing step further comprises the steps of;

obtaining prediction information from said predicting step;

supplying said prediction information to a recognition-model selection step to select appropriate recognition model(s) from a recognition-model holding means based on said prediction information, and supplying the selected recognition model(s) to a matching step, wherein a method for recognizing a user'"'"'s emotion is based on the Bayes decision rule, wherein an emotion recognition result output from said emotion recognizing unit is fed back to said emotion predicting unit so that said emotion predicting means can be adapted to said user, and wherein the emotion recognition result is used as information indicating the user'"'"'s preference to obtain information suited to the preference of the user when obtaining information via a network, wherein the emotion is an emotion of a user who is viewing the content.

12. A computer-readable program recorded on a non-transitory computer-readable medium for causing a computer to carry out an information processing method for recognizing a user'"'"'s emotion, the method comprising the steps of:

obtaining meta-information concerning content from an obtaining unit;

predicting an emotion of a user who is viewing the content from the obtained meta-information using a predicting unit; and

recognizing an emotion of the user using the predicted emotion and information acquired from the user by a recognizing unit, wherein said recognizing step further comprises the steps of;

obtaining prediction information from said predicting step,

supplying said prediction information to a recognition-model selection step to select appropriate recognition model(s) from a recognition-model holding means based on said prediction information, and

supplying the selected recognition model(s) to a matching step,

wherein a method for recognizing a user'"'"'s emotion is based on the Bayes decision rule,

wherein an emotion recognition result output from said emotion recognizing unit is fed back to said emotion predicting unit so that said emotion predicting unit can be adapted to said user, and wherein the emotion recognition result is used as information indicating the user'"'"'s preference to obtain information suited to the preference of the user when obtaining information via a network, wherein the emotion is an emotion of a user who is viewing the content.

13. An information processing apparatus comprising:

an obtaining unit that obtains meta-information concerning content;

a predicting unit that predicts an emotion of a user who is viewing the content from the meta-information obtained by the obtaining unit; and

a recognizing unit that recognizes an emotion of the user using the emotion predicted by the predicting unit and user information acquired from the user, wherein said recognizing unit comprises a prediction information obtaining unit, a recognition-model selection unit, a recognition-model holding unit, and a matching unit, wherein a method for recognizing a user'"'"'s emotion is based on the Bayes decision rule,

wherein said prediction information obtaining unit obtains prediction information from said predicting unit and supplies said prediction information to said recognition-model selection unit to select appropriate recognition model(s) from said recognition-model holding unit based on said prediction information, and supplies the selected recognition model(s) to said matching unit, wherein an emotion recognition result output from said emotion recognizing unit is fed back to said emotion predicting unit so that said emotion predicting unit can be adapted to said user, and wherein the emotion recognition result is used as information indicating the user'"'"'s preference to obtain information suited to the preference of the user when obtaining information via a network, wherein the emotion is an emotion of a user who is viewing the content, wherein the obtaining, predicting, and recognizing units comprise hardware implementation.