Annotating audio-visual data

• US 20080275830A1
• Filed: 05/03/2007
• Published: 11/06/2008
• Est. Priority Date: 05/03/2007
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Greig, Darryl
• Granted Patent: US 8,126,220 B2
• Time in Patent Office: Days
• US Class Current: 706/21
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method of annotating audio-visual data is disclosed. The method includes detecting a plurality of facial expressions in an audience based on a stimulus, determining an emotional response to the stimulus based on the facial expressions and generating at least one annotation of the stimulus based on the determined emotional response.

Claims

1. A method of annotating audio-visual data comprising:

detecting a plurality of facial expressions in an audience based on a stimulus;

determining an emotional response to the stimulus based on the facial expressions; and

generating at least one annotation of the stimulus based on the determined emotional response.

2. The method of claim 1 wherein the stimulus further comprises an audio-video stimulus.

3. The method of claim 1 wherein detecting a plurality of facial expressions further comprises:

scanning the audience in real time; and

detecting face locations in the audience.

4. The method of claim 1 wherein determining an emotional response further comprises:

analyzing the plurality of facial expressions; and

classifying the real time emotional response from the plurality of facial expressions.

5. The method of claim 1 wherein creating at least one annotation of the stimulus further comprises:

indexing the at least one annotation based on the determined emotional response of the audience.

6. A computer program product for annotating audio-visual data, the computer program product comprising a computer usable medium having computer readable program means for causing a computer to perform the steps of:

initiating a stimulus to an audience;

detecting a plurality of facial expressions in the audience;

determining an emotional response to the stimulus based on the facial expressions; and

generating at least one annotation of the stimulus based on the determined emotional response.

7. The computer program product of claim 6 wherein the stimulus further comprises an audio-video stimulus.

8. The computer program product of claim 6 wherein detecting a plurality of facial expressions further comprises:

scanning the audience in real time; and

detecting face locations in the audience.

9. The computer program product of claim 6 wherein determining an emotional response further comprises:

analyzing the plurality of facial expressions; and

classifying the real time emotional response from the plurality of facial expressions.

10. The computer program product of claim 6 wherein creating at least one annotation of the stimulus further comprises:

indexing the at least one annotation based on the determined emotional response of the audience.

11. A system for annotating audio-visual data comprising:

video detection means;

computer processing means coupled to the video detection means wherein the computer processing means includes an annotation generation module wherein the annotation generation module comprises logic for

detecting a plurality of facial expressions in an audience based on a stimulus;

determining an emotional response to the stimulus based on the facial expressions; and

generating at least one annotation of the stimulus based on the determined emotional response.

12. The system of claim 11 wherein the stimulus further comprises a display means coupled to the video detection means and the computer processing means.

13. The system of claim 11 wherein the stimulus further comprises an audio-video stimulus.

14. The system of claim 11 wherein detecting a plurality of facial expressions further comprises:

scanning the audience in real time; and

detecting face locations in the audience.

15. The system of claim 11 wherein determining an emotional response further comprises:

analyzing the plurality of facial expressions; and

classifying the real time emotional response from the plurality of facial expressions.

16. The system of claim 11 wherein creating at least one annotation of the stimulus further comprises:

indexing the at least one annotation based on the determined emotional response of the audience.