System and method for granular tagging and searching multimedia content based on user reaction

• US 9,202,251 B2
• Filed: 11/07/2011
• Issued: 12/01/2015
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: Anurag Bist, Hamid Lalani
• Original Assignee: Anurag Bist, Hamid Lalani
• Inventors: Bist, Anurag, Lalani, Hamid
• Primary Examiner(s): DENNISON, JERRY B
• Application Number:
  US13/291,064
• Publication Number:
  US 20130117375A1
• Time in Patent Office: 1,485 Days
• Field of Search: 725/10, 725/11, 725/12
• US Class Current: 1/1
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 50/01 Social networking

Abstract

A system and a method for generating an emotional profile of the user and deriving inference from the analytics of generated emotional profile is provided. The method involves sharing media content or online event in a connected environment; capturing user'"'"'s reaction to the said content or event; generating an emotional score of the user to rate the media content or event; and sharing the emotional score within the connected environment.

Claims

1. A system for tagging an online multimedia content or event in a time granular manner, the system comprising:

a network of a plurality of users with a plurality of client devices connected with each other;

a repository storing the online multimedia content or event and sharing the online multimedia content within the network of users;

a module inbuilt in each of the client devices configured to capture emotional expressions of the user reflecting emotional states, continuously along the length of the online multimedia event or content, while the user is interacting with the online multimedia;

a processor configured for generating an emotional score for each emotional state component in the time granular manner, continuously along the length of the online multimedia, by calculating the intensity of each emotional state component of the user at a given time stamp by determining the deviation of each generated emotional score from a baseline emotional score and normalizing the deviation;

the processor configured for calculating an average emotional score by calculating a mean of plurality of emotional scores reflected by a plurality of users in a time interval of the online multimedia content or event; and

a second module for tagging the online multimedia content in the time granular manner by associating the emotional scores and the average emotional scores continuously along the length of online multimedia content or event.

2. The system of claim 1 wherein the client device can be a mobile phone, a Smartphone, a laptop, a camera with WiFi connectivity, a desktop, a tablet computer, or any other sensory device with connectivity.

3. The system of claim 1 wherein the emotional expressions of the users correspond to critical auditory, visual or any other sensory input or facial expressions of the users, such as Happy, Surprised, Fearful, Normal, Angry, Disgusted, Sad and the like.

4. The system of claim 1 wherein the emotional score corresponds to the emotion, response, attention span, gestures, hand and head movement or other reactions and stimuli of the user.

5. The system of claim 1 wherein the processor generates the emotional score as a function of time.

17. The system of claim 1, wherein the client devices may transmit the captured emotional states of the users to one or more servers to process and generate emotional scores for the users.

6. A method for tagging an online multimedia event or content in a time granular manner comprising:

distributing or sharing the online multimedia event or content, from a central multimedia repository, to one or more users with client devices, connected in a network;

capturing and processing emotional expressions reflecting emotional states of the user in the granular manner with respect to time length of the online multimedia content, by a module of the client device, while the user is interacting with the multimedia content;

providing a processor configured to generate an emotional score, corresponding to the intensity of each emotional state component in the time granular manner, continuously along the length of the multimedia content, by determining the deviation of each emotional state component from a baseline emotional state component and normalizing the deviation;

tagging the online multimedia content in the time granular manner by associating the emotional scores continuously along the length of multimedia content;

characterizing the online multimedia event or content based on the granular tagging of the emotional scores; and

sharing the granular tagging characteristics of the online multimedia content in the one or more user'"'"'s network.

7. The method of claim 6 wherein the online multimedia event or content includes video or audio content;

or an online video conferencing.

8. The method of claim 6 wherein the client device can be a mobile phone, a Smartphone, a laptop, a camera with WiFi connectivity, a desktop, a tablet computer, or any other sensory device with connectivity.

9. The method of claim 6 wherein the emotional states of the users correspond to any critical auditory, visual or any other sensory input or facial expressions of the users, such as Happy, Surprised, Fearful, Normal, Angry, Disgusted, Sad and the like.

10. The method of claim 6 wherein the emotional scores designates emotions, response, attention span, gestures, hand and head movement, or other reactions or stimuli of the user.

11. The method of claim 6 wherein the emotional score is tagged to the online multimedia content as a function of time.

12. The method of claim 6 wherein the online multimedia event is tagged with average emotional scores of plurality of users in granular manner.

13. A method for granular tagging of an online multimedia event or content comprising:

capturing a user'"'"'s instantaneous emotional state when interacting with the online multimedia event or content by a detection module for each frame of the online multimedia content;

providing a processor configured for generating an emotional score corresponding to each emotional state component as a function of time, by determining deviation of the emotional state component from a baseline emotional state component;

providing the processor configured for calculating an average emotional score by determining a mean of a plurality of emotional state components of a plurality of users at each frame of the online multimedia content;

tagging the multimedia content with the user'"'"'s emotional score and the average emotional score of the plurality of users for the corresponding frame at the specific time stamp of the content;

characterizing the content in a granular manner using the user'"'"'s emotional score and the average emotional score; and

sharing the granular tagging characteristics of the content in the user'"'"'s network.

14. The method of claim 13 wherein the online multimedia content are tagged granularly with respect to time.

15. The method of claim 13 wherein the emotional score corresponds to emotions, behavior, response, attention span, gestures, hand and head movement, or other reactions or stimuli of the user.

16. The method of claim 13 wherein the tagged multimedia content with average emotional scores are stored in a repository and are shared in the connected network and with other users.