System and method for segment relevance detection for digital content using multimodal correlations
First Claim
1. A system for evaluating media content comprising:
- a web-based application to stream a media content to a first set of participants and a second set of participants;
a server having a processor, and a facial detection engine, the server is configured to;
receive facial expression of one or more participants of the first set of participants in the form of video recordings and processing by the facial detection engine to identify one or more emotions of the first set of participants in frame-by-frame manner;
receive reactions of one or more participants in second set of participants, in which the reactions are captured by presenting one or more emojis to the second set of participants while the media content is playing, and asking the second set of participants to click emojis at different time of displayed media content to mark corresponding reactions;
wherein the server plots a graphical representation of the facial expression captured for the first set of participants and the reactions of second set of participants to identify one or more segment of the media content that are engaging for both the first set of participants and the second set of participants.
1 Assignment
0 Petitions
Accused Products
Abstract
A system and method for media content evaluation based on combining multi-modal inputs from the audiences that may include reactions and emotions that are recorded in real-time on a frame-by-frame basis as the participants are watching the media content is provided. The real time reactions and emotions are recorded in two different campaigns with two different sets of people and which include different participants for each. For the first set of participants facial expression are captured and for the second set of participants reactions are captured. The facial expression analysis and reaction analysis of both set of participants are correlated to identify the segments which are engaging and interesting to all the participants.
65 Citations
16 Claims
-
1. A system for evaluating media content comprising:
-
a web-based application to stream a media content to a first set of participants and a second set of participants; a server having a processor, and a facial detection engine, the server is configured to; receive facial expression of one or more participants of the first set of participants in the form of video recordings and processing by the facial detection engine to identify one or more emotions of the first set of participants in frame-by-frame manner; receive reactions of one or more participants in second set of participants, in which the reactions are captured by presenting one or more emojis to the second set of participants while the media content is playing, and asking the second set of participants to click emojis at different time of displayed media content to mark corresponding reactions; wherein the server plots a graphical representation of the facial expression captured for the first set of participants and the reactions of second set of participants to identify one or more segment of the media content that are engaging for both the first set of participants and the second set of participants. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method for evaluating media content comprising:
-
streaming a media content to a first set of participants and a second set of participants; providing a server having a processor, and a facial detection engine, the server is configured to; receive facial expression of one or more participants of the first set of participants in form of video recordings and processing by the facial detection engine to identify one or more emotions of the first set of participants in frame-by-frame manner; receive reactions of one or more participants in second set of participants, in which the reactions are captured by presenting one or more emojis to the second set of participants while the media content is playing, and asking the second set of participants to click emojis at different time of displayed media content to mark corresponding reactions; wherein the server plots a graphical representation of the facial expression captured for the first set of participants and the reactions of second set of participants to identify one or more segment of the media content that are engaging for both the first set of participants and the second set of participants. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
Specification