Consumption of content with reactions of an individual

• US 9,571,879 B2
• Filed: 01/10/2012
• Issued: 02/14/2017
• Est. Priority Date: 01/10/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Cao, Xiang, Lin, Xiumin, Dang, Yingnong, Paolantonio, Sergio, Zhang, Dongmei
• Primary Examiner(s): SALCE, JASON P
• Application Number:
  US13/347,546
• Publication Number:
  US 20130179911A1
• Time in Patent Office: 1,862 Days
• Field of Search: 348 1401- 1412, 725/10, 725/12, 725/133, 725/141, 725/152
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04N 21/42201 biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Techniques for obtaining a reaction of an individual to content and outputting the reaction of the individual with the content are described herein. The techniques may include obtaining the reaction of the individual as the content is displayed to the individual. The reaction may be obtained by capturing a video of the individual. The reaction of the individual and the content may be output at a same time to another individual. These techniques may also include obtaining the reaction of another individual as the content and the reaction of the individual are output to another individual.

Claims

1. A system comprising:

one or more processors;

a rear-facing camera communicatively coupled to the one or more processors and configured to capture content of a surrounding of an individual, the rear-facing camera configured to capture images in a first direction;

a front-facing camera communicatively coupled to the one or more processors and configured to capture a video of the individual in the surrounding of the individual as the rear-facing camera captures the content, the front-facing camera configured to capture images in a second direction that is opposite to the first direction; and

memory accessible by the one or more processors storing a reaction module executable by the one or more processors to perform operations including;

analyzing the video to identify a portion of the video that is associated with a reaction of the individual, the portion of the video being identified without a selection by a user;

associating the reaction of the individual with a section of the content; and

providing the portion of the video associated with the reaction of the individual and the section of the content for display to one or more individuals, the reaction of the individual and the content for display at a same time to the one or more individuals.

2. The system of claim 1, wherein the operations further include capturing audio of the individual, the audio to be included as part of the reaction of the individual.

3. The system of claim 1, wherein the associating includes storing the reaction of the individual as metadata associated with the content.

4. The system of claim 1, wherein the associating includes storing the reaction of the individual and the content on one or more servers accessible to the system over a network.

5. The system of claim 1, the operations further comprising:

causing display of the section of the content along with the portion of the video associated with the reaction of the individual.

6. The system of claim 5, wherein the reaction of the individual is displayed on at least a portion of the content while the content is displayed.

7. The system of claim 1, wherein the providing includes providing an avatar for display which expresses the reaction of the individual.

8. The system of claim 1, wherein the providing comprises providing the portion of the video associated with the reaction of the individual and the section of the content to a social networking site for displaying the reaction of the individual and the section of the content at the same time on the social networking site.

9. The system of claim 1, the operations further comprising:

providing a notification to the individual before capturing the video of the individual, the notification indicating that the individual will be recorded while the content is captured.

10. The system of claim 1, the operations further comprising:

selecting, based at least in part on the reaction of the individual, one or more sections of the content that are associated with a particular type of reaction, the particular type of reaction comprising at least one of a smile, a frown, a head motion, an arm motion, a laugh, a yell, or a cry.

11. One or more computer storage media storing computer-readable instructions that, when executed, instruct one or more processors to perform operations comprising:

causing a computing device to output content to an individual via a display associated with the computing device;

obtaining, by one or more sensors coupled to the one or more processors, data including reactions of the individual to the content while the content is output to the individual, the one or more sensors comprising at least one of;

a forward facing camera to capture a video or image of the individual;

a microphone;

a heart rate monitor;

a blood pressure monitor;

or

a gaze tracking sensor;

analyzing the data including the reactions of the individual to identify a particular reaction;

analyzing the content to identify a section of the content that is associated with the particular reaction of the individual and that is associated with a reaction of another individual, the section of the content being identified without receiving a selection by a user; and

causing simultaneous output of the section of the content and at least one of the particular reaction of the individual or the reaction of the other individual.

12. The one or more computer storage media of claim 11, wherein the obtaining the data including the reactions of the individual includes at least one of capturing a video of the individual, capturing audio of the individual, or monitoring a physical response of the individual.

13. The one or more computer storage media of claim 11, wherein the content comprises at least one of video or audio data.

14. The one or more computer storage media of claim 11, wherein analyzing the data including the reactions of the individual to identify the particular reaction comprises identifying at least one of a smile, a frown, a head motion, an arm motion, a laugh, a yell, a cry, a nerve response, a heart rate, or a blood pressure.

15. The one or more computer storage media of claim 11, wherein the section of the content is caused to be output to a further individual that has a similar location, interest, or an age to either the individual or the other individual.

16. A computing device comprising:

one or more processors;

a rear facing camera communicatively coupled to the one or more processors and configured to capture content of an environment of a user, the rear facing camera configured to capture images in a first direction;

one or more sensors communicatively coupled to the one or more processors and configured to capture reactions of the user as the rear facing camera captures the content of the environment; and

memory communicatively coupled to the one or more processors and storing instructions that, when executed by the one or more processors, perform operations including;

analyzing the reactions of the user to identify a section that includes a particular reaction of the user, the section of the video being identified without a selection by an individual;

associating the section of the video with a portion of the content; and

outputting the section of the reactions that includes the particular reaction of the user and the portion of the content for display at a same time to one or more individuals.

17. The computing device of claim 16, the one or more sensors comprising at least one of:

a forward facing camera to capture images in a second direction opposite the first direction;

a microphone;

a heart rate monitor;

a blood pressure monitor;

or

a gaze tracking sensor.

18. The computing device of claim 17, wherein analyzing the reactions of the user to identify the section that includes the particular reaction of the user comprises identifying at least one of a smile, a frown, a head motion, an arm motion, a laugh, a yell, a cry, a nerve response, a heart rate, or a blood pressure.

19. The computing device of claim 16, wherein outputting the section of the reactions that includes the particular reaction comprises at least one of:

presenting the section of the reactions that includes the particular reaction of the user and the portion of the content at a same time on a display associated with the computing device;

or

sending the section of the reactions that includes the particular reaction of the user and the portion of the content to another computing device to be presented at a same time.

20. The computing device of claim 16, wherein outputting the section of the reactions that includes the particular reaction comprises:

presenting, via a display associated with the computing device, a progress bar representing a duration of a presentation of the reactions; and

presenting, via the display, an indicator on the progress bar at a time corresponding to the section of the reactions that includes the particular reaction.