INTELLIGENT DETERMINATION OF REPLAYS BASED ON EVENT IDENTIFICATION

• US 20120093481A1
• Filed: 10/15/2010
• Published: 04/19/2012
• Est. Priority Date: 10/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: McDowell, Brian, Paulus, Owen, Jurenka, David, Alvarado, Ivann
• Granted Patent: US 9,484,065 B2
• Time in Patent Office: Days
• US Class Current: 386/241
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

A system for intelligently determining replay locations in a multimedia content stream based on identifying events in the multimedia content stream is provided. In one embodiment, events in the multimedia content stream are identified by analyzing information in the multimedia content stream, in real time. In another embodiment, events in the multimedia content stream are identified by analyzing the viewing behavior and an emotional response from users viewing the multimedia content, in real time. One or more replay locations in the multimedia content stream are determined based on the events identified in the multimedia content stream. The multimedia content stream with the replay locations are displayed to a user via a user interface in the user'"'"'s processing device.

Claims

1. A method for determining one or more replay locations in a multimedia content stream, the method comprising:

receiving a multimedia content stream related to a current broadcast;

analyzing the multimedia content stream in real time to identify one or more events in the multimedia content stream;

determining one or more replay locations in the multimedia content stream based on the one or more events identified in the multimedia content stream; and

providing the multimedia content stream with the one or more replay locations to one or more users, in real time.

2. The method of claim 1, wherein analyzing the multimedia content stream in real time to identify one or more events in the multimedia content stream further comprises:

detecting at least one of an audio cue or a visual cue in the multimedia content stream to identify the one or more events.

3. The method of claim 2, wherein the audio cue or the visual cue comprises at least one of an applause, a cheer, a remark, a score update or a commentary update in the multimedia content stream.

4. The method of claim 1, wherein analyzing the multimedia content stream in real time to identify one or more events in the multimedia content stream further comprises:

analyzing user-feedback information related to the one or more users to identify the one or more events.

5. The method of claim 4, wherein the user-feedback information comprises viewing behavior information related to the one or more users viewing the multimedia content stream, wherein the viewing behavior information further comprises:

one or more controls utilized by a user viewing the multimedia content stream on a processing device, wherein the controls comprise one or more of a pause, forward, rewind, jump or a stop control.

6. The method of claim 4, wherein the user-feedback information comprises an emotional response to the multimedia content stream viewed by the one or more users, wherein the emotional response further comprises:

one or more of facial expressions, vocal responses, gestures, movements and user interface interaction performed by the user.

7. The method of claim 1, wherein determining the one or more replay locations in the multimedia content stream further comprises:

determining the time of occurrence of the one or more events identified in the multimedia content stream as the one or more replay locations in the multimedia content stream.

8. The method of claim 1, wherein determining the one or more replay locations in the multimedia content stream further comprises:

determining the time of occurrence of a point of interest, the duration of the point of interest or the type of the point of interest related to the identified event in the multimedia content stream as the one or more replay locations in the multimedia content stream.

9. The method of claim 1, wherein determining the one or more replay locations in the multimedia content stream further comprises:

embedding information about the one or more replay locations into a metadata stream associated with the multimedia content stream.

10. The method of claim 9, wherein the information about the one or more replay locations further comprises:

information about the one or more events identified in the multimedia content stream, the time of occurrence of the one or more events, a point of interest related to the one or more events, the time of occurrence of the point of interest, a start time of the one or more replay locations and the duration of time to replay the multimedia content stream from the one or more replay locations.

11. The method of claim 1, wherein:

the analyzing the multimedia content stream in real time further comprises detecting at least one of an audio cue or a visual cue in a multimedia content stream associated with a current broadcast;

the analyzing the multimedia content stream in real time further comprises analyzing viewing behavior information related to the one or more users;

the analyzing the multimedia content stream in real time further comprises analyzing an emotional response to the multimedia content stream viewed by the one or more users; and

the determining one or more replay locations in the multimedia content stream based on the one or more events is based on at least one of the audio cue, the visual cue, the viewing behavior information and the emotional response.

12. One or more processor readable storage devices having processor readable code embodied on said one or more processor readable storage devices, the processor readable code for programming one or more processors to perform a method comprising:

analyzing at least one of an audio cue and a visual cue in a multimedia content stream associated with a current broadcast;

receiving viewing behavior information from one or more users viewing the multimedia content stream;

receiving emotional response information from the one or more users viewing the multimedia content stream;

identifying one or more events in the multimedia content stream based on at least one of the audio cue, the visual cue, the viewing behavior information and the emotional response information, in real time;

determining one or more replay locations in the multimedia content stream based on the one or more events identified in the multimedia content stream; and

providing the multimedia content stream with the one or more replay locations to one or more users, in real time.

13. One or more processor readable storage devices of claim 12, further comprising:

receiving an input from the one or more users to replay the multimedia content stream; and

replaying one or more of the events identified in the multimedia content stream from the one or more replay locations, to the one or more users.

14. One or more processor readable storage devices of claim 12, wherein identifying the one or more events in the multimedia content stream further comprises:

determining if the viewing behavior information and the emotional response information from a threshold percentage of users viewing the multimedia content stream corresponds to an event in the multimedia content stream.

15. One or more processor readable storage devices of claim 12, wherein identifying the one or more events in the multimedia content stream further comprises:

determining if the audio cue or the visual cue corresponds to one or more of an applause, a cheer, a remark, a score update or a commentary update in the multimedia content stream.

16. An apparatus, comprising:

a depth camera;

a computing device connected to the depth camera to receive a multimedia content stream related to a current broadcast, identify one or more users in a field of view of a capture device connected to the computing device, track an emotional response of the one or more users to the multimedia content stream viewed by the one or more users by tracking movements, gestures and facial expressions performed by the one or more users in the field of view, provide information about the emotional response to a remote computing system for analysis;

receive a multimedia content stream with one or more replay locations from the remote computing system based on the analysis, and replay at least a portion the multimedia content stream with the one or more replay locations to the one or more users.

17. The apparatus of claim 16, wherein:

the computing device detects viewing behavior information related to the one or more users viewing the multimedia content stream and provides the viewing behavior information to the remote computing system for analysis.

18. The apparatus of claim 16, further comprising:

an audiovisual device connected to the computing device, wherein the audiovisual device displays the multimedia content stream with the one or more replay locations to the one or more users.

19. The apparatus of claim 18, wherein:

the audiovisual device receives an input from the one or more users to re-watch a portion the multimedia content stream, via a user interface and displays the portion of the multimedia content stream to the one or more users.

20. The apparatus of claim 18, wherein:

the computing device identifies the emotional response of the one or more users based on identifying a vocal response of the one or more users, in real time.