Decoupled Playback of Media Content Streams

• US 20180270517A1
• Filed: 03/19/2017
• Published: 09/20/2018
• Est. Priority Date: 03/19/2017
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Maillot, Alain Philippe, Berger, Roy, McAfee, Hayden William, Disano, Robert, Skobov, Mikhail
• Application Number:
  US15/462,888
• Publication Number:
  US 20180270517A1
• Time in Patent Office: Days
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4398 involving reformatting operations of audio signals details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

A technique is described herein for decoupling the playback of media content streams that compose a media item. In one implementation, the technique involves: in a synchronized state, presenting a stream of first media content (such as audio content) in synchronization with a stream of second media content (such as video content); detecting a desynchronization event; in response to the desynchronization event, transitioning from the synchronized state to a desynchronized state by changing (e.g., slowing) a rate at which the stream of second media content is presented, relative to the stream of first media content; detecting a resynchronization-initiation event; and, in response to the resynchronization-initiation event, returning to the synchronized state by providing a compressed presentation of the stream of second media content. The technique further involves presenting the stream of first media content at a given non-zero rate while in the desynchronized state.

Claims

1. A computer-readable storage medium storing computer-readable instructions, the computer-readable instructions, when executed by one or more processor devices, performing a method that comprises:

in a synchronized state, presenting a stream of audio content in synchronization with a stream of video content using a playback application, the playback application presenting parts of the audio content concurrently with corresponding parts of the video content;

detecting a desynchronization event that indicates that a user has interacted with another application other than the playback application;

in response to the desynchronization event, transitioning from the synchronized state to a desynchronized state by slowing a rate at which the stream of video content is presented by the playback application, relative to the stream of audio content, while maintaining a rate at which the stream of audio content is presented by the playback application;

detecting a resynchronization-initiation event that indicates that the user can once again consume the video content; and

in response to the resynchronization-initiation event, returning to the synchronized state by providing a compressed presentation of the stream of video content via the playback application.

2. A method, performed by a computing device, the method comprising:

in a synchronized state, presenting a stream of first media content of a media item in synchronization with a stream of second media content of the media item, wherein parts of the stream of first media content are presented at a same time as corresponding parts of the stream of second media content;

detecting a desynchronization event;

in response to the desynchronization event, transitioning from the synchronized state to a desynchronized state by changing a rate at which the stream of second media content is presented relative to the stream of first media content while continuing to present the stream of first media content at a non-zero rate;

detecting a resynchronization-initiation event; and

in response to the resynchronization-initiation event, returning to the synchronized state by providing a digest of the second media content, the digest comprising an abbreviated set of video frames from the stream of second media content.

3. The method of claim 2,

wherein the desynchronization event corresponds to a determination that a user will no longer be able to attend to a presentation of the stream of second media content;

and

wherein the resynchronization-initiation event corresponds to a determination that the user can once again attend to the presentation of the stream of second media content.

4. The method of claim 3, further comprising:

detecting the desynchronization event when a user of the media item diverts attention away from a display device showing the media item; and

detecting the resynchronization-initiation event when the user of the media item returns attention to the display device showing the media item.

5. The method of claim 2, wherein the first media content is audio content, and the second media content is video content.

6. The method of claim 2, wherein the stream of first media content is presented in the desynchronized state at the same rate at which the stream of first media content is presented in the synchronized state.

7. The method of claim 2, wherein said changing comprises slowing the rate at which the stream of second media content is presented based at least on a prescribed slow-down function, until the rate at which the stream of second media content equals a prescribed second media pause rate.

8. The method of claim 2, wherein said returning to the synchronized state comprises increasing the rate at which the stream of second media content is presented based at least on a prescribed speed-up function, until the synchronized state is achieved.

9. The method of claim 8, wherein the prescribed speed-up function includes at least one part that corresponds to a nonlinear function.

10. The method of claim 2, wherein said returning to the synchronized state comprises:

assessing an amount of time in which the stream of first media content has been presented in desynchronization with the stream of second media content; and

choosing a second media resumption strategy based, at least in part, on the amount of time.

11. The method of claim 2, wherein said returning to the synchronized state comprises:

assessing a processing capability of the computing device; and

choosing a second media resumption strategy based, at least in part, on the processing capability.

12. The method of claim 2, wherein said returning to the synchronized state comprises:

identifying an amount of time to reach the synchronized state, following the resynchronization-initiation event;

identifying an entire span of second media content to be presented in the amount of time;

partitioning the entire span of second media content into plural second media content segments, each second media content segment corresponding to a temporal sub-span of the entire span of second media content; and

presenting plural second media streams of second media content at a same time, the plural second media streams of second media content being associated with the plural second media content segments.

13. The method of claim 2, wherein said returning to the synchronized state comprises:

identifying an amount of time to reach the synchronized state, following the resynchronization-initiation event;

identifying an entire span of second media content to be presented in the amount of time;

forming the digest for the entire span of second media content, the digest corresponding to an abbreviated version of the entire span of second media content; and

presenting the stream of second media content based at least on the digest.

14. The method of claim 2, further comprising:

forming the digest by;

identifying different scenes within a span of second media content to be presented during an amount of time occurring between the resynchronization-initiation event and reaching the synchronized state; and

selecting representative video frames of the different scenes to produce the digest.

15. The method of claim 2, further comprising:

forming the digest by;

identifying low-value video frames in a span of second media content to be presented during an amount of time occurring between the resynchronization-initiation event and reaching the synchronized state, wherein the low-value video frames are assessed with respect to one or more characteristics; and

eliminating the low-value video frames from the span of second media content to produce the digest.

16. The method of claim 15, further comprising:

identifying a redundant video frame from the span of second media content as redundant with respect to at least one other video frame from the span of second media content; and

removing the redundant video frame from the span of second media content to produce the digest.

17. The method of claim 2, further comprising:

forming the digest by;

identifying high-value video frames in a span of second media content to be presented during an amount of time occurring between the resynchronization-initiation event and reaching the synchronized state, wherein the high-value portions video frames are assessed with respect to one or more characteristics; and

including at least some of the high-value video frames in the digest.

18-20. -20. (canceled)

21. A computing device comprising:

one or more hardware processor devices; and

one or more storage resources storing machine-readable instructions which, when executed by the one or more hardware processor devices, cause the one or more hardware processor devices to;

in a synchronized state, present audio content in synchronization with video content, wherein parts of the audio content are presented concurrently with corresponding parts of the video content;

based at least on a physical location or physical orientation of a user, detect a desynchronization event that indicates that the user has diverted attention away from a display device presenting the video content;

in response to the desynchronization event, transition from the synchronized state to a desynchronized state by slowing a rate at which the video content is presented relative to the audio content while maintaining a rate at which the audio content is presented;

based at least on the physical location or the physical orientation of the user, detect a resynchronization-initiation event that indicates that the user has resumed paying attention to the display device; and

in response to the resynchronization-initiation event, return to the synchronized state by providing a compressed presentation of the video content.

22. The computing device of claim 21, wherein the machine-readable instructions, when executed by the one or more hardware processor devices, cause the one or more hardware processor devices to:

detect the desynchronization event and the resynchronization-initiation event based at least on the physical location of the user relative to the display device.

23. The computing device of claim 21, wherein the machine-readable instructions, when executed by the one or more hardware processor devices, cause the one or more hardware processor devices to:

detect the desynchronization event and the resynchronization-initiation event based at least on the physical orientation of the user'"'"'s head relative to the display device.