Systems and methods for providing audio content during trick-play playback

• US 10,231,001 B2
• Filed: 05/24/2016
• Issued: 03/12/2019
• Est. Priority Date: 05/24/2016
• Status: Active Grant

• Current Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Original Assignee: DivX, LLC (f/k/a DivX CF Holdings LLC) (SoftBank Group Corp.)
• Inventors: Frantz, William, Grab, Eric William
• Primary Examiner(s): Alam, Mushfikh I
• Application Number:
  US15/163,370
• Publication Number:
  US 20170347136A1
• Time in Patent Office: 1,022 Days
• Field of Search: 725 74-104
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

Systems and methods for providing playback of audio content during a trick-play mode in accordance with embodiments of the invention are disclosed. Media content can include audio content and video content. The audio content is divided into segments and the video content includes frames. Synchronization information associates a presentation time of each of the segments of the audio content with a presentation time of one or more frames associated with the images of the video content. When a trick-play mode is requested, the next frame from the video content to present is determined. The synchronization information is used to determine the segments of the audio content associated with the next frame. The determined segments audio content are presented as the frames are displayed in trick-play mode.

Claims

1. A method for providing playback of audio content in a trick-play mode during playback of media content, the method comprising:

storing a plurality of frames of a video content portion of media content in a buffer in a playback device, wherein each video frame of the plurality video frames has a presentation time;

storing a plurality of segments of an audio content portion of media content in a buffer in the playback device,

wherein each audio segment is divided from a first audio stream and stored as an individual segment,

wherein each audio segment is associated with a set of one or more video frames, and

wherein each audio segment has a presentation time such that each audio segment presentation time is the same presentation time of the associated set of video frames;

receiving a command for trick-play mode of the audio content in the playback device, wherein the trick-play mode is selected from the group consisting of fast-forward, slow-forward, and rewind;

determining a first set of one or more audio segments from the plurality of audio segments of the first audio stream to be presented based upon the command for trick play of the media content using the playback device and using the presentation time of at least one segment of the first set of segments;

determining a second set of one or more audio segments from the plurality of audio segments of the first audio stream to be presented based upon the trick play of the media content using the playback device and using the presentation time of at least one segment of the second set of segments, wherein the second set of audio segments would not immediately succeed the first set of audio segments if the first and second sets of audio segments were played back in a normal play mode; and

presenting the first set of audio segments and then immediately presenting the second set of audio segments in the trick-play mode using the playback device.

2. The method of claim 1 further comprising:

presenting a set of video frames in the trick-play mode on a display of the playback device.

3. The method of claim 2 wherein the second set of audio segments is presented concurrently with the presentation of the set of video frames.

4. The method of claim 2 further comprising:

adding each set of audio segments to a queue based on the command for the trick-play mode; and

wherein each set of audio segments is presented based upon the queue in an order as determined by the trick-play mode and is independent of the presentation of the set of video frames on the display of the playback device.

5. The method of claim 2 further comprising:

generating a display of a scrubber for the video content indicating a presentation time of the set of video frames using the playback device; and

overlaying the display of the scrubber for the video content over the presentation of the image on the display.

6. The method of claim 2 further comprising:

generating a display of a scrubber for the audio content indicating a presentation time of each the first and the second set of audio segments using the playback device; and

overlaying the display of the scrubber for the audio content over the presentation of the image on the display.

7. The method of claim 6 wherein the scrubber for the audio content is separate from a scrubber for the video content in the display.

8. The method of claim 1 further comprising:

receiving data for the plurality of segments of the audio content portion and data for the plurality of frames of the video content in the playback device from a content provider system over a network using adaptive bitrate streaming.

9. The method of claim 8 wherein the receiving of the data for the plurality of segments of the audio content portion and the data for the plurality of frames of the video content in the playback device from a content provider system over a network using adaptive bitrate streaming comprises:

receiving a top level index file from the content provider system in the playback device over a network wherein the top level index files identifies a plurality of alternative streams of video content wherein at least a portion of the plurality of alternative streams are encoded at different maximum bitrates, and at least one stream of audio content; and

requesting portions of the video content from the plurality of alternative streams from the content provider system using the playback device based upon network bandwidth between the playback device and the content provider system;

receiving the requested portions of the video content in the playback device in response to the requests;

generating the plurality of frames of the video content in the playback device from the portions of video content received by the playback device;

storing the plurality of frames in a buffer of the playback device;

requesting portions of the audio content from the at least one streams of audio content from the content provider system using the playback device;

receiving the requested portions of the audio content in the playback device;

generating the plurality of segments of the audio content from the portions of audio content received using the playback device;

storing the plurality of segments of audio content in a buffer of the playback device;

determining the presentation time of each audio segment from the association between each audio segment with a set of one or more video frames using the playback device based upon information in the top level index file; and

storing the presentation time of each audio segment in the memory of the playback device.

10. The method of claim 9, wherein the determining the presentation time of each audio segment comprises:

reading a pointer to a file including the association between each audio segment with a set of one or more video frames from the top level index file using the playback device;

requesting the file from content provider system using the playback device; and

receiving the requested file in the playback device.

21. The method of claim 1, wherein the presentation time of at least one audio segment of the plurality of audio segments is determined from a subtitle track or a closed caption track.

22. The method of claim 1, wherein each audio segment has a playback duration that is approximately equal to or less than the playback duration of images associated with a video frame.

23. The method of claim 1, wherein the first set of audio segments of and the second set of segments each consist of the same set of audio segments.

26. The method of claim 1, wherein the first set of audio segments of and the second set of segments each consist of the same set of audio segments.

11. A playback device comprising:

a memory;

a network interface; and

a processor that reads instructions stored in the memory that direct the processor to;

store a plurality of frames of a video content portion of media content in a buffer in the playback device,

store a plurality of segments of an audio content portion of media content in a buffer in the playback device,

wherein each audio segment is divided from a first audio stream and stored as an individual segment,

wherein each audio segment is associated with a set of one or more video frames, and

wherein each audio segment has a presentation time such that that each audio segment presentation time is the same presentation time of the associated set of video frames,

receive a command for trick-play mode of the audio content, wherein the trick-play mode is selected from the group consisting of fast-forward, slow-forward, and rewind,

determine a first set of one or more audio segments from the plurality of audio segments of the first audio stream to be presented based upon the command for trick play of the media content using the playback device and using the presentation time of at least one segment of the first set of segments,

determine a second set of one or more audio segments from the plurality of audio segments of the first audio stream to be presented based upon the trick play of the media content using the playback device and using the presentation time of at least one segment of the second set of segments, wherein the second set of audio segments would not immediately succeed the first set of audio segments if the first and second sets of audio segments were played back in a normal play mode, and

present the first set of audio segments and then immediately present the second set of audio segments in the trick-play mode.

12. The playback device of claim 11 wherein the instructions further direct the processor to:

present the a set of video frames in the trick-play mode on a display of the playback device.

13. The playback device of claim 12 wherein the second set of audio segments is presented concurrently with the presentation of the set of video frames.

14. The playback device of claim 12 wherein the instructions further direct the processor to:

add each set of audio segments to a queue on the command for the trick-play mode; and

wherein each set of audio segments is presented based upon the queue in an order as determined by the trick-play mode and is independent of the presentation of the set of video frames on the display of the playback device.

15. The playback device of claim 12 wherein the instructions further direct the processor to:

generate a display of a scrubber for the video content indicating a presentation time of the set of video frames; and

overlay the display of the scrubber for the video content over the presentation of the next frame on the display.

16. The playback device of claim 12 wherein the instructions further direct the processor to:

generate a display of a scrubber for the audio content indicating a presentation time of each the first and the second set of audio segments; and

overlay the display of the scrubber for the audio content over the presentation of the image on the display.

17. The playback device of claim 16 wherein the scrubber for the audio content is separate from a scrubber for the video content in the display.

18. The playback device of claim 11 wherein the instructions further direct the processor to:

receive data for the plurality of segments of the audio content portion and data for the plurality of frames of the video content in the playback device from a content provider system over a network using adaptive bitrate streaming.

19. The playback of claim 18 wherein instructions to receive of the data for the plurality of segments of the audio content portion and the data for the plurality of frames of the video content from a content provider system over a network using adaptive bitrate streaming further direct the processing system to:

receive a top level index file from the content provider system from a content provider system over a network wherein the top level index files identifies a plurality of alternative streams of video content wherein at least a portion of the plurality of alternative streams are encoded at different maximum bitrates, and at least one stream of audio content; and

request portions of the video content from the plurality of alternative stream from the content provider system based upon network bandwidth between the playback device and the content provider system;

receive the requested portions of the video content;

generate the plurality of frames of the video content from the portions of video content received;

store the plurality of images in a buffer;

request portions of the audio content from the at least one streams of audio content from the content provider system;

receive the requested portions of the audio content;

generate the plurality of segments of the audio content from the portions of audio content received;

store the plurality of segments of audio content in a buffer of the playback device;

determine the presentation time of each audio segment from the association between each audio segment with a set of one or more video frames based upon information in the top level index file; and

store the presentation time of each audio segment in the memory of the playback device.

24. The method of claim 11, wherein the presentation time of at least one audio segment of the plurality of audio segments is determined from a subtitle track or a closed caption track.

25. The method of claim 11, wherein each audio segment has a playback duration that is approximately equal to or less than the playback duration of images associated with a video frame.

20. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process comprising:

storing a plurality of frames of a video content portion of media content in a buffer in a playback device, wherein each video frame of the plurality video frames has a presentation time;

storing a plurality of segments of an audio content portion of media content in a buffer in the playback device,

wherein each audio segment is divided from a first audio stream and stored as an individual segment,

wherein each audio segment is associated with a set of one or more video frames, and

wherein each audio segment has a presentation time such that each audio segment presentation time is the same presentation time of the associated set of video frames;

receiving a command for trick-play mode of the audio content, wherein the trick-play mode is selected from the group consisting of fast-forward, slow-forward, and rewind;

determining a first set of one or more audio segments from the plurality of audio segments of the first audio stream to be presented based upon the command for trick play of the media content using the playback device and using the presentation time of at least one segment of the first set of segments;

determining a second set of one or more audio segments from the plurality of audio segments of the first audio stream to be presented based upon the trick play of the media content using the playback device and using the presentation time of at least one segment of the second set of segments, wherein the second set of audio segments would not immediately succeed the first set of audio segments if the first and second sets of audio segments were played back in a normal play mode; and

presenting the first set of audio segments and then immediately presenting the second set of audio segments of the audio content in the trick-play mode.