Adaptive streaming systems and methods for performing trick play

• US 10,368,096 B2
• Filed: 10/09/2018
• Issued: 07/30/2019
• Est. Priority Date: 01/05/2011
• Status: Active Grant

• Current Assignee: DivX, LLC (f/k/a DivX CF Holdings LLC) (SoftBank Group Corp.)
• Original Assignee: DivX, LLC (f/k/a DivX CF Holdings LLC) (SoftBank Group Corp.)
• Inventors: Braness, Jason, van der Schaar, Auke Sjoerd, Soroushian, Kourosh
• Primary Examiner(s): Walker, Jared
• Assistant Examiner(s): Fereja, Samuel D
• Application Number:
  US16/155,840
• Publication Number:
  US 20190045220A1
• Time in Patent Office: 294 Days
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/26258 for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list retrieval of multimedia data based on playlists G06F16/40
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/42607 for processing the incoming bitstream
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 : involving a specific file format, e.g. MP4 format
• CPC Class Codes: H04N 21/8543 : using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80

Abstract

Systems and methods for performing trick play functionality using trick play streams during adaptive bitrate streaming in accordance with embodiments of the invention are disclosed. One embodiment includes requesting a video container index from a video container file containing a video stream from a plurality of alternative streams of video; requesting at least one portion of the video stream using at least one entry from the video container index; decoding the at least one portion of the video stream; receiving at least one user instruction to perform a visual search of the media; requesting a trick play container index from a trick play container file containing a trick play stream; requesting at least one frame of video from the at least one trick play stream; and decoding and displaying the at least one frame of video from the trick play stream.

Claims

1. A playback device, comprising:

a set of one or more processors; and

a non-volatile storage containing an application for causing the set of one or more processors to perform the steps of;

obtaining top level index information identifying a plurality of alternative streams of video, an audio stream, and at least one trick play stream that are each stored in a separate container file, where;

each video container file containing a given stream from the plurality of alternative streams of video comprises;

portions of the given video stream within the video container file, where the portions of the given video stream comprise an encoded group of pictures that commences with a picture encoded without reference to another picture in the given video stream; and

a video container index, where entries in the video container index indicate sizes of portions of the given video stream within the video container file;

each trick play container file containing a given trick play stream from the at least one trick play stream comprises;

frames of the given trick play stream, where each frame of the given trick play stream is a picture encoded without reference to another picture in the trick play stream; and

a trick play container index, where entries in the trick play container index comprise a timecode and a location of a frame in the given track play stream;

requesting a video container index from a video container file containing a video stream from the plurality of alternative streams of video;

requesting at least one portion of the video stream from the plurality of alternative streams of video using at least one entry from the video container index;

decoding the at least one portion of the video stream from the plurality of alternative streams of video;

receiving at least one user instruction to perform a visual search of the media;

requesting a trick play container index from a trick play container file containing a trick play stream from the at least one trick play stream;

requesting at least one frame of video from the at least one trick play stream using at least one entry from the trick play container index; and

decoding and displaying the at least one frame of video from the at least one trick play stream.

2. The playback device of claim 1, wherein the at least one trick play stream comprises a plurality of trick play streams.

3. The playback device of claim 1, wherein a trick play stream from the at least one trick play stream is encoded at a lower bit rate than the plurality of alternative streams of video.

4. The playback device of claim 3, wherein:

the top level index information includes information indicating the portion of each video container file containing the video container index for the video container file; and

the application further is for causing the set of one or more processors to perform the step of requesting a video container index from a video container file containing a video stream from the plurality of alternative streams of video by requesting the portion of the video container file indicated as containing the video container index for the video container file by the top level index information.

5. The playback device of claim 1, wherein the application is further for causing the set of processors to perform the step of performing adaptive streaming with respect to the plurality of trick play streams.

6. The playback device of claim 1, wherein the application is further for causing the set of processors to perform the steps of:

manipulating at least one timecode assigned to the at least one frame of video from the at least one trick play stream prior to providing the at least one frame to a decoder on the playback device, where the at least one timecode is manipulated in order to achieve a rate of playback based on a desired rate for the visual search; and

decoding and presenting the at least one frame of video from the at least one trick play stream in accordance with the manipulated timecodes.

7. The playback device of claim 1, wherein the application is further for causing the set of processors to perform the steps of:

detecting streaming conditions for the playback device;

selecting a stream from the plurality of alternative streams of video based on the detected streaming conditions; and

requesting portions of video from the selected stream.

8. The playback device of claim 7, wherein:

the application is further for causing the set of processors to perform the step of converting at least one entry in a video container index for a video container file containing the selected stream into a byte range within the video container file containing the selected stream corresponding to at least one portion of the selected stream; and

the step of requesting portions of the selected stream further comprises the step of generating at least one byte range request for the at least one portion of the selected stream.

9. The playback device of claim 1, wherein the encoded group of pictures is a single closed group of pictures that commences with a picture encoded without reference to another picture in the given video stream.

10. The playback device of claim 1, wherein:

the at least one user instruction to perform a visual search of the media indicates a time code; and

the application further is for causing the set of one or more processors to perform the step of requesting at least one frame of video from the at least one trick play stream using at least one entry from the trick play container index based upon the time code.

11. The playback device of claim 10, wherein the application further is for causing the set of one or more processors to perform the steps of:

receiving at least one user instruction to resume playback of the media and determining an associated timecode;

requesting at least one portion of the video stream from the plurality of alternative streams of video using at least one entry from a retrieved video container index based upon the associated timecode; and

decoding the at least one portion of the video stream from the plurality of alternative streams of video.

12. A method of playing back content on a playback device, comprising:

obtaining top level index information identifying a plurality of alternative streams of video, an audio stream, and at least one trick play stream that are each stored in a separate container file, where;

each video container file containing a given stream from the plurality of alternative streams of video comprises;

portions of the given video stream within the video container file, where the portions of the given video stream comprise an encoded group of pictures that commences with a picture encoded without reference to another picture in the given video stream; and

a video container index, where entries in the video container index indicate sizes of portions of the given video stream within the video container file;

each trick play container file containing a given trick play stream from the at least one trick play stream comprises;

frames of the given trick play stream, where each frame of the given trick play stream is a picture encoded without reference to another picture in the trick play stream; and

a trick play container index, where entries in the trick play container index comprise a timecode and a location of a frame in the given track play stream;

requesting a video container index from a video container file containing a video stream from the plurality of alternative streams of video;

requesting at least one portion of the video stream from the plurality of alternative streams of video using at least one entry from the video container index;

decoding the at least one portion of the video stream from the plurality of alternative streams of video;

receiving at least one user instruction to perform a visual search of the media;

requesting a trick play container index from a trick play container file containing a trick play stream from the at least one trick play stream;

requesting at least one frame of video from the at least one trick play stream using at least one entry from the trick play container index; and

decoding and displaying the at least one frame of video from the at least one trick play stream.

13. The method of claim 12, wherein the at least one trick play stream comprises a plurality of trick play streams.

14. The method of claim 12, wherein a trick play stream from the at least one trick play stream is encoded at a lower bit rate than the plurality of alternative streams of video.

15. The method of claim 12, further comprising performing adaptive streaming with respect to the plurality of trick play streams.

16. The method of claim 12, further comprising:

manipulating at least one timecode assigned to the at least one frame of video from the at least one trick play stream prior to providing the at least one frame to a decoder on the playback device, where the at least one timecode is manipulated in order to achieve a rate of playback based on a desired rate for the visual search; and

decoding and presenting the at least one frame of video from the at least one trick play stream in accordance with the manipulated timecodes.

17. The method of claim 12, further comprising:

detecting streaming conditions for the playback device;

selecting a stream from the plurality of alternative streams of video based on the detected streaming conditions; and

requesting portions of video from the selected stream.

18. The method of claim 17, further comprising:

converting at least one entry in a video container index for a video container file containing the selected stream into a byte range within the video container file containing the selected stream corresponding to at least one portion of the selected stream; and

generating at least one byte range request for the at least one portion of the selected stream.

19. The method of claim 12, wherein the encoded group of pictures is a single closed group of pictures that commences with a picture encoded without reference to another picture in the given video stream.

20. The method of claim 12, wherein:

the top level index information includes information indicating the portion of each video container file containing the video container index for the video container file; and

requesting a video container index from a video container file containing a video stream from the plurality of alternative streams of video comprises requesting the portion of the video container file indicated as containing the video container index for the video container file by the top level index information.

21. The method of claim 12, wherein the at least one user instruction to perform a visual search of the media indicates a time code and the method further comprises requesting at least one frame of video from the at least one trick play stream using at least one entry from the trick play container index based upon the time code.

22. The method of claim 21, further comprising:

receiving at least one user instruction to resume playback of the media and determining an associated timecode;

requesting at least one portion of the video stream from the plurality of alternative streams of video using at least one entry from a retrieved video container index based upon the associated timecode; and

decoding the at least one portion of the video stream from the plurality of alternative streams of video.

23. A playback device, comprising:

a set of one or more processors; and

a non-volatile storage containing an application for causing the set of one or more processors to perform the steps of;

obtaining top level index information identifying a plurality of alternative streams of video, an audio stream, and at least one trick play stream that are each stored in a separate container file, where;

each video container file containing a given stream from the plurality of alternative streams of video comprises;

portions of the given video stream within the video container file, where the portions of the given video stream comprise an encoded group of pictures that commences with a picture encoded without reference to another picture in the given video stream; and

a video container index, where entries in the video container index indicate sizes of portions of the given video stream within the video container file;

each trick play container file containing a given trick play stream from the at least one trick play stream comprises;

frames of the given trick play stream, where each frame of the given trick play stream is a picture encoded without reference to another picture in the trick play stream; and

a trick play container index, where entries in the trick play container index comprise a timecode and a location of a frame in the given track play stream;

requesting a video container index from a video container file containing a video stream from the plurality of alternative streams of video;

requesting at least one portion of the video stream from the plurality of alternative streams of video using at least on entry from the video container index;

decoding the at least one portion of the video stream from the plurality of alternative streams of video;

receiving at least one user instruction to perform a visual search of the media indicating a time code;

requesting a trick play container index from a trick play container file containing a trick play stream from the at least one trick play stream;

requesting at least one frame of video from the at least one trick play stream using at least one entry from the trick play container index based upon the indicated time code;

manipulating at least one timecode assigned to the at least one frame of video from the at least one trick play stream, where the at least one timecode is manipulated in order to achieve a rate of playback based on a desired rate for the visual search; and

decoding and displaying the at least one frame of video from the at least one trick play stream in accordance with the at least one manipulated timecode.