Packed I-frames

• US 10,715,776 B2
• Filed: 06/20/2017
• Issued: 07/14/2020
• Est. Priority Date: 05/30/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Pantos, Roger N., Li, Zhenheng
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Park, Sunghyoun
• Application Number:
  US15/628,073
• Publication Number:
  US 20170289514A1
• Time in Patent Office: 1,120 Days
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Systems and methods for enabling playback control functions of a media player are disclosed. For example, a user of a client device receiving streaming playback of a video stream may perform rewind and fast forward control functions. The client device may implement these playback control functions by retrieving an enhanced playback segment. Using the enhanced playback segment, the media player may display selected frames at a predetermined interval while maintaining a visual cadence that is pleasing to a viewer. In the described embodiments, a client device may render a video stream, receive a command to control a fast forward or rewind playback mode for the video stream, and retrieve, from a distribution server or associated edge cache, one or more enhanced playback segments adapted to implement the user command.

Claims

1. A method of streaming a media asset at a media player, comprising:

storing, at a media source, first segments of a first media stream for the media asset, the first segments containing video data of frames of the media asset coded according to temporal prediction techniques;

storing, at the media source, second segments of a second media stream for the media asset that contain frames of the media asset coded solely as intra-coded (I) frames, wherein the second segments contains frames of the media asset that are redundant to corresponding frames of the first segments and wherein at least one I frame in the second segments corresponds a frame in the first segments that is not an I frame;

storing, at the media source, manifest data that contains for the first media stream;

an identifier of a network location from which each of the first segments may be downloaded,

configuration information for the first media stream including a bitrate of the stream,

an identifier of a network location from which each of the second segments may be downloaded, and

an indication that both the first segments and the second segments are associated with the first media stream, that the first segments are normal playback segments, and the that second segments are enhanced playback segments; and

providing, by the media source, data from the first segments and the second segments in response to request(s) from a client device.

2. The method of claim 1, wherein the media asset is live streamed content.

3. The method of claim 1, wherein the media asset includes:

a plurality of media streams, each associated with its own bandwidth, and

a plurality of sets of second segments, each associated with a respective media stream.

4. The method of claim 1, wherein the manifest data correlates segments of the first and second media streams with corresponding segments having varying bitrates.

5. The method of claim 1, wherein the manifest data identifies first segments as normal playback type segments and second segments as enhanced playback type segments.

6. The method of claim 1, wherein the frames of the second segments are redundant to a cadence of frames spaced at a predetermined interval of the first segments.

7. The method of claim 1, further comprising:

re-encoding frames of the first media stream segments to create the I-frames of the second media stream segments, wherein the frames selected from the first media stream segments for inclusion in the second media stream segments are spaced at a cadence shorter than intervals between I-frames in the first media stream segments.

8. The method of claim 1, wherein the at least one I frame in the second segments is re-encoded from the corresponding frame in the first segments that is not an I frame.

9. The method of claim 1, wherein the cadence of first media stream corresponding to second media stream frames is shorter that the cadence of I frames in the first media stream.

10. The method of claim 1, wherein the frames of the second media stream are in reverse order of their corresponding frames of the first media stream.

11. A non-transitory computer readable medium having instructions for storing a media stream, the instructions when executed by at least one processing system, cause the at least one processing system to:

store, at a media source, first segments of a first media stream, the first segments containing video data of frames of a media asset coded according to temporal prediction techniques;

store, at the media source, second segments that contain frames of the media asset coded solely as intra-coded (I) frames, wherein the second segments contains frames of the media asset that are redundant to frames of the first segments and wherein at least one I frame in the second segments corresponds a frame in the first segments that is not an I frame;

storing, at the media source, manifest data that contains for the first media stream;

an identifier of a network location from which each of the first segments may be downloaded,

configuration information for the first media stream including a bitrate of the stream, and

an identifier of a network location from which each of the second segments may be downloaded, and

an indication that both the first segments and the second segments are associated with the first media stream, that the first segments are normal playback segments, and the that second segments are enhanced playback segments; and

providing, by the media source, data from the first segments and the second segments in response to request(s) from a client device.

12. The non-transitory computer readable medium of claim 11, wherein the media asset is live streamed content.

13. The non-transitory computer readable medium of claim 11, wherein the media asset includes:

a plurality of media streams, each associated with its own bandwidth, and

a plurality of sets of second segments, each associated with a respective media stream.

14. The non-transitory computer readable medium of claim 11, wherein the manifest data correlates segments of the first and second media streams with corresponding segments having varying bitrates.

15. A distribution server comprising:

a processing system;

memory storing one or more programs for execution by the processing system, the one or more programs including instructions for;

storing first segments of a first media stream, the first segments containing video data of frames of a media asset coded according to temporal prediction techniques;

storing second segments that contain frames of the media asset coded solely as intra-coded (I) frames, wherein the second segments contains frames of the media asset that are redundant to frames of the first segments and wherein at least one I frame in the second segments corresponds a frame in the first segments that is not an I frame;

storing, at the media source, manifest data that contains for the first media stream;

an identifier of a network location from which each of the first segments may be downloaded,

configuration information for the first media stream including a bitrate of the stream, and

an identifier of a network location from which each of the second segments may be downloaded, and

an indication that both the first segments and the second segments are associated with the first media stream, that the first segments are normal playback segments, and the that second segments are enhanced playback segments; and

providing, by the media source, data from the first segments and the second segments in response to request(s) from a client device.

16. The distribution server of claim 15, wherein the media asset is live streamed content.

17. The distribution server of claim 15, wherein the media asset includes:

a plurality of media streams, each associated with its own bandwidth, and

a plurality of sets of second segments, each associated with a respective media stream.

18. The distribution server of claim 15, wherein the manifest data correlates segments of the first and second media streams with corresponding segments having varying bitrates.

19. A method of rendering a media asset at a media player, comprising:

receiving, at the media player, a manifest file of the media asset, the manifest file containing information identifying;

at least one coded representation of the media asset comprised of a plurality of first transmission segments, each first transmission segment containing coded video data of the media asset at a first temporal spacing, the manifest file identifying media source resource addresses respectively of the first transmission segments, and

a packed I-frame representation of the media asset comprised of a plurality of second transmission segments, each second transmission segment containing coded video data of the media asset at a second temporal spacing greater than the first temporal spacing, wherein all frames of the second transmission segments are redundant to corresponding frames of the first transmission segments and are coded by intra-coding, at least one I frame in the second segments corresponds a frame in the first segments that is not an I frame, and the manifest file identifying media source resource addresses respectively of the second transmission segments;

during a normal playback mode;

selecting, from the manifest file, first transmission segment(s) that contain coded video data of the media asset to be rendered according to the normal playback mode; and

issuing retrieval requests to the media source at the resource address(es) of the selected first transmission segment(s);

during an enhanced playback mode;

selecting, from the manifest file, second transmission segment(s) that contain coded video data of the media asset to be rendered according to the enhanced playback mode,

issuing retrieval requests to the media source at the resource address(es) of the selected second transmission segment(s);

extracting intra-coded frames from retrieved second transmission segment(s) based on a playback speed of the enhanced playback mode;

decoding, at the media player, the coded video data of the selected intra-coded frames; and

rendering the decoded video data on a display of the media player.