Enhanced distortion signaling for MMT assets and ISOBMFF with improved MMT QoS descriptor having multiple QoE operating points

• US 9,788,078 B2
• Filed: 11/18/2014
• Issued: 10/10/2017
• Est. Priority Date: 03/25/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Li, Zhu, Bouazizi, Imed, Park, Kyungmo
• Primary Examiner(s): PIERORAZIO, MICHAEL
• Application Number:
  US14/546,906
• Publication Number:
  US 20150281801A1
• Time in Patent Office: 1,057 Days
• Field of Search: 725109
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/2404 Monitoring of server processing errors or hardware failure error or fault detection G06F11/07; monitoring in general G06F11/30
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16

Abstract

A method for providing media content in a computer network includes storing the media content, where the media content includes a segment having a group of frames. The method also includes determining a transmission rate for traffic to a client device. The method further includes selecting a subset of frames to drop from the group of frames based on (i) the transmission rate and (ii) a frame difference distortion (FDIFF) metric of each frame in the subset of frames. The method also includes shaping the segment by dropping the selected subset of frames from the group of frames, where the shaped segment has a lower bitrate than the segment. In addition, the method includes transmitting the shaped segment to the client device.

Claims

1. An apparatus for providing media content in a computer network, the apparatus comprising:

a memory configured to store the media content, the media content including a segment comprising a group of frames; and

at least one processing device configured to;

determine a transmission rate for traffic between the apparatus and a client device;

select a subset of frames to drop from the group of frames based on (i) the transmission rate and (ii) a frame difference distortion (FDIFF) metric of each frame in the subset of frames;

calculate a frame significance (FSIG) value that indicates a relative importance of the frames in the group of frames in a sequence, wherein the FSIG value is defined according to an equation;

v_k=d(f_k, f<sub>k−

1</sub>)=(S·

f_k−

S·

f<sub>k−

1</sub>)^TA^TA(S·

f_k−

S·

f<sub>k−

1</sub>),

where v_k is a vector representation of the FSIG value, f_k denotes a k^th frame, f<sub>k−

1</sub>, denotes a previous frame, S denotes a bi-cubicle filtering and down-sampling function, and A denotes a metric,

shape the segment by dropping the selected subset of frames from the group of frames, wherein the shaped segment has a lower bitrate than the segment; and

initiate transmission of the shaped segment to the client device.

2. The apparatus of claim 1, wherein the FSIG value defines a temporal distortion of a frame corresponding to a timestamp t if the frame corresponding to the timestamp is lost.

3. The apparatus of claim 2, wherein the at least one processing device is configured to select, from the group of frames, a frame whose FSIG value is less than a threshold significance level.

4. The apparatus of claim 1, wherein the at least one processing device is further configured to determine a sequence activity level within the segment using a differential significance of frames in the group of frames.

5. The apparatus of claim 1, wherein the at least one processing device is further configured to:

receive the QoS description and the QoE description aggregated as the ADC, wherein each set of QoE parameters includes;

a frame drop index set indicating a list of frames to drop from the group of frames yielding a bitrate reduction corresponding to the associated operating point,

an aggregate bitrate resulting from the bitrate reduction obtained by dropping the frames indicated by the frame drop index set,

a spatial distortion metric of the segment, and

a frame significance weighted frame loss temporal distortion metric of the segment based on FSIG values; and

select the operating point whose aggregate bitrate is closest to the transmission rate,

wherein the at least one processing device is configured to select the subset of frames as the list of frames indicated by the frame drop index set of the selected operating point.

6. An apparatus for providing media content in a computer network, the apparatus comprising:

a memory configured to store the media content, the media content including a segment comprising a group of frames; and

at least one processing device configured to;

determine a transmission rate for traffic between the apparatus and a client device;

select a subset of frames to drop from the group of frames based on (i) the transmission rate and (ii) a frame difference distortion (FDIFF) metric of each frame in the subset of frames;

calculate the FDIFF metric according to an equation;

d(f_j, f_k)=(S·

f_j−

S·

f_k)^TA^TA(S·

f_j−

S·

f_k),

where f_j denotes a frame actually being displayed, f_k denotes a frame scheduled to be displayed, S denotes a bi-cubicle filtering and down-sampling function, and A denotes a metric;

shape the segment by dropping the selected subset of frames from the group of frames, wherein the shaped segment has a lower bitrate than the segment; and

initiate transmission of the shaped segment to the client device.

7. The apparatus of claim 6 wherein the at least one processing device is configured to select, from the group of frames, a frame whose FDIFF metric is less than a threshold distortion level.

8. An apparatus for providing media content in a computer network, the apparatus comprising:

a memory configured to store the media content, the media content including a segment comprising a group of frames; and

at least one processing device configured to;

determine a transmission rate for traffic between the apparatus and a client device;

select a subset of frames to drop from the group of frames based on (i) the transmission rate and (ii) a frame difference distortion (FDIFF) metric of each frame in the subset of frames;

shape the segment by dropping the selected subset of frames from the group of frames, wherein the shaped segment has a lower bitrate than the segment; and

initiate transmission of the shaped segment to the client device

wherein;

a frame loss temporal distortion (FLTD) metric is an F SIG-weighted FLTD metric based on frame significance (F SIG) values; and

the at least one processing device is configured to calculate the F SIG-weighted FLTD metric according to an equation;

9. The apparatus of claim 8, wherein the at least one processing device is further configured to calculate the FSIG-weighted FLTD metric based on a number of dependent frames having a decoding dependency from an encoded frame in order to increase the FSIG-weighted FLTD metric for each of the dependent frames.

10. A system for providing media content in a computer network, the system comprising:

a memory configured to store the media content, the media content including a segment comprising a group of frames; and

at least one processing device configured to;

generate multiple operating points of bitrate reduction by performing a gradient search for each of the operating points;

generate a set of Quality of Experience (QoE) parameters for each of the operating points, wherein the set QoE parameters for each of the operating points includes;

a frame drop index set indicating a list of frames that when dropped from the group of frames yield (i) an aggregate bitrate reduction corresponding to the associated operating point and (ii) a shaped segment containing remaining frames; and

at least one of;

an aggregate bitrate resulting from dropping the frames indicated by the frame drop index set,

a spatial distortion metric of the segment, and

a frame loss temporal distortion (FLTD) metric of the segment;

calculate a frame significance (FSIG) value that indicates a relative importance of the frames in the group of frames in a sequence, the FSIG value defined according to an equation;

v_k=d(f_k, f<sub>k−

1</sub>)=(S·

f_k−

S·

f<sub>k−

1</sub>)^TA^TA(S·

f_k−

S·

f<sub>k−

1</sub>),

where v_k is a vector representation of the FSIG value, f_k denotes a k^th frame, f<sub>k−

1</sub> denotes a previous frame, S denotes a bi-cubicle filtering and down-sampling function, and A denotes a metric; and

initiate transmission of an Asset Delivery Characteristic (ADC) of the media content, the ADC including the operating points and the set of QoE parameters corresponding to each of the operating points.

11. The system of claim 10, wherein the at least one processing device is further configured to calculate for each frame in the segment:

a frame rate reduction;

a frame distortion; and

a frame loss gradient comprising a ratio of the frame distortion to the frame rate reduction.

12. The system of claim 11, wherein the at least one processing device is configured, when performing the gradient search for each of the operating points, to:

sort the group of frames by frame loss gradient; and

generate a list of frames that when dropped from the group of frames yields an aggregate bitrate reduction corresponding to the operating point, the at least one processing device configured to generate the list of frames by (i) adding a frame having a smallest frame loss gradient to the list of frames and (ii) repeatedly adding a next frame from the sorted group of frames to the list of frames until a sum of the frame rate reductions of the frames added to the list of frames is at least the bitrate reduction corresponding to the operating point.

13. The system of claim 10, wherein the frame drop index set comprises one of:

an index of each remaining frame; and

an index of the frames to be dropped from the group of frames.

14. The system of claim 10, wherein the at least one processing device is configured to select, from the group of frames, a frame having at least one of:

an FSIG value less than a threshold significance level; and

a FLTD metric less than a threshold distortion level.

15. The system of claim 10, wherein:

the FLTD metric is an FSIG-weighted FLTD metric; and

the at least one processing device is configured to calculate the F SIG-weighted FLTD metric according to an equation;

16. The system of claim 15, wherein the at least one processing device is further configured to calculate the FSIG-weighted FLTD metric based on a number of dependent frames having a decoding dependency from an encoded frame in order to increase the FSIG-weighted FLTD metric for each of the dependent frames.

17. The system of claim 10, further comprising a router configured to:

receive the transmitted ADC;

determine a transmission rate for traffic between the router and a client device;

select a subset of frames to drop from the group of frames based on (i) the transmission rate and (ii) a frame loss temporal distortion (FLTD) metric of each frame in the subset of frames;

shape the segment by dropping the selected subset of frames from the group of frames, wherein the shaped segment has a lower bitrate than the segment; and

initiate transmission of the shaped segment to the client device.

18. The system of claim 17, wherein:

the router is further configured to select the operating point whose aggregate bitrate is closest to the transmission rate; and

the router is configured to select the subset of frames as a list of frames indicated by a frame drop index set associated with the selected operating point.