Adaptive real-time transcoding method and streaming server therefor

• US 10,003,626 B2
• Filed: 10/22/2014
• Issued: 06/19/2018
• Est. Priority Date: 10/22/2013
• Status: Active Grant

• Current Assignee: Industry Academic Cooperation Foundation
• Original Assignee: Industry Academic Cooperation Foundation
• Inventors: Kim, Sung Ho, Park, Keun Baek, Chang, Joon Kee, Cho, Sung Tak, Ro, Won Woo, Oh, Doo Hwan, Kim, Keun Soo, Cho, Young Jae
• Primary Examiner(s): Nguyen, Tammy
• Application Number:
  US15/030,969
• Publication Number:
  US 20160269462A1
• Time in Patent Office: 1,336 Days
• Field of Search: 709231, 709242, 370332, 345425
• CPC Class Codes: H04L 47/38 by adapting coding or compression rate
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/238 Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2401 Monitoring of the client buffer
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/64769 for rate control flow control in packet networks H04L47/10

Abstract

The present invention relates to an adaptive real-time transcoding method and a streaming server therefor to improve service quality by minimizing the disconnection of images in an environment, such as a mobile device, through adaptively changing a set value of a transcoding operation on a media streaming service to match a network environment. The adaptive real-time transcoding method of the present invention is a real-time transcoding method for sequentially transmitting a media data segment, divided and transcoded, in a streaming server to the client side over a network, the method comprising the steps of: determining the fill level of a client-side playback buffer; determining, on the basis of the fill level of the playback buffer, the image quality of media data segments that are not transmitted to the client side; and transcoding the non-transmitted media data segments according to the determined image quality.

Claims

1. An adaptive real-time transcoding method for segmenting, at a streaming server, media data into media data segments, transcoding the media data segments, and sequentially transmitting the media data segments to a client end over a network, the method comprising:

determining a fill level of a playback buffer of the client end by comparing a sum of playback times of transmitted media data segments with an elapse time from a streaming start time;

determining a quality of a media data segment that is not transmitted to the client end, based on the fill level of the playback buffer; and

transcoding the non-transmitted media data segment based on the determined quality.

2. The method of claim 1, further comprising:

temporarily storing the segmented and transcoded media data segments in a transcoding buffer,

wherein the transcoding comprises;

extracting a media data segment stored in the transcoding buffer; and

re-transcoding the extracted media data segment based on the determined quality.

5. The method according to claim 1, wherein the fill level of the playback buffer is a reference value used at the streaming server to estimate whether the playback buffer is empty or whether the playback buffer becomes to be empty after a preset period of time is elapsed.

6. The method of claim 1, wherein, in response to the sum of playback times of transmitted media data segments being detected to be less than or equal to the elapse time from the streaming start time, the streaming server transcodes the media data segment at a quality lower than the quality of the transmitted media data segment.

7. The method of claim 1, wherein, in response to the sum of playback times of transmitted media data segments being greater than the elapse time of the streaming start time, the streaming server alters a tuning ratio of each quality based on an additional elapse time from a point in time at which the sum of playback times of transmitted media data segments is detected to be less than or equal to the streaming start time.

8. The method of claim 7, wherein the streaming server finely tunes the quality in response to the additional elapse time being greater than a threshold reference, and tunes the quality at a relatively high tuning ratio in response to the additional elapse time being less than the threshold reference.

3. An adaptive real-time transcoding method comprising:

segmenting, at a streaming server, media data into media data segments and transcoding the media data segments; and

sequentially transmitting, at the stream server, the media data segments to a client end over a network,

wherein a quality of each of the media data segments is tuned in real time at the streaming server based on a fill level of a playback buffer of the client end, and

wherein the fill level of the playback buffer is determined by comparing a sum of playback times of transmitted data segments with an elapse dine from a streaming start time.

4. The method of claim 3, wherein each of the media data segments includes a playback time and quality data in a header as metadata, and

the quality data includes information about the real-time tuned quality.

12. The method according to claim 3, wherein the fill level of the playback buffer is a reference value used at the streaming server to estimate whether the playback buffer is empty or whether the playback buffer becomes to be empty after a preset period of time is elapsed.

13. The method of claim 3, wherein, in response to the sum of playback times of transmitted media data segments being detected to be less than or equal to the elapse time from the streaming start time, the streaming server transcodes the media data segment at a quality lower than the quality of the transmitted media data segment.

14. The method of claim 3, wherein, in response to the sum of playback times of transmitted media data segments being greater than the elapse time of the streaming start time, the streaming server alters a tuning ratio of each quality based on an additional elapse time from a point in time at which the sum of playback times of transmitted media data segments is detected to be less than or equal to the streaming start time.

15. The method of claim 14, wherein the streaming server finely tunes the quality in response to the additional elapse time being greater than a threshold reference, and tunes the quality at a relatively high tuning ratio in response to the additional elapse time being less than the threshold reference.

9. A streaming server for sequentially transmitting media data segments to a playback buffer of a client end for a streaming service, the streaming server comprising:

a playback buffer estimator configured to determine a fill level of the playback buffer of the client end by comparing a sum of playback times of transmitted media data segments with an elapse time from a streaming start time;

a quality controller configured to determine a quality of a media data segment that is not transmitted to the client end, based on the fill level of the playback buffer; and

a transcoder configured to transcode the non-transmitted media data segment based on the determined quality.

10. The streaming server of claim 9, further comprising:

a transcoding buffer configured to temporarily store media data segments corresponding to a transcoding result of the transcoder,

wherein the transcoder is configured to extract a media data segment stored in the transcoding buffer, and to re-transcode the extracted media data segment based on the determined quality.

11. The streaming server of claim 9, wherein the fill level of the playback buffer is a reference value used at the streaming server to estimate whether the playback buffer is empty or whether the playback buffer becomes to be empty after a preset period of time is elapsed.