METHOD FOR AUDIO AND VIDEO CONTROL RESPONSE AND BANDWIDTH ADAPTATION BASED ON NETWORK STREAMING APPLICATIONS AND SERVER USING THE SAME

• US 20100161761A1
• Filed: 06/22/2009
• Published: 06/24/2010
• Est. Priority Date: 12/22/2008
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Kuo, Hui-Ping, Yu, Jen-Yu, Lee, Hsin-Hua
• Granted Patent: US 8,554,879 B2
• Time in Patent Office: Days
• US Class Current: 709/219
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: H04N 21/2401 Monitoring of the client buffer
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A method for audio and video control response and bandwidth adaptation based on network streaming applications is disclosed. A server delivers a received control command to a stream controller while a player empties the expired streaming data stored in a buffer thereof. The stream controller controls the input source of a video/audio capturer. The receiver buffer estimator of the receiver estimates whether the amount of the streaming data stored in the buffer is within a safe range and delivers the estimation result and bandwidth variation information to a stream adaptor. If the amount of the stream data is less than a threshold value, the timestamp scalar is adjusted based on the estimation result and the bandwidth variation information, thereby adjusting the consumption rate of the streaming data in the buffer of the player for re-accumulating streaming data of the buffer.

Claims

1. A method for audio and video control response and bandwidth adaptation based on network streaming applications, comprising:

transmitting a control command received by a stream communicator of the server to a stream controller of the server;

the player can either discover the expiration of the buffered streaming data by itself and empty its buffer or flush the expired buffered data notified with the buffer expiration message sent by the communicator of the server;

enabling the stream controller to control an input source of a video/audio capturer;

estimating whether the amount of streaming data stored in the buffer is within a safe range;

delivering the estimation result and bandwidth variation information detected by the server to a stream adaptor;

determining whether the amount of the stream data stored in the buffer is less than a threshold value; and

enabling the stream adaptor to adjust a timestamp scalar according to the estimation result and the bandwidth variation information if the amount of the stream data is less than the threshold value, thereby adjusting a consumption rate of the streaming data buffered in the player for re-accumulating streaming data of the buffer.

2. The method for audio and video control response and bandwidth adaptation based on network streaming applications as claimed in claim 1, wherein the timestamp scalar is restored to a default value when the streaming data of the buffer is accumulated to a predetermined amount.

3. The method for audio and video control response and bandwidth adaptation based on network streaming applications as claimed in claim 1, wherein one of the settings in a quality profile is selected according to the estimation result and the bandwidth variation information to adjust the bit rate of the streaming data if the amount of the streaming data buffered in the player is less than the threshold value.

4. A server, comprising:

a video/audio capturer for selecting an input source of stream data;

a QoS controller, further comprising a receiving buffer estimator and a congestion detector;

a communicator for receiving a control command from a player;

a stream controller for receiving the control command from the communicator, enabling the player to empty the expired streaming data stored in a buffer of the player, controlling the input source and streaming contents of a video/audio capturer, and resetting the receiving buffer estimator; and

a stream adaptor, further comprising a profile selector, a quality profile, and a timestamp adjuster,

wherein the receiving buffer estimator determines whether the amount of stream data stored in the buffer of the player is within a safe range, the stream adaptor retrieves the estimation result and bandwidth variation information detected by the congestion detector and determines whether the amount of the streaming data stored in the buffer of the player is less than a threshold value, and

the profile selector adjusts a timestamp scalar according to the estimation result and the bandwidth variation information if the amount of the stream data is less than the threshold value, thereby adjusting a consumption rate of the streaming data stored in the buffer for re-accumulating streaming data of the buffer.

5. The server as claimed in claim 4, wherein the receiving buffer estimator restores the timestamp scalar to a default value when the streaming data of the buffer is accumulated to a predetermined amount.

6. The server as claimed in claim 4, wherein the profile selector selects one of the settings in a quality profile according to the estimation result and the bandwidth variation information to adjust bit rate of the streaming data if the amount of the streaming data in the buffer of the player is less than the threshold value.

7. The server as claimed in claim 4, wherein a reset command is sent, when the stream controller receives the control command, to enable the receiving buffer estimator to reset a parameter used for determining the buffer of the player.

8. The server as claimed in claim 4, further comprising a stream encoder, wherein the stream controller sets encoding settings of the stream encoder, extracting settings of the video/audio extractor, or an input source and stream contents of an input switcher.

9. The server as claimed in claim 8, further comprising a stream sender, wherein the congestion detector receives feedback information generated by transmitting the streaming data by the stream sender to determine whether a network is congested.

10. The server as claimed in claim 9, wherein the QoS controller enables the stream adaptor to select one of the settings in a quality profile according to the network congestion information for adapting bandwidth.

11. The server as claimed in claim 10, wherein the QoS controller estimates the capability to endure bandwidth variation for the player using the receiving buffer estimator.

12. The server as claimed in claim 11, wherein the receiving buffer estimator estimates the streaming data length buffered in the player to determine whether video/audio quality and the timestamp scalar should be adjusted.

13. The server as claimed in claim 4, wherein the settings in the quality profile comprise an encoding bit rate, resolution, a frame rate, frame drop option, and the timestamp scalar.

14. The server as claimed in claim 13, further comprising an audio re-sampler for re-sampling the audio data captured by the video/audio capturer when the timestamp adjuster has adjusted the timestamp scalar.

15. The server as claimed in claim 14, wherein video data of the stream data and the re-sampled audio data are delivered to the stream encoder and the output streaming data are encoded based on the selected setting from the quality profile.

16. The server as claimed in claim 15, wherein the stream sender transmits the encoded streaming with timestamp which is adjusted based on the timestamp scalar to the player and the feedback information retrieved by the stream sender is returned to the QoS controller.

17. A computer-readable medium encoded with computer executable instructions for performing a method for audio and video control response and bandwidth adaptation based on network streaming applications, wherein the computer executable instructions comprise:

transmitting a control command received by stream communicator of the server to a stream controller of the server to enable a player to empty the expired streaming data stored in a buffer of the player;

enabling the stream controller to control the input source of a video/audio capturer;

estimating whether the amount of streaming data stored in the buffer is within a safe range;

delivering the estimation result and bandwidth variation information detected by the server to a stream adaptor;

determining whether the amount of the streaming data stored in the buffer is less than a threshold value; and

enabling the stream adaptor to adjust the timestamp scalar based on the estimation result and the bandwidth variation information if the amount of the streaming data is less than the threshold value, thereby adjusting the consumption rate of the streaming data buffered in the player for re-accumulating streaming data of the buffer.

18. The computer-readable medium as claimed in claim 17, wherein the timestamp scalar is restored to a default value when the streaming data of the buffer is accumulated to a predetermined amount.

19. The computer-readable medium as claimed in claim 17, wherein one of the settings in a quality profile is selected according to the estimation result and the bandwidth variation information to adjust the bit rate of the streaming data if the amount of the streaming data in the buffer of the player is less than the threshold value.