Adaptive media transport management for continuous media stream over LAN/WAN environment

• US 7,984,179 B1
• Filed: 10/11/2005
• Issued: 07/19/2011
• Est. Priority Date: 06/29/2004
• Status: Expired due to Fees

• Current Assignee: Sextant Navigation Incorporated
• Original Assignee: Sextant Navigation Incorporated
• Inventors: Huang, Jeffrey
• Primary Examiner(s): Tang; Karen C
• Application Number:
  US11/247,054
• Time in Patent Office: 2,107 Days
• Field of Search: 709/233, 709/231, 709/232, 709/235
• US Class Current: 709/233
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10

Abstract

Methods and systems for transmitting compressed audio/video data streams across conventional networks or channels in real time. Each system employs the Adaptive Rate Control (ARC) technique that is based on the detection of defective packets and other influencing factors such as overall system performance, usage model and channel characteristics. To control data stream congestions in the channel and maintain the visual display quality above a certain level, the present invention exploits the human visual perception and adaptability to changing visual conditions. The ARC technique relies on the client'"'"'s capacity for calculating the quality of video packets received from a server and sending information of a desired transmission rate to the server. This approach simplifies the hardware and software implementation complexity otherwise imposed on the server and reduces the overall design cost by shifting the burden of monitoring bandwidth and transmission control from the server to the client.

Claims

1. A method for controlling a transmission rate of a data stream from a server to a client in real time, wherein the client generates a look-up table by using a set of dynamically-changing client characteristics and accesses the look-up table by using a dynamically-updated data packet loss ratio as a look-up table key, the method comprising:

receiving a first portion of the data stream from the server at a first transmission rate, the first portion including a plurality of data packets;

checking integrity of the data packets;

deriving the dynamically-updated data packet loss ratio based on the integrity;

generating a client request for an adjustment of the first transmission rate to the server, wherein the client independently determines an adjusted transmission rate based on the dynamically-updated data packet loss ratio and one or more dynamically-changing values in the look-up table in the client associated with the set of dynamically-changing client characteristics, which changes during a process of data transmission from the server to the client;

sending the client request to the server;

processing the client request in an intelligent video transport manager associated with the server for the adjustment of the first transmission rate or an adjustment for a desired Luminance-Chrominance (YUV) conversion format for more video color content information to less video color content information or vice versa; and

receiving a second portion of the data stream at the adjusted transmission rate from the server.

2. The method of claim 1, further comprising, prior to the step of receiving the first portion of the data stream:

establishing a connection between the client and the server;

causing the client to send the one or more values from the set of client characteristics to the server; and

causing the server to determine the first transmission rate using the one or more values from the set of client characteristics sent by the client and transmit the first portion at the first transmission rate to the client.

3. The method of claim 1, wherein the set of client characteristics contained in the look-up table include the desired Luminance-Chrominance (YUV) conversion format for more video color content information to less video color content information or vice versa, depending on a current value of the dynamically-updated data packet loss ratio.

4. The method of claim 1, wherein the data stream is a multiplexed data stream including an audio stream and a video stream, further comprising, prior to the step of checking integrity:

de-multiplexing the first portion of the data stream into the audio stream and the video stream.

5. The method of claim 4, further comprising, prior to the step of checking integrity:

decoding the audio stream and the video stream.

6. The method of claim 1, wherein the step of deriving the dynamically-updated data packet loss ratio includes the step of:

taking a percentage of defective data packets in the plurality of data packets, wherein each of the defective data packets is a corrupt data packet or a data packet lost in the step of receiving the first portion of the data stream.

7. The method of claim 1, wherein the step of generating the client request for the adjustment of the first transmission rate includes steps of:

determining whether the dynamically-updated data packet loss ratio indicates a congested state;

if the dynamically-updated data packet loss ratio indicates a congested state;

generating the client request for a decrease in the first transmission rate; and

determining whether the dynamically-updated data packet loss ratio indicates an unloaded state, a maximum transmission rate, or a steady state;

else if the dynamically-updated data packet loss ratio does not indicate an unloaded state;

generating the client request including a No-Operation (NOP) signal; and

else if the dynamically-updated data packet loss ratio indicates the unloaded state;

determining whether the first transmission rate has reached the maximum transmission rate;

if the first transmission rate has not reached a maximum transmission rate;

generating the client request for an increase of the first transmission rate; and

if the first transmission rate has reached the maximum transmission rate;

generating the client request including a No-Operation (NOP) signal.

8. The method of claim 1, wherein the data stream is formatted according to a user datagram protocol (UDP).

9. The method of claim 1, wherein the data stream is transmitted through a network, the network including at least one of a local area network (LAN) and a wide area network (WAN).

10. The method of claim 1, wherein the client includes a visual device.

11. The method of claim 1, further comprising:

repeating from the step of checking integrity to the step of receiving the second portion until an entire portion of the data stream is received by the client.

12. The method of claim 1, wherein the first transmission rate has a set of parameters including at least one of a target frame rate, a target display bit rate, a target resolution, a target audio sampling rate, a quantization value, a Luminance-Chrominance (YUV) conversion, a scale size and a connection history describing a previous state of connection.

13. A method for controlling a transmission rate of a data stream from a server to a client in real time, wherein the client generates a look-up table associated with client characteristics and accesses the look-up table by using a dynamically-updated data packet loss ratio as a look-up table key, the method comprising:

receiving a client request for an adjustment of the transmission rate or an adjustment for a desired Luminance-Chrominance (YUV) conversion format for more video color content information to less video color content information or vice versa from the client, wherein the client uses the dynamically-updated data packet loss ratio accumulated by the client as the look-up table key to access the one or more values in the look-up table to generate the client request, and wherein the client independently determines an adjusted transmission rate based on the dynamically-updated data packet loss ratio and one or more dynamically-changing values in the look-up table in the client associated with a set of dynamically-changing client characteristics, which changes during the data stream from the server to the client in real time;

receiving at least one audio input and at least one video input via an audio/video multiplexer system block; and

accepting the client request from the client for the adjustment of the transmission rate or the adjustment for the desired Luminance-Chrominance (YUV) conversion format for more video color content information to less video color content information or vice versa, wherein the step of accepting the client request is ultimately up to an intelligent video transport manager regardless of the client request for the adjustment;

processing the client request in the intelligent video transport manager for the adjustment of the transmission rate or the adjustment for the desired Luminance-Chrominance (YUV) conversion format for more video color content information to less video color content information or vice versa, wherein the processing of the client request in the intelligent video transport manager comprises;

encoding the audio input and the video input in accordance with

the client request;

multiplexing the encoded audio and video inputs into the data stream in accordance with the client request; and

transmitting the data stream to the client in accordance with the request from the client.

14. The method of claim 13, wherein the look-up table generated by the client further includes at least one of a display size, a maximum resolution, a color depth, a display type, a CPU type, one or more decoder capabilities, an available memory, a connection type, a previous connection history having transmission information, and a video display line structure.

15. An apparatus for real-time adaptive media transport management of continuous streaming of data packets, the apparatus comprising:

a CPU and a memory unit operatively connected to a server;

the server transmitting a first portion of a stream of multimedia data packets to a client via a LAN or WAN, wherein the server contains an intelligent video transport manager providing an adjustable data rate or an adjustable desired Luminance-Chrominance (YUV) conversion format for more video color content information to less video color content information or vice versa, depending on a current level of data packet loss ratio provided by the client;

the client receiving the first portion of the stream of the multimedia data packets from the server via the LAN or WAN, wherein the client has a look-up table containing information to determine a need for adjusting a YUV conversion format for a second portion of the multimedia data packets which is about to be transmitted to the client, and wherein the client independently requests to the server for an adjustment of the adjustable rate and the adjustable desired YUV conversion format based on a dynamically-updated data packet loss ratio and one or more dynamically-changing values in the look-up table in the client associated with a set of dynamically-changing client characteristics, which changes during the continuous streaming of data packets from the server to the client; and

the LAN or WAN operatively connected to the server and the client, wherein the LAN or WAN provides a pathway for transmission of data packets.

16. The apparatus of claim 15, wherein the server further comprises an audio/video multiplexer system to receive and encode an audio/video input stream, and a MAC buffer to place an encoded stream of multimedia data packets prior to transmission to the client.

17. The apparatus of claim 15, wherein the client further comprises an incoming buffer to accumulate the encoded stream of multimedia data packets transmitted by the client and an audio/video de-multiplexer system to decode at least some portions of the encoded stream of multimedia data.

18. The apparatus of claim 17, wherein the client further comprises a packet error accumulator calculating the data packet loss ratio in real-time, which is used as a look-up table key in the look-up table in the client.

19. The apparatus of claim 15, wherein the client sends a request to the server for adjusting the YUV conversion format for the second portion of the multimedia data packets which is about to be transmitted to the client by the server.

20. The apparatus of claim 19, wherein the server adjusts the YUV conversion format for the second portion of the multimedia data packets.