Multimedia networked system detecting congestion by monitoring buffers' threshold and compensating by reducing video transmittal rate then reducing audio playback rate

• US 5,822,537 A
• Filed: 04/05/1996
• Issued: 10/13/1998
• Est. Priority Date: 02/24/1994
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Katseff, Howard P., Robinson, Bethany Scott
• Primary Examiner(s): DINH, DUNG C
• Application Number:
  US08/628,391
• Time in Patent Office: 921 Days
• Field of Search: 395/200.13, 395/220.19, 395/250, 395/877, 395/878, 395/880, 395/806, 395/807, 395/200.61, 395/200.62, 395/200.63, 395/200.34, 395/200.36, 370/229, 370/231, 370/235, 370/302, 348/439, 348/419, 345/302
• US Class Current: 709/231
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 47/30 in combination with information about buffer occupancy at either end or at transit nodes
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2401 Monitoring of the client buffer
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/654 : Transmission by server directed to the client
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: Y02D 30/50 : in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

Disclosed is a networked multimedia information system which may be utilized to record, store and distribute multimedia presentations together with any supplemental materials that may be referenced during the presentation. The recorded presentation, together with the associated supplemental materials, may be simultaneously presented on a display containing two separate viewing windows. The effects of network congestion are minimized by prefetching audio and video data for storage in audio and video buffers. An adaptive control algorithm compensates for network congestion by dynamically varying the rate at which video frames are retrieved over the network, in response to network traffic conditions. The audio playback speed is reduced if the audio data does not arrive fast enough over the network to maintain the desired size of the audio buffer after the amount of video data transmitted across the network has been reduced to a minimum value.

Claims

1. A method for use by a computing system receiving audio and video data over a network for presentation to a user, said method compensating for congestion on said network which causes delayed arrival of said audio and video data, said video data being transmitted over said network to said computing system at a requested video transmittal rate, said audio data being presented to said user at an audio playback rate, said network congestion compensation method comprising the steps of:

maintaining an audio buffer for storing a predefined amount of said audio data received over said network;

maintaining a video buffer for storing a predefined amount of said video data received over said network;

monitoring said audio and video buffers to determine when said amount of audio or video data in said buffer falls below said predefined amounts;

reducing said requested video transmittal rate if said monitoring step determines that said amount of audio or video data in said buffers has fallen below said predefined amounts; and

reducing said audio playback rate if said amount of audio data in said audio buffer is below said predefined amount of audio data after said step of reducing said requested video transmittal rate.

2. The network congestion compensation method according to claim 1, further including the step of increasing said audio playback rate once said amount of audio data in said audio buffer is above said predefined amount of audio data.

3. The network congestion compensation method according to claim 1, wherein said step of reducing said audio playback rate is not performed unless said step of reducing said requested video transmittal rate has reduced said requested video transmittal rate to a minimum value.

4. The network congestion compensation method according to claim 1, wherein said step of reducing said audio playback rate plays said audio at a reduced speed by dividing said audio data in said audio buffer into a plurality of segments and playing one or more of said segments at least twice.

5. The network congestion compensation method according to claim 1, wherein said step of reducing said audio playback rate plays said audio at a reduced speed by utilizing a pitch extraction process.

6. The network congestion compensation method according to claim 1, wherein said step of reducing said audio playback rate will gradually adjust said audio playback rate to make the transition from one speed to another less noticeable to said user.

7. A computer-readable storage medium comprising encoded computer-readable program instructions for use in conjunction with a programmable computer receiving audio and video data over a network for presentation to a user, which instructions cause the computer to compensate for congestion on the network that results in delayed arrival of the audio and video data, the video data being transmitted over the network at a requested video transmittal rate to a video buffer for storing a predefined amount of the video data and the audio data being presented to the user at an audio playback rate, wherein a predefined amount of the audio data is stored in an audio buffer, the program instructions defining steps to be performed by the programmable computer, the steps comprising:

monitoring the audio and video buffers to determine when the amount of audio or video data in the buffers falls below the predefined amounts;

reducing the requested video transmittal rate if the monitoring step determines that the amount of audio or video data in the buffers has fallen below the predefined amounts; and

reducing the audio playback rate if the amount of audio data in the audio buffer is below the predefined amount of audio data after the step of reducing the requested video transmittal rate.

8. The computer-readable storage medium of claim 7, wherein the steps further comprise increasing the audio playback rate once the amount of audio data in the audio buffer is above the predefined amount of audio data.

9. The computer-readable storage medium of claim 7, wherein the step of reducing the audio playback rate is not performed unless the step of reducing the requested video transmittal rate has reduced the requested video transmittal rate to a minimum value.

10. The computer-readable storage medium of claim 7, wherein the step of reducing the audio playback rate plays the audio at a reduced speed by dividing the audio data in the audio buffer into a plurality of segments and playing one or more of the segments at least twice.

11. The computer-readable storage medium of claim 7, wherein the step of reducing the audio playback rate plays the audio at a reduced speed by utilizing a pitch extraction process.

12. The computer-readable storage medium of claim 7, wherein the step of reducing the audio playback rate will gradually adjust the audio playback rate to make the transition from one speed to another less noticeable to the user.