Filter and method for adaptively modifying the bit rate of synchronized video and audio streams to meet packet-switched network bandwidth constraints

• US 6,747,991 B1
• Filed: 04/26/2000
• Issued: 06/08/2004
• Est. Priority Date: 04/26/2000
• Status: Expired due to Fees

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Gross, Thomas, Hemy, Michael, Steenkiste, Peter
• Primary Examiner(s): Kizou, Hassan
• Assistant Examiner(s): Levitan, Dmitry
• Application Number:
  US09/558,518
• Time in Patent Office: 1,504 Days
• Field of Search: 370/229, 370/235, 370/235.1, 370/236, 370/464, 370/465, 370/468, 370/469, 370/473, 709/231, 709/232, 709/233, 709/235, 709/236
• US Class Current: 370/468
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16

Abstract

A filter for adaptively modifying a bit rate of a synchronized video and audio stream transmitted over a packet-switched network, including a layer detection module, an adaptive frame removal module in communication with the layer detection module, and a network packetizing and pacing module in communication with the adaptive frame removal module.

Claims

1. A filter for adaptively modifying a bit rate of a synchronized video and audio stream transmitted over a packet-switched network to a receiver, comprising:

a layer detection module for detecting encoded video frames in the stream;

an adaptive frame removal module in communication with the layer detection module for removing certain of the encoded video frames based on a bandwidth condition of the network; and

a network packetizing and pacing module in communication with the adaptive frame removal module for packetizing the modified stream into network packets and for pacing transmission of the packets of the modified stream over the network, wherein one of the packets includes information regarding the frame removal rate of the adaptive frame removal module.

2. The filter of claim 1, wherein the layer detection module is for detecting encoded video frames in the stream by detecting a structure of all layers of the stream without demultiplexing and decoding of the stream.

3. The filter of claim 1, wherein the network packetizing and placing module is for distributing network packets over an interval created by removal of the encoded video frames.

4. The filter of claim 1, wherein the layer detection module is for detecting encoded video frames by analyzing bits in the stream to identify patterns indicative of a current position in the stream relative to other layers in the stream.

5. The filter of claim 1, wherein the encoded video frames are encoded according to a hierarchy.

6. The filter of claim 5, wherein the adaptive frame removal module is for adaptively removing encoded video frames having an increasingly higher order in the hierarchy based on a bandwidth condition of the network.

7. The filter of claim 1, wherein the layer detection module is for detecting I-frames, P-frames, and B-frames in an MPEG system stream.

8. The filter of claim 7, where the adaptive frame removal module is for progressively removing, in order, B-frames, P-frames, and I-frames, based on a bandwidth condition of the network.

9. A filter for adaptively modifying a bit rate of a synchronized video and audio stream transmitted over a packet-switched network, comprising:

means for detecting encoded video frames of the stream;

means for adaptively removing certain of the encoded video frames based on a bandwidth condition of the network; and

means for packetizing the stream into network packets, wherein one of the packets includes information regarding the frame removal rate of the means for adaptively removing certain of the encoded video frames.

10. The filter of claim 9, further comprising means for pacing transmission of the network packets over the network.

11. The filter of claim 9, wherein the means for pacing includes means for distributing network packets over an interval created by removal of the encoded video frames.

12. The filter of claim 9, wherein the means for detecting includes means for detecting I-frames, P-frames, and B-frames in an MPEG system stream.

13. The filter of claim 9, wherein the means for detecting includes means for detecting encoded video frames in the stream by detecting a structure of all layers of the stream without demultiplexing and decoding of the stream.

14. The filter of claim 9, wherein the encoded video frames are encoded according to a hierarchy, and wherein the means for adaptively removing includes means for progressively removing encoded video frames of an increasingly higher order in the hierarchy.

15. The filter of claim 9, wherein the bandwidth condition is communicated to the filter over the network.

16. A filter for a packet-switched network, comprising:

a layer detection module for detecting encoded video frames in a synchronized video and audio stream;

an adaptive frame removal module in communication with the decoding module for removing certain of the encoded video frames in the stream based on a bandwidth condition of the network; and

a network packetizing and pacing module in communication with the adaptive frame removal module for packetizing the stream into network packets and for pacing transmission of the packets over the network, wherein one of the packets includes information regarding the frame removal rate of the adaptive frame removal module.

17. The filter of claim 16, wherein the bandwidth condition is communicated to the adaptive frame removal module over the network.

18. The filter of claim 16, wherein the layer detection module is for detecting encoded video frames in the stream by detecting a structure of all layers of the stream without demultiplexing and decoding of the stream.

19. The filter of claim 16, wherein the layer detection module is for detecting encoded video frames by analyzing bits in the stream to identify patterns indicative of a current position in the stream relative to other layers in the stream.

20. The filter of claim 16, wherein the network packetizing and pacing module is for pacing transmission of the network packets by distributing the network packets over an interval created by removal of the encoded video frames.

21. The filter of claim 16, wherein the encoded video frames are encoded according to a hierarchy, and wherein the adaptive frame removal module is for progressively removing encoded video frames of an increasingly higher order in the hierarchy.

22. A network for transmitting a synchronized video and audio stream, comprising:

a video server; and

a filter in communication with the video server, wherein the filter includes;

a layer detection module for detecting encoded video frames in a synchronized video and audio stream transmitted by the video server;

an adaptive frame removal module in communication with the decoding module for removing certain of the encoded video frames in the stream based on a bandwidth condition of the network; and

a network packetizing and pacing module in communication with the adaptive frame removal module for packetizing the stream into network packets and pacing transmission of the packets over the network, wherein one of the packets includes information regarding the frame removal rate of the adaptive frame removal module.

23. The network of claim 22, wherein the video server is for transmitting the stream to a receiver via the filter, and wherein the receiver communicates the bandwidth condition of the network to the filter.

24. The network of claim 22, wherein the filter is co-located with the server.

25. A network, comprising:

a video server for transmitting a synchronized video and audio stream to a receiver;

a first filter in communication with the video server; and

a second filter in communication with the first filter, and wherein the first and second filters each include;

a layer detection module for detecting encoded video frames in the stream;

an adaptive frame removal module in communication with the decoding module for removing certain of the encoded video frames in the stream based on a bandwidth condition of the network; and

a network packetizing and pacing module in communication with the adaptive frame removal module for packetizing the stream into network packets and pacing transmission of the packets over the network; and

wherein the receiver communicates a first bandwidth condition of the network to the second filter and the second filter communicates a second bandwidth condition of the network to the first filter.

26. A network, comprising:

a video server for transmitting a synchronized video and audio stream;

means for detecting encoded video frames in the stream;

means for adaptively removing certain of the encoded video frames based on a bandwidth condition of the network; and

means for packetizing the modified stream into network packets and pacing transmission of the packets over the network, wherein one of the packets includes information regarding the frame removal rate of the means for adaptively removing certain of the encoded video frames.

27. A computer-readable medium having stored thereon instructions, which when executed by a processor, cause the processor to:

detect encoded video frames in a synchronized video and audio stream;

remove certain of the encoded video frames based on a frame removal rate of a packet-switched network over which the stream is to be transmitted;

packetize the modified stream into network packets; and

transmit the packets to a receiver, wherein one of the packets includes information regarding the frame removal rate.

28. The computer-readable medium of claim 27, having further stored thereon instructions, which when executed by the processor, cause the processor to transmit the network packets over the network, wherein the packets are distributed over an interval created by removal of the encoded video frames.

29. The computer-readable medium of claim 27, having further stored thereon instructions, which when executed by the processor, cause the processor to detect encoded video frames in the stream by detecting a structure of all layers of the stream without demultiplexing and decoding of the stream.

30. The computer-readable medium of claim 27, having further stored thereon instructions, which when executed by the processor, cause the processor to detect encoded video frames by analyzing bits in the stream to identify patterns indicative of a current position in the stream relative to other layers in the stream.

31. The computer-readable medium of claim 27, having further stored thereon instructions, which when executed by the processor, cause the processor to detect I-frames, P-frames, and B-frames in an MPEG system stream.

32. A method for adaptively modifying a bit rate of a synchronized video and audio stream in real-time, comprising:

detecting encoded video frames in the stream;

removing certain of the encoded video frames from the stream based on a frame removal rate of a packet-switched network over which the stream is to be transmitted;

packetizing the modified stream into network packets; and

transmitting the packets to a receiver, wherein one of the packets includes information regarding the frame removal rate.

33. The method of claim 32, further comprising pacing transmission of the network packets.

34. The method of claim 33, wherein pacing includes distributing the network packets over an interval created by removal of the encoded video frames.

35. The method of claim 32, wherein detecting includes detecting encoded video frames in the stream by detecting a structure of all layers of the stream without demultiplexing and decoding of the stream.

36. The method of claim 32, wherein detecting includes detecting encoded video frames in the stream by analyzing bits in the stream to identify patterns indicative of a current position in the stream relative to other layers in the stream.

37. The method of claim 32, wherein the video frames are encoded according to a hierarchy, and wherein removing includes progressively removing certain encoded video frames of an increasingly higher order in the hierarchy.

38. The method of claim 32, further comprising receiving information regarding the bandwidth condition of the network from a receiver of the packetized stream.

39. The method of claim 38, further comprising communicating information to the receiver regarding a frame removal rate.

40. The method of claim 32, wherein detecting includes detecting I-frames, P-frames, and B-frames of an MPEG system stream.