Receiver-driven layered error correction multicast over heterogeneous packet networks

• US 6,532,562 B1
• Filed: 05/21/1999
• Issued: 03/11/2003
• Est. Priority Date: 05/21/1999
• Status: Expired due to Term

• Current Assignee: Rovi Technologies Corporation (Adeia Inc.)
• Original Assignee: Microsoft Corporation
• Inventors: Chou, Philip A., Mehrotra, Sanjeev, Wang, Albert S.
• Primary Examiner(s): Ton, David
• Application Number:
  US09/316,869
• Time in Patent Office: 1,390 Days
• Field of Search: 370/216, 370/229, 370/432, 710/57, 710/130, 714/752, 714/820, 714/786, 714/748, 714/766, 714/746
• US Class Current: 714/776
• CPC Class Codes: H04L 1/0002 by adapting the transmission rate
• CPC Class Codes: H04L 1/0059 Convolutional codes
• CPC Class Codes: H04L 1/06 using space diversity
• CPC Class Codes: H04L 1/08 by repeating transmission, e.g. Verdan system H04L1/1858 and H04L1/189 take precedence

Abstract

“Correction of errors and losses occurring during a receiver-driven layered multicast (RLM) of real-time media over a heterogeneous packet network such as the Internet is accomplished by augmenting RLM with one or more layers of error correction information. Each receiver separately optimizes the quality of received audio and video information by subscribing to at least one error correction layer. Ideally, each source layer in a RLM would have one or more associated multicasted error correction data streams (i.e., layers). Each error correction layer contains information that can be used to replace lost packets from the associated source layer. More than one error correction layer is proposed as some of the error correction packets contained in the data stream needed to replace the packets lost in the associated source stream may themselves be lost in transmission.”

Claims

1. A computer-implemented error correction process for use in a receiver-driven layered multicast of real-time media over a heterogeneous packet network to a plurality of receivers, wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information, said error correction process comprising using a computer to perform the following step:

multicasting at least one stream of packetized error correction information for at least one of the streams of packetized source data, wherein each stream of packetized error correction information is capable of being used by a receiver to assist in the recovery of packets of source data lost during transmission.

2. The process of claim 1 wherein the step of multicasting streams of packetized error correction information comprises the steps of:

associating each stream of packetized error correction information with a one of the multiple streams of packetized source data; and

incorporating in each error correction stream information necessary to assist a receiver in recovering said packets of lost source data belonging to the particular stream of packetized source data associated with the error correction stream.

3. The process of claim 2, wherein there are multiple streams of error correction information associated with each stream of source data, and wherein the incorporating step comprises the step of incorporating error correction information in each error correction stream associated with the same source data stream which makes the streams redundant, said redundant streams making it possible for a receiver to employ more than one error correction stream and obtain a desired amount of the error correction information needed to replace packets in an associated source data stream that were lost during the transmission even if some of the error correction stream packets themselves are lost in transmission.

4. The process of claim 3, wherein the step of multicasting streams of packetized error correction information comprises the step of incorporating an identifier into each packet in each of the streams of packetized error correction information wherein a different identifier is employed for each of the error correction information streams, thereby allowing each receiver to select the number of error correction information streams to be received by specifying that only packets having particular identifiers be routed over the network to the receiver.

5. The process of claim 4, further comprising the step of a receiver improving the quality of the received broadcast by first selecting a desired number of source data streams while leaving enough bandwidth available to also select a number of error correction information streams for one or more of the source streams that will compensate, at least partially, for an inherent packet loss rate associated with the receivers connection to the network, and then selecting said number of error correction information streams.

6. A computer-implemented process for multicasting real-time media over a heterogeneous packet network to a plurality of receivers, comprising using a computer to perform the following steps:

forming a series of source data streams from a media signal, said source data streams comprising a base layer representing the media signal at a least acceptable level of quality, and at least one enhancement layer which when combined with the base layer improves the level of quality of the media signal derivable therefrom;

for each source data stream, creating at least one error correction information stream from the source layer, each error correction stream comprising a series of data packets that can be used by a receiver to assist in the recovery of data packets of the associated source layer that may be lost during transmission;

respectively multicasting each source layer and error correction layer to a different network address for routing on to a receiver.

7. A computer-implemented process for a receiver to receive a real-time media multicast over a heterogeneous packet network wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information and associated error correction streams, comprising using a computer to perform the following steps:

determining a currently available bandwidth associated with the receiver'"'"'s connection to the network;

determining a current packet loss rate associated with receiving data over the network;

selecting at least one source data stream and at least one associated error correction information stream which are to be received so as to allow the best possible level of quality in a media signal reconstructed from received source data streams, wherein said selecting is based on the currently available bandwidth and the current packet loss rate;

subscribing over the network to the selected source data streams and error correction information streams, and receiving the same;

using the received error correction information stream or streams associated with each received source data stream to assist in the recovery of packets of that source data stream which were lost during transmission to produce a recovered source data stream; and

reconstruct the media signal from the recovered source data stream or streams.

8. The process of claim 7, further comprising the step of continuously monitoring the available bandwidth and packet loss rate and repeating the selection and subscription steps as needed to maintain the best level of quality of the reconstructed media signal as possible.

9. The process of claim 8, wherein the monitoring step comprises the steps of:

rechecking the available bandwidth and packet loss rate periodically during the multicast of the real-time media;

determining whether the available bandwidth has changed more than a prescribed maximum bandwidth change threshold;

determining whether the packet loss rate has changed more than a prescribed maximum packet loss rate change threshold; and

repeating the selection and subscription steps whenever either or both of the prescribed thresholds is exceeded.

10. A computer-implemented error correction system for use in a receiver-driven layered multicast of real-time media over a heterogeneous packet network to a plurality of receivers, wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information, said error correction system comprising:

a broadcaster capable of multicasting at least one stream of packetized error correction information for at least one of the streams of packetized source data; and

at least one receiver, each of which is capable of using a stream of packetized error correction information associated with a particular source data stream to assist in the recovery of packets of that source data lost during transmission.

11. The system of claim 10, wherein the broadcaster comprises:

an encoder and packetizer module capable of producing said multiple stream of packetized source data form a media signal; and

at least one error correction module, wherein each error correction module is capable producing at least one stream of packetized error correction information from a stream of packetized source data and thereafter multicasting each source data stream and its associated error correction streams onto the network.

12. The system of claim 10, wherein each receiver comprises:

a subscription module capable of instructing the network to route particular ones of the source data streams and error correction streams to the receiver;

at least one recovery module capable of receiving a source data stream and its associated error correction stream or streams, and thereafter using each error correction stream received to assist in the recovery of packets identical to those of the source data stream that were lost during transmission and inserting the recovered packets into the source data stream to produce a recovered source data stream; and

a unpacketizer and decoder module for reconstructing a media signal from the recovered source data stream or streams.

13. The system of claim 12, wherein each receiver further comprises a subscription optimization module capable of instructing the subscription module which source data streams and which error correction streams to request from the network, wherein the subscription optimization module instructs the subscription module based on the number of source data streams it is desired to receive so as to obtain a desired quality in the media signal, while leaving enough bandwidth available to also select a number of error correction information streams for one or more of the source streams that will compensate, at least partially, for an inherent packet loss rate associated with the receiver'"'"'s connection to the network.

14. An error correction system for use in a receiver-driven layered multicast of real-time media over a heterogeneous packet network to a plurality of receivers, wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information, said error correction process comprising:

a general purpose computing device;

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

multicast at least one stream of packetized error correction information for at least one of the streams of packetized source data, wherein each stream of packetized error correction information is capable of being used by a receiver to assist in the recovery of packets of source data lost during transmission.

15. The system of claim 14 wherein the program module for multicasting streams of packetized error correction information comprises sub-modules for:

associating each stream of packetized error correction information with a one of the multiple streams of packetized source data; and

incorporating in each error correction stream information necessary to assist a receiver in recovering said packets of lost source data belonging to the particular stream of packetized source data associated with the error correction stream.

16. The system of claim 15, wherein there are multiple streams of error correction information associated with each stream of source data, and wherein the incorporating sub-module comprises a sub-module for incorporating error correction information in each error correction stream associated with the same source data stream which makes the streams redundant, said redundant streams making it possible for a receiver to employ more than one error correction stream and obtain a desired amount of the error correction information needed to replace packets in an associated source data stream that were lost during the transmission even if some of the error correction stream packets themselves are lost in transmission.

17. The system of claim 16, wherein the program module for multicasting streams of packetized error correction information comprises a sub-module for incorporating an identifier into each packet in each of the streams of packetized error correction information wherein a different identifier is employed for each of the error correction information streams, thereby allowing each receiver to select the number of error correction information streams to be received by specifying that only packets having particular identifiers be routed over the network to the receiver.

18. The system of claim 17, further comprising a program module for a receiver improving the quality of the received broadcast by first selecting a desired number of source data streams while leaving enough bandwidth available to also select a number of error correction information streams for one or more of the source streams that will compensate, at least partially, for an inherent packet loss rate associated with the receiver'"'"'s connection to the network, and then selecting said number of error correction information streams.

19. An error correction system for multicasting real-time media over a heterogeneous packet network to a plurality of receivers, comprising:

a general purpose computing device;

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

form a series of source data streams from a media signal, said source data streams comprising a base layer representing the media signal at a least acceptable level of quality, and at least one enhancement layer which when combined with the base layer improves the level of quality of the media signal derivable therefrom.

for each source data stream, create at least one error correction information stream from the source layer, each error correction stream comprising a series of data packets that can be used by a receiver to assist in the recovery of data packets of the associated source layer that may be lost during transmission, and

respectively multicast each source layer and error correction layer to a different network address for routing on to a receiver.

20. An error correction system for allowing a receiver to receive a real-time media multicast over a heterogeneous packet network wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information and associated error correction streams, comprising:

a general purpose computing device;

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

determine a currently available bandwidth associated with the receiver'"'"'s connection to the network,

determine a current packet loss rate associated with receiving data over the network,

select at least one source data stream and at least one associated error correction information stream which are to be received so as to allow the best possible level of quality in a media signal reconstructed from received source data streams, wherein said selecting is based on the currently available bandwidth and the current packet loss rate,

subscribe over the network to the selected source data streams and error correction information streams, and receiving the same,

use the received error correction information stream or streams associated with each received source data stream to assist in the recovery of packets of that source data stream which were lost during transmission to produce a recovered source data stream, and

reconstruct the media signal from the recovered source data stream or streams.

21. The system of claim 20, further comprising a program module for continuously monitoring the available bandwidth and packet loss rate and repeating the selection and subscription steps as needed to maintain the best level of quality of the reconstructed media signal as possible.

22. The system of claim 21, wherein the monitoring sub-module comprises sub-modules for:

rechecking the available bandwidth and packet loss rate periodically during the multicast of the real-time media;

determining whether the available bandwidth has changed more than a prescribed maximum bandwidth change threshold;

determining whether the packet loss rate has changed more than a prescribed maximum packet loss rate change threshold; and

repeating the selection and subscription steps whenever either or both of the prescribed thresholds is exceeded.

23. A computer-readable memory for use in a receiver-driven layered multicast of real-time media over a heterogeneous packet network to a plurality of receivers, wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information, comprising:

a computer-readable storage medium; and

a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,

multicast at least one stream of packetized error correction information for at least one of the streams of packetized source data, wherein each stream of packetized error correction information is capable of being used by a receiver to assist in the recovery of packets of source data lost during transmission.

24. The computer-readable memory of claim 23 wherein the program module for multicasting streams of packetized error correction information comprises sub-modules for:

associating each stream of packetized error correction information with a one of the multiple streams of packetized source data; and

incorporating in each error correction stream information necessary to assist a receiver in recovering said packets of lost source data belonging to the particular stream of packetized source data associated with the error correction stream.

25. The computer-readable memory of claim 24, wherein there are multiple streams of error correction information associated with each stream of source data, and wherein the incorporating sub-module comprises a sub-module for incorporating error correction information in each error correction stream associated with the same source data stream which makes the streams redundant, said redundant streams making it possible for a receiver to employ more than one error correction stream and obtain a desired amount of the error correction information needed to replace packets in an associated source data stream that were lost during the transmission even if some of the error correction stream packets themselves are lost in transmission.

26. The computer-readable memory of claim 25, wherein the program module for multicasting streams of packetized error correction information comprises a sub-module for incorporating an identifier into each packet in each of the streams of packetized error correction information wherein a different identifier is employed for each of the error correction information streams, thereby allowing each receiver to select the number of error correction information streams to be received by specifying that only packets having particular identifiers be routed over the network to the receiver.

27. The computer-readable memory of claim 26, further comprising a program module for a receiver improving the quality of the received broadcast by first selecting a desired number of source data streams while leaving enough bandwidth available to also select a number of error correction information streams for one or more of the source streams that will compensate, at least partially, for an inherent packet loss rate associated with the receiver'"'"'s connection to the network, and then selecting said number of error correction information streams.

28. A computer-readable memory for multicasting real-time media over a heterogeneous packet network to a plurality of receivers, comprising:

a computer-readable storage medium; and

a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,

form a series of source data streams from a media signal, said source data streams comprising a base layer representing the media signal at a least acceptable level of quality, and at least one enhancement layer which when combined with the base layer improves the level of quality of the media signal derivable therefrom.

for each source data stream, create at least one error correction information stream from the source layer, each error correction stream comprising a series of data packets that can be used by a receiver to assist in the recovery of data packets of the associated source layer that may be lost during transmission, and

respectively multicast each source layer and error correction layer to a different network address for routing on to a receiver.

29. A computer-readable memory for allowing a receiver to receive a real-time media multicast over a heterogeneous packet network wherein the real-time media is transmitted over the network in multiple streams of packetized source data forming hierarchical layers of information and associated error correction streams, comprising:

a computer-readable storage medium; and

a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,

determine a currently available bandwidth associated with the receiver'"'"'s connection to the network,

determine a current packet loss rate associated with receiving data over the network,

select at least one source data stream and at least one associated error correction information stream which are to be received so as to allow the best possible level of quality in a media signal reconstructed from received source data streams, wherein said selecting is based on the currently available bandwidth and the current packet loss rate,

subscribe over the network to the selected source data streams and error correction information streams, and receiving the same,

use the received error correction information stream or streams associated with each received source data stream to assist in the recovery of packets of that source data stream which were lost during transmission to produce a recovered source data stream, and

reconstruct the media signal from the recovered source data stream or streams.

30. The computer-readable memory of claim 29, further comprising a program module for continuously monitoring the available bandwidth and packet loss rate and repeating the selection and subscription steps as needed to maintain the best level of quality of the reconstructed media signal as possible.

31. The computer-readable memory of claim 30, wherein the monitoring sub-module comprises sub-modules for:

rechecking the available bandwidth and packet loss rate periodically during the multicast of the real-time media;

determining whether the available bandwidth has changed more than a prescribed maximum bandwidth change threshold;

determining whether the packet loss rate has changed more than a prescribed maximum packet loss rate change threshold; and

repeating the selection and subscription steps whenever either or both of the prescribed thresholds is exceeded.