System and method for cross-fading between audio streams

• US 7,302,396 B1
• Filed: 04/27/1999
• Issued: 11/27/2007
• Est. Priority Date: 04/27/1999
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: RealNetworks, Inc.
• Inventors: Cooke, Kenneth E.
• Primary Examiner(s): Armstrong; Angela A.
• Application Number:
  US09/300,798
• Time in Patent Office: 3,136 Days
• Field of Search: 704/500, 704/503, 704/270, 704/201, 375/316, 375/355, 714/747
• US Class Current: 704/500
• CPC Class Codes: G11B 27/038 Cross-faders therefor

Abstract

A system and method of the present invention cross-fade a first transmitted audio stream to a second transmitted audio stream, wherein both first and second audio streams represent the same original audio signal, but at different quality levels. A client computer receives timestamped packets of compressed encoded audio data from the first audio stream, decodes that data and resamples it to a highest sampling rate supported by playback equipment such as a sound card. A server computer responds to a change in available bandwidth, by transmitting timestamped packets of the second audio stream which correspond to a playback time earlier than that of the final transmitted packet of the first audio stream. The client computer buffers in a first buffer the decoded resampled samples from the final packets of the first audio stream, which represent a playback time period t₁. The client computer then buffers in a second buffer decoded resampled samples from the initial packets of the second audio stream representing a playback time period t₂. A cross-fade overlap window is defined by a time period t₃ over which t₁ and t₂ overlap. A cross-fader cross-fades sample pairs drawn from both buffers, each pair corresponding to a playback time in the cross-fade overlap window. A cross-fade table holds a predetermined number of values decreasing from 1 to 0, which values approximate a cross-fade curve. The cross-fader applies a weight value to each sample pair, the weight value calculated by applying linear interpolation across adjacent values in the cross-fade table, by multiplying a sample from the first audio stream by the weight value, and by multiplying a time-corresponding sample from the second audio stream by one minus the weight value. The resulting contributions from both samples are combined and sent to audio reproduction equipment.

Claims

1. A method comprising:

receiving, within a first stream via a network communication link, first audio data generated by sampling of a common audio signal of an audio signal at a first sampling rate for a first time period;

receiving thereafter, based at least in part upon a change in a bandwidth capability of the network communication link, second audio data within a second stream generated by sampling of said audio source at a second sampling rate different than said first sampling rate for a second time period, the first and second audio data corresponding to different, but overlapping, portions of the common audio signal;

generating a plurality of samples by normalizing a portion of said first audio data to said second sampling rate, said portion of said first audio data being normalized at least in part corresponding to the overlapping portion of said common audio signal sampled at said first sampling rate;

cross-fading and combining pairs of samples, each pair substantially corresponding to a playback time, one sample of each pair being selected from one of said plurality of samples, the other sample of each pair being selected from a portion of said second audio data, said portion of said second audio data being selected at least in part corresponding to said overlapping portion of said common audio signal sampled at said second sampling rate; and

rendering said cross-faded and combined samples.

2. The method as defined in claim 1, wherein said cross-fading and combining includes applying a first cross-fade weight to a first sample of each of said pair of samples to obtain a first contribution, applying a second cross-fade weight to a second sample of each of said pair of samples to obtain a second contribution, and combining said first and second contributions to generate a cross-fade sample.

3. The method as recited in claim 1 further comprising storing the cross-faded combined pairs of sample on a hard disk drive.

4. A method comprising:

receiving in a receive buffer via a network communication link first audio data of a first data stream, the first audio data representing a time period t1 and sampled at a first target sampling rate of an original audio signal;

decoding said first audio data and re-sampling the decoded first audio data to generate first audio samples as the first audio data are received, initially into an audio output buffer, and in response to an indication of a change in a data capacity of the network communication link, into an old stream buffer instead;

receiving thereafter in said receive buffer, second audio data from the second data stream representing a time period t₂ of said original audio signal and sampled at a second target sampling rate different from said first target sampling rate, said time period t1 and t₂ overlapping by a time period t₃ in said original audio signal;

decoding said second audio data and re-sampling the decoded second audio data to generate second audio samples as the second audio data are received, initially into a new stream buffer, for at least a portion of the time period t₃;

cross-fading each sample pair comprising corresponding sample pairs corresponding to a time within said at least a portion of the time period t₃ from said old and new stream buffers, by applying a first cross-fade weight to a first sample of said sample pair to obtain a first contribution, a second cross-fade weight to a second sample of said sample pair to obtain a second contribution, and by combining said first and second contributions, to successively generate a plurality of cross-faded combined samples; and

outputting successively the generated cross-faded combined samples into the audio output buffer.

5. The method as described in claim 4, wherein said first stream represents said original audio signal at a first sampling rate and said second stream represents said original audio signal at a second sampling rate.

6. The method as described in claim 5, wherein each applied first cross-fade weight represents a value between 1 and 0, and the sum of said first cross-fade weight and said second cross-fade weight applied to each said sample pair is 1.

7. The method as described in claim 6, wherein each applied first cross-fade weight represents a point along a curve defined by one-half cycle of the cosine function offset and scaled to begin at a value of one and end at a value of zero.

8. A system comprising:

a receive buffer to successively receive and store a first and a second stream of an audio signal transmitted via a network, the second stream being received based at least in part upon a change in conditions of a network, the first and second streams respectively correspond to first and second portions of the audio signal, and the first and second portions audio signal overlap;

a first and a second decoder coupled with the receive buffer to respectively decode the first and second received streams;

a sample-rate converter coupled with the first and second decoders to resample the decoded first and second received streams adapted to generate a first and second plurality of digital samples respectively;

an old stream buffer coupled with the sample-rate converter to receive the first digital samples, after an initial time period, and at substantially a beginning of the receipt of a portion of the first stream corresponding to the overlap of the first and second portions of the audio signal;

a new stream buffer coupled with the sample-rate converter to receive the second digital samples for at least a portion of the second stream corresponding to the overlap of the first and second portions of the audio signal;

a cross-fader coupled with the old and new stream buffers to cross-fade and combine corresponding ones of the first and second digital samples; and

a renderer to render the cross-faded and combined digital samples.

9. The system as described in claim 8, wherein said cross-fader applies cross-fade weights to paired resampled samples from said first and second streams to generate cross-faded samples, each of said pairs of resampled samples substantially corresponding to a playback time.

10. The system as described in claim 9, wherein said cross-fader applies a first cross-fade weight to a first of each pair of said resampled samples and applies a second cross-fade weight to a second of each pair of said resampled samples, said first and second cross-fade weights summing to one.

11. The system as recited in claim 8 further comprising a receiver to receive the first and second streams from a server via the communications link.

12. A method comprising:

receiving, via a network communication link, first audio data within a first stream of an audio signal for a first time period;

receiving thereafter, via the network communication link, second audio data within a second stream of the audio signal for a second time period, said second audio data being received in response to a change in bandwidth capability of the network communication link, and having a common portion of the audio signal that is also a part of the first audio data;

decoding said first and second audio data as they are received;

generating successively pairs of samples of said first and second audio data for at least a portion of the common portion of the audio signal that is a part of the first audio data and a part of the second audio data, each pair substantially corresponding to a playback time, one sample of each pair being selected from said first decoded audio data, said other sample of each pair being selected from said second decoded audio data;

cross-fading and combining successively said successively generated pairs of samples; and

successively rendering the cross-faded and combined samples.

13. The method as recited in claim 12, wherein said first audio data'"'"'s compression rate is different than said second audio data'"'"'s compression rate.

14. The method as recited in claim 12, further comprising playing as an audio stream a portion of said first audio data, said cross-faded pairs of samples and said portion of said second audio data.

15. The method as recited in claim 12, wherein said first audio source is pre-recorded music.

16. The method as recited in claim 12 wherein the first and second audio data are received from a server via the communications link.

17. A computer readable media having a set of instructions adapted to enable a processing system to practice a method including:

receiving via a communication link first audio data within a first audio stream of an audio signal for a first time period;

receiving thereafter, a second audio data within a second stream of the audio signal, via the communication link, for a second time period, in response to a change in bandwidth capability of the network communication link;

decoding said first and second audio data, the first and second data, both having a common portion of the audio signal; and

generating pairs of samples of said first and second audio data for at least a portion of the common portion of the audio signal, each pair substantially corresponding to a playback time, one sample of each pair being selected from a portion of said first decoded audio data, said other sample of each pair being selected from a portion of said second decoded audio data;

cross-fading to combine said pairs of samples; and

rendering the cross-faded and combined samples.

18. The computer readable media as recited in claim 17, wherein said first audio data is generated via encoding of the common audio signal at a first sampling rate and said second audio data is generated via encoding of the common audio signal at a second sampling rate, wherein said first sampling rate is different than said second sampling rate.

19. The computer readable media as recited in claim 17, further comprising playing as an audio stream a portion of said first audio data, said cross-faded combined pairs of samples, and a portion of said second audio data.

20. A method comprising

streaming first audio data to a remote rendering client device for a first period of time, the first audio data having been generated by sampling a first portion of an audio signal at a first sampling rate;

detecting a change in operating condition; and

streaming second audio data to the remote rendering client device for a second period of time, the second audio data having been generated by sampling a second portion of the audio signal at a second sampling rate, the first and second portions of the audio signal having a common portion of the audio signal.

21. The method of claim 20, wherein the method further comprises sampling the first portion of the audio signal at the first sampling rate to generate the first audio data, and the streaming of the first audio data is performed as the first audio data are generated.

22. The method of claim 20, wherein the change in operating condition comprises a change in bandwidth of a communication link to the remote rendering client device.

23. The method of claim 20, wherein the method further comprises pre-notifying the remote rendering client device of the change in the generating sampling rate of the audio data being streamed to the remote rendering client device.

24. An apparatus comprising

streaming means for streaming audio data generated from sampling an audio signal at a sampling rate to a remote rendering client device;

and

control means for

first controlling the streaming means to stream first audio data to the remote rendering client device for a first period of time, the first audio data having been generated by sampling a first portion of the audio signal at a first sampling rate, and

then on detecting a change in operating condition, controlling the streaming means to stream thereafter, second audio data to the remote rendering client device for a second period of time, the second audio data having been generated by sampling a second portion of the audio signal at a second sampling rate, the first and second portions of the audio signal having a common portion of the audio signal.

25. The apparatus of claim 24, wherein the apparatus further comprises sampling means for sampling the first portion of the audio signal at the first sampling rate to generate the first audio data, and sampling the second portion of the audio signal at the second sampling rate to generate the second audio data.

26. The apparatus of claim 24, wherein the apparatus further comprises detection means to detect the change in operating condition, the change being a change in bandwidth of a communication link to the remote rendering client device.

27. The apparatus of claim 24, wherein the apparatus further comprises pre-notification means for pre-notifying the remote rendering client device of the change in the generating sampling rate of the audio data being streamed to the remote rendering client device.