Video Game System with Mixing of Independent Pre-Encoded Digital Audio Bitstreams

• US 20110028215A1
• Filed: 07/31/2009
• Published: 02/03/2011
• Est. Priority Date: 07/31/2009
• Status: Active Grant

• Current Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Original Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Sigmund, Ulrich, Herr, Stefan
• Granted Patent: US 8,194,862 B2
• Time in Patent Office: Days
• US Class Current: 463/35
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/0204 using subband decomposition
• CPC Class Codes: G10L 19/173 Transcoding, i.e. converting between two coded representations avoiding cascaded coding-decoding

Abstract

A computer-implemented method of encoding audio includes accessing a plurality of independent audio source streams, each of which includes a sequence of source frames. Respective source frames of each sequence include respective pluralities of pulse-code modulated audio samples. Each of the plurality of independent audio source streams is separately encoded to generate a plurality of independent encoded streams, each of which corresponds to a respective independent audio source stream. The encoding includes, for respective source frames, converting respective pluralities of pulse-code modulated audio samples to respective pluralities of floating-point frequency samples that are divided into a plurality of frequency bands. An instruction to mix the plurality of independent encoded streams is received; in response, respective floating-point frequency samples of the independent encoded streams are combined. An output bitstream is generated that includes the combined respective floating-point frequency samples.

Claims

1. A computer-implemented method of encoding audio, comprising:

prior to execution of a video game by a computer system;

accessing a plurality of independent audio source streams, each source stream comprising a sequence of source frames, respective source frames of each sequence comprising respective pluralities of pulse-code modulated audio samples; and

separately encoding each of the plurality of independent audio source streams to generate a plurality of independent encoded streams, each independent encoded stream corresponding to a respective independent audio source stream, the encoding comprising, for respective source frames, converting respective pluralities of pulse-code modulated audio samples to respective pluralities of floating-point frequency samples, the respective pluralities of floating-point frequency samples being divided into a plurality of frequency bands; and

during execution of the video game by the computer system;

receiving an instruction to mix the plurality of independent encoded streams;

in response to the instruction to mix the plurality of independent encoded streams, combining respective floating-point frequency samples of the independent encoded streams; and

generating an output bitstream comprising the combined respective floating-point frequency samples.

2. The method of claim 1, further comprising transmitting the output bitstream to a client device for decoding and playback.

3. The method of claim 1, wherein converting the respective pluralities of pulse-code modulated audio samples to respective pluralities of floating-point frequency samples comprises performing Pseudo-Quadrature Mirror Filtering (PQMF) of the respective pluralities of pulse-code modulated audio samples.

4. The method of claim 1, wherein the encoding further comprises applying a fixed psycho-acoustic model (PAM) to successive respective pluralities of floating-point frequency samples.

5. The method of claim 4, wherein the fixed PAM comprises a predefined table having a plurality of entries, wherein each entry corresponds to a signal-to-mask ratio (SMR) for a respective frequency band of the plurality of frequency bands.

6. The method of claim 1, wherein a respective encoded stream generated from a respective source stream comprises a sequence of encoded frames that correspond to respective source frames in the respective source stream,

the encoding further comprising, for each encoded frame in the sequence of encoded frames of the respective encoded stream:

applying a fixed PAM to the respective plurality of floating-point frequency samples; and

for each respective frequency band in the plurality of frequency bands, calculating a single respective scale factor to scale mantissas of the respective plurality of floating-point frequency sample in the respective frequency band of the respective frame.

7. The method of claim 6, wherein:

successive encoded frames of the respective encoded stream each comprise three blocks, each block having twelve floating-point frequency samples per frequency band; and

for each of the successive encoded frames, the single respective scale factor in each respective frequency band scales each of the twelve floating-point frequency samples in each of the three blocks.

8. The method of claim 7, wherein the encoding comprises selecting a transmission pattern to indicate, for each respective frequency band of each of the successive encoded frames, that the single scale factor scales the mantissas in the three blocks.

9. The method of claim 6, further comprising:

encoding respective frames of an independent audio source stream of the plurality of independent audio source streams in accordance with the MPEG-1 Layer II standard;

receiving an instruction to play audio associated only with the independent audio source stream; and

in response to the instruction to play audio associated only with the independent audio source stream, generating an output bitstream comprising the respective frames of the independent audio source stream as encoded in accordance with the MPEG-1 Layer II standard.

10. The method of claim 1, wherein the encoding comprises:

calculating a first scale factor to scale floating-point frequency samples in a respective frequency band of a plurality of frequency bands of a respective frame of a first independent encoded stream of the plurality of independent encoded streams, and

calculating a second scale factor to scale floating-point frequency samples in a respective frequency band of a plurality of frequency bands of a respective frame of a second independent encoded stream of the plurality of independent encoded streams.

11. The method of claim 10, wherein the encoding further comprises:

for the first independent encoded bitstream;

scaling the floating-point frequency samples of the respective frequency band of the respective frame by the first scale factor, and

storing the floating-point frequency samples of the respective frequency band of the respective frame, as scaled by the first scale factor; and

for the second independent encoded bitstream;

scaling the floating-point frequency samples of the respective frequency band of the respective frame by the second scale factor, and

storing the floating-point frequency samples of the respective frequency band of the respective frame, as scaled by the second scale factor.

12. The method of claim 10, wherein the combining comprises:

calculating an adjusted scale factor to scale the floating-point frequency samples of the respective frequency band and respective frame of the first independent encoded bitstream and the floating-point frequency samples of the respective frequency band and respective frame of the second independent encoded bitstream.

13. The method of claim 12, wherein the combining further comprises:

scaling the floating-point frequency samples of the respective frequency band and respective frame of the first independent encoded bitstream by a first ratio of the first scale factor to the adjusted scale factor;

scaling the floating-point frequency samples of the respective frequency band and respective frame of the second independent encoded bitstream by a second ratio of the second scale factor to the adjusted scale factor; and

adding respective floating-point frequency samples of the first independent encoded bitstream, as scaled by the first ratio, to respective floating-point frequency samples of the second independent encoded bitstream, as scaled by the second ratio.

14. The method of claim 13, wherein the combining further comprises:

determining that a combined floating-point frequency sample, generated by adding respective floating-point frequency samples of the first and second encoded bitstreams, exceeds a predefined limit; and

in response to the determination, assigning the combined floating-point frequency sample to equal the predefined limit.

15. The method of claim 13, wherein respective mantissas of combined floating-point frequency samples, generated by adding respective floating-point frequency samples of the first and second encoded bitstreams, are stored in respective single bytes.

16. The method of claim 12, wherein the adjusted scale factor is calculated as a first function of a difference between the first and second scale factors.

17. The method of claim 12, wherein the first, second, and adjusted scale factors are encoded as indices referencing scale factor values stored in a table, the indices each being represented with more than six bits.

18. The method of claim 16, wherein the first function comprises addition of an offset to the first or second scale factor, the offset being a monotonic second function of the magnitude of the difference between the first and second scale factors.

19. The method of claim 16, wherein:

the first, second, and adjusted scale factors are encoded as indices referencing scale factor values stored in a table; and

the difference between the first and second scale factors is calculated by subtracting the smaller of the indices corresponding to the first and second scale factors from the larger of the indices corresponding to the first and second scale factors.

20. The method of claim 19, wherein the first function comprises subtraction of an offset from the lower of the indices encoding the first or second scale factor, the offset being a monotonic second function of the magnitude of the difference between the indices encoding the first and second scale factors.

21. The method of claim 19, wherein each of the indices encoding the first, second, and adjusted scale factors is stored in a single byte.

22. The method of claim 12, wherein the first, second, and adjusted scale factors are encoded as indices referencing scale factor values stored in a table, the combining further comprising:

scaling the floating-point frequency samples of the respective frequency band and respective frame of the first independent encoded bitstream by a scale factor value having an index corresponding to a difference between indices encoding the adjusted and first scale factors;

scaling the floating-point frequency samples of the respective frequency band and respective frame of the second independent encoded bitstream by a scale factor value having an index corresponding to a difference between indices encoding the adjusted and second scale factors; and

adding respective floating-point frequency samples, as scaled, of the first and second independent encoded bitstreams.

23. The method of claim 22, wherein the first, second, and adjusted scale factors are encoded as indices referencing scale factor values stored in a table, the indices each being represented with more than six bits, the combining further comprising:

dividing the index encoding the adjusted scale factor to produce a divided scale factor index being represented by six bits; and

writing the divided scale factor index to the encoded bitstream.

24. The method of claim 22, wherein the combining further comprises:

determining that a combined floating-point frequency sample, generated by adding respective floating-point frequency samples of the first and second encoded bitstreams, exceeds a predefined limit; and

in response to the determination, assigning the combined floating-point frequency sample to equal the predefined limit.

25. The method of claim 1, wherein the combining comprises calculating respective sums of respective floating-point frequency samples and dividing the respective sums by a constant value.

26. The method of claim 22, wherein the constant value equals 2 or √

{square root over (2)}.

27. The method of claim 1, wherein:

first and second independent audio source streams of the plurality of independent audio source streams and corresponding first and second independent encoded streams of the plurality of independent encoded streams each comprise a left channel and a right channel; and

the combining comprises;

mixing the left channels of the first and second independent encoded streams to generate a left channel of the output bitstream; and

mixing the right channels of first and second independent encoded streams to generate a right channel of the output bitstream.

28. The method of claim 1, wherein:

a first independent audio source stream and corresponding first independent encoded stream of the plurality of independent encoded streams each comprise a left channel and a right channel;

a second independent encoded stream of the plurality of independent encoded streams and corresponding second independent encoded stream of the plurality of independent encoded streams each comprise a mono channel; and

the combining comprises;

mixing the left channel of the first independent encoded stream with the mono channel of the second independent encoded stream to generate a left channel of the output bitstream; and

mixing the right channel of the first independent encoded stream with the mono channel of the second independent encoded stream to generate a right channel of the output bitstream.

29. The method of claim 1, wherein:

first and second independent encoded streams each comprise first and second stereo channels for frequency bands below a predefined limit and a mono channel for frequency bands above the predefined limit; and

the combining comprises separately mixing the first stereo channels, second stereo channels, and mono channels of the first and second independent encoded streams.

30. The method of claim 1, wherein:

a first independent audio source stream of the plurality of independent audio source streams comprises a continuous source of non-silent audio data; and

a second independent audio source stream of the plurality of independent audio source streams comprises an episodic source of non-silent audio data.

31. The method of claim 1, wherein:

a first independent audio source stream of the plurality of independent audio source streams comprises a first episodic source of non-silent audio data; and

a second independent audio source stream of the plurality of independent audio source streams comprises a second episodic source of non-silent audio data.

32. A computer-implemented method of encoding audio, comprising:

prior to execution of a video game by a computer system;

storing a plurality of independent encoded audio streams in a computer-readable medium of the computer system, each independent encoded stream comprising a sequence of frames, respective frames of each sequence comprising respective pluralities of floating-point frequency samples, the respective pluralities of floating-point frequency samples being divided into a plurality of frequency bands; and

during execution of the video game by the computer system;

receiving an instruction to mix the plurality of independent encoded streams;

in response to the instruction to mix the plurality of independent encoded streams, accessing the plurality of independent encoded audio streams stored in the computer-readable medium and combining the respective floating-point frequency samples of the independent encoded streams; and

generating an output bitstream comprising the combined respective floating-point frequency samples.

33. A system for encoding audio, comprising:

memory;

one or more processors;

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including;

instructions for storing a plurality of independent encoded audio streams in the memory prior to execution of a video game by the one or more processors, each independent encoded stream comprising a sequence of frames, respective frames of each sequence comprising respective pluralities of floating-point frequency samples, the respective pluralities of floating-point frequency samples being divided into a plurality of frequency bands;

instructions for accessing the plurality of independent encoded audio streams stored in the memory and combining the respective floating-point frequency samples of the independent encoded streams, in response to an instruction to mix the plurality of independent encoded streams during execution of the video game by the one or more processors; and

instructions for generating an output bitstream comprising the combined respective floating-point frequency samples.

34. A computer readable storage medium for use in encoding audio, the computer readable storage medium storing one or more programs configured to be executed by a computer system, the one or more programs comprising:

instructions for accessing a plurality of independent encoded audio streams stored in a memory of the computer system prior to execution of a video game by the computer system, in response to an instruction to mix the plurality of independent encoded streams during execution of the video game by the computer system, each independent encoded stream comprising a sequence of frames, respective frames of each sequence comprising respective pluralities of floating-point frequency samples, the respective pluralities of floating-point frequency samples being divided into a plurality of frequency bands;

instructions for combining the respective floating-point frequency samples of the independent encoded streams, in response to the instruction to mix the plurality of independent encoded streams; and

instructions for generating an output bitstream comprising the combined respective floating-point frequency samples.