Video game system using pre-encoded digital audio mixing

• US 8,270,439 B2
• Filed: 01/05/2007
• Issued: 09/18/2012
• Est. Priority Date: 07/08/2005
• Status: Expired due to Fees

• Current Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Original Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Herr, Stefan, Sigmund, Ulrich
• Primary Examiner(s): Yao, Kwang B
• Assistant Examiner(s): LIU, JUNG
• Application Number:
  US11/620,593
• Publication Number:
  US 20070105631A1
• Time in Patent Office: 2,083 Days
• Field of Search: 463/42, 370/395.21, 370/395.42, 370/395.43, 370/474, 370537-541
• US Class Current: 370/537
• CPC Class Codes: G07F 17/32 for games, toys, sports, or amusements
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
• CPC Class Codes: H04S 2420/03 Application of parametric coding in stereophonic audio systems
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

A method and related system of encoding audio is disclosed. In the method, data representing a plurality of independent audio signals is accessed. The data representing each respective audio signal comprises a sequence of source frames. Each frame in the sequence of sources frames comprises a plurality of audio data copies. Each audio data copy has an associated quality level that is a member of a predefined range of quality levels, ranging from a highest quality level to a lowest quality level. The plurality of source frame sequences is merged into a sequence of target frames that comprise a plurality of target channels. Merging corresponding source frames into a respective target frame includes selecting a quality level and assigning the audio data copy at the selected quality level of each corresponding source frame to at least one respective target channel.

Claims

1. A method of encoding audio, comprising:

at a computer system including one or more processors and memory;

storing data representing a plurality of independent audio signals, the data representing each respective audio signal comprising a respective sequence of source frames of audio data;

wherein each source frame in the respective sequence of sources frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data of the source frame having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level;

receiving a user command;

in response to the user command, selecting a first audio signal; and

merging the sequences of source frames for the first audio signal and a second audio signal into a sequence of target frames, wherein;

the target frames comprise a plurality of target channels in the target frames;

the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence;

the second audio signal comprises a continuous source of non-silent audio data; and

the merging includes, for a respective target frame;

selecting a quality level;

selecting a first source frame for the first audio signal at the selected quality level;

selecting a second source frame for the second audio signal at the selected quality level; and

assigning the first source frame and the second source frame to separate respective target channels in the respective target frame.

2. The method of claim 1, wherein a respective copy of the audio data of the first source frame comprises one or more fractional mantissa groups, wherein each fractional mantissa group is full.

10. The method of claim 1, wherein:

the command corresponds to an action by a user playing a video game; and

the first audio signal corresponds to a sound effect to be played in response to the command; and

the second audio signal corresponds to background audio for the video game.

11. The method of claim 1, wherein the quality level is selected to maintain a constant bit rate for the sequence of target frames.

3. A method of encoding audio, comprising:

at a computer system including one or more processors and memory;

in advance of execution of an application;

receiving audio data from a plurality of respective independent sources including a first audio signal and a second audio signal, wherein the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence and the second audio signal comprises a continuous source of non-silent audio data; and

encoding the audio data from each respective independent source into a respective sequence of source frames, to produce a plurality of sequences of source frames of audio data, wherein each source frame in each respective sequence of source frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data in the source frame having a distinct associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level; and

during execution of the application;

receiving a command corresponding to an action in the application; and

in response to receiving the command, merging the plurality of sequences of source frames into a sequence of target frames, wherein the target frames comprise a plurality of independent target channels in the target frames and each sequence of source frames is uniquely assigned to one or more target channels of the plurality of independent target channels in the target frames.

20. The method of claim 3, wherein encoding the audio data comprises:

for a frame in a respective sequence of sources frames, generating a plurality of copies of the frame, each copy having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level.

21. The method of claim 20, wherein encoding the audio data further comprises:

for each copy, performing a bit allocation process; and

if the bit allocation process creates one or more incomplete fractional mantissa groups, modifying results of the bit allocation process to either fill or empty each incomplete fractional mantissa group.

22. The method of claim 21, wherein for a respective copy, if each incomplete fractional mantissa group cannot be either filled or emptied, the respective copy is not included in the frame.

4. A system for encoding audio, comprising:

memory;

one or more processors;

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including instructions for;

storing data representing a plurality of independent audio signals, the data representing each respective audio signal comprising a respective sequence of source frames of audio data;

wherein each source frame in the respective sequence of sources frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data of the source frame having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level;

receiving a user command;

in response to the user command, selecting a first audio signal; and

merging the sequences of source frames for the first audio signal and a second audio signal into a sequence of target frames, wherein;

the target frames comprise a plurality of target channels in the target frames;

the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence;

the second audio signal comprises a continuous source of non-silent audio data; and

the instructions for merging include, for a respective target frame;

instructions for selecting a quality level;

instructions for selecting a first source frame for the first audio signal at the selected quality level;

instructions for selecting a second source frame for the second audio signal at the selected quality level; and

instructions for assigning the first source frame and the second source frame to separate respective target channels in the respective target frame.

12. The system of claim 4, wherein a respective copy of the audio data of the first source frame comprises one or more fractional mantissa groups, wherein each fractional mantissa group is full.

13. The system of claim 4, wherein:

the command corresponds to an action by a user playing a video game; and

the first audio signal corresponds to a sound effect to be played in response to the command; and

the second audio signal corresponds to background audio for the video game.

14. The system of claim 4, wherein the quality level is selected to maintain a constant bit rate for the sequence of target frames.

5. A system for encoding audio, comprising:

memory;

one or more processors;

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including instructions for;

in advance of execution of an application;

receiving audio data from a plurality of respective independent sources including a first audio signal and a second audio signal, wherein the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence and the second audio signal comprises a continuous source of non-silent audio data;

encoding the audio data from each respective independent source into a respective sequence of source frames, to produce a plurality of sequences of source frames of audio data, wherein each source frame in each respective sequence of source frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data in the source frame having a distinct associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level; and

during execution of the application;

receiving a command corresponding to an action in the application; and

in response to receiving the command, merging the plurality of sequences of source frames into a sequence of target frames, wherein the target frames comprise a plurality of independent target channels in the target frames and each sequence of source frames is uniquely assigned to one or more target channels of the plurality of independent target channels in the target frames.

18. The system of claim 5, wherein:

the application is a video game application; and

the command corresponds to an action by a user playing the video game.

19. The system of claim 5, wherein at least one of the sequences of source frames corresponds to a sound effect in the video game.

26. The system of claim 5, wherein the audio data from a respective independent source is a pulse-code-modulated bitstream.

27. The system of claim 26, wherein the pulse-code-modulated bitstream is a WAV, W64, AU, or AIFF file.

28. The system of claim 5, wherein the instructions for encoding the audio data comprise instructions for:

for a frame in a respective sequence of sources frames, generating a plurality of copies of the frame, each copy having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level.

29. The system of claim 28, wherein the instructions for encoding the audio data further comprise instructions for:

for each copy, performing a bit allocation process; and

if the bit allocation process creates one or more incomplete fractional mantissa groups, modifying results of the bit allocation process to either fill or empty each incomplete fractional mantissa group.

30. The system of claim 29, wherein the instructions for performing the bit allocation process comprise instructions for modifying results of the bit allocation process by performing delta bit allocation.

31. The system of claim 30, wherein the delta bit allocation is determined by a backtracking algorithm.

32. The system of claim 29, wherein for a respective copy, if each incomplete fractional mantissa group cannot be either filled or emptied, the respective copy is not included in the frame.

33. The system of claim 28, wherein the associated quality levels correspond to specified signal-to-noise ratios.

34. The system of claim 29, wherein the instructions for merging the plurality of sequences of source frames into the sequence of target frames comprise instructions for:

selecting a signal-to-noise ratio for a source frame; and

merging the copy having the selected signal-to-noise ratio into a target frame in the sequence of target frames.

35. The system of claim 34, wherein the instructions for selecting the signal-to-noise ratio comprise instructions for maintaining a constant bit rate for the sequence of target frames.

36. The system of claim 5, wherein the target frames are in the AC-3 format.

6. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computer system, cause the computer system to:

store data representing a plurality of independent audio signals, the data representing each respective audio signal comprising a respective sequence of source frames of audio data;

wherein each source frame in the respective sequence of sources frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data of the source frame having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level;

receive a user command; and

in response to the user command, select a first audio signal; and

merge the sequences of source frames for the first audio signal and a second audio signal into a sequence of target frames, wherein;

the target frames comprise a plurality of target channels in the target frames;

the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence;

the second audio signal comprises a continuous source of non-silent audio data; and

the instructions for merging include, for a respective target frame;

instructions for selecting a quality level;

instructions for selecting a first source frame for the first audio signal at the selected quality level;

instructions for selecting a second source frame for the second audio signal at the selected quality level; and

instructions for assigning the first source frame and the second source frame to separate respective target channels in the respective target frame.

15. The non-transitory computer readable storage medium of claim 6, wherein a respective copy of the audio data of the first source frame comprises one or more fractional mantissa groups, wherein each fractional mantissa group is full.

16. The non-transitory computer readable storage medium of claim 6, wherein:

the command corresponds to an action by a user playing a video game; and

the first audio signal corresponds to a sound effect to be played in response to the command; and

the second audio signal corresponds to background audio for the video game.

17. The non-transitory computer readable storage medium of claim 6, wherein the quality level is selected to maintain a constant bit rate for the sequence of target frames.

7. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computer system, cause the computer system to:

in advance of execution of an application;

receive audio data from a plurality of respective independent sources including a first audio signal and a second audio signal, wherein the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence and the second audio signal comprises a continuous source of non-silent audio data;

encode the audio data from each respective independent source into a respective sequence of source frames, to produce a plurality of sequences of source frames of audio data, wherein each source frame in each respective sequence of source frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data in the source frame having a distinct associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level; and

during execution of the application;

receive a command corresponding to an action in the application; and

in response to receiving the command, merge the plurality of sequences of source frames into a sequence of target frames, wherein the target frames comprise a plurality of independent target channels in the target frames and each sequence of source frames is uniquely assigned to one or more target channels of the plurality of independent target channels in the target frames.

23. The non-transitory computer readable storage medium of claim 7, wherein the instructions to encode the audio data comprise instructions to:

for a frame in a respective sequence of sources frames, generate a plurality of copies of the frame, each copy having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level.

24. The non-transitory computer readable storage medium of claim 23, wherein the instructions to encode the audio data further comprise instructions to:

for each copy, perform a bit allocation process; and

if the bit allocation process creates one or more incomplete fractional mantissa groups, modify results of the bit allocation process to either fill or empty each incomplete fractional mantissa group.

25. The non-transitory computer readable storage medium of claim 24, wherein for a respective copy, if each incomplete fractional mantissa group cannot be either filled or emptied, the respective copy is not included in the frame.

8. A system for encoding audio, comprising:

means for storing data representing a plurality of independent audio signals, the data representing each respective audio signal comprising a respective sequence of source frames of audio data;

wherein each source frame in the respective sequence of sources frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data of the source frame having an associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level;

means for receiving a user command;

means, responsive to the user command, for selecting a first audio signal; and

means for merging the sequences of source frames for the first audio signal and a second audio signal into a sequence of target frames, wherein;

the target frames comprise a plurality of target channels in the target frames

the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence;

the second audio signal comprises a continuous source of non-silent audio data; and

the merging includes, for a respective target frame;

selecting a quality level;

selecting a first source frame for the first audio signal at the selected quality level;

selecting a second source frame for the second audio signal at the selected quality level; and

assigning the first source frame and the second source frame to separate respective target channels in the respective target frame.

9. A system for encoding audio, comprising:

in advance of execution of an application;

means for receiving audio data from a plurality of respective independent sources including a first audio signal and a second audio signal, wherein the first audio signal comprises an episodic source of non-silent audio data that includes sequences of audio data representing silence and the second audio signal comprises a continuous source of non-silent audio data;

means for encoding the audio data from each respective independent source into a respective sequence of source frames, to produce a plurality of sequences of source frames of audio data, wherein each source frame in each respective sequence of source frames comprises a plurality of copies of the audio data of the source frame, each copy of the audio data in the source frame having a distinct associated quality level, the quality level of each copy being a member of a predefined range of quality levels that range from a highest quality level to a lowest quality level; and

during execution of the application;

means for receiving a command corresponding to an action in the application; and

means, responsive to receiving the command, for merging the plurality of sequences of source frames into a sequence of target frames, wherein the target frames comprise a plurality of independent target channels in the target frames and each sequence of source frames is uniquely assigned to one or more target channels of the plurality of independent target channels in the target frames.