Method and system for mixing audio streams in a computing system

• US 5,703,794 A
• Filed: 06/20/1995
• Issued: 12/30/1997
• Est. Priority Date: 06/20/1995
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yerrace, Frank Dominic, Dahl, Geoff, Heddle, Robert M.
• Primary Examiner(s): Olms, Douglas W.
• Assistant Examiner(s): HOM, SHICK C
• Application Number:
  US08/492,709
• Time in Patent Office: 924 Days
• Field of Search: 364/514 R, 364/514 C, 364/514 A, 370/62, 370/110.1, 370/260, 370/263, 370/266, 370/267, 370/522, 379/202
• US Class Current: 700/94
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios

Abstract

The disclosed invention constitutes a method and system, including an audio mixer, for selecting audio streams from one or more applications running on a computer system. An application can register its audio streams with the audio mixer either as sound focus audio streams or as priority audio streams. With respect to the sound focus audio streams, the audio mixer determines which of the applications has the sound focus and generates an audio output signal from those sound focus audio streams to drive the computer'"'"'s sound device and thereby to produce an audible sound for the user. Sound focus is determined when an application gains the input focus, opens an audio stream, closes an audio stream, or registers with the audio mixer. With respect to the priority audio streams, those priority audio streams are always selected by the audio mixer and are mixed with the sound focus audio streams from the application having sound focus to generate a composite audio output signal. The audio streams are scaled so that mixing audio streams does not normally exceed the volume limit of the sound device.

Claims

1. A method for generating an audio output signal from one or more audio streams produced by one or more audio sources operating on a general purpose computer system in which each audio source can selectively have input focus, comprising the steps:

a. for each audio source, registering zero or more audio streams as sound focus audio streams;

b. determining which of the audio sources has sound focus based on the input focus and on the registration the audio streams of the audio sources as sound focus audio streams;

c. selecting the sound focus audio streams from the audio source that has the sound focus;

d. generating the audio output signal by mixing the selected sound focus audio streams of the audio source having the sound focus; and

e. converting the audio output signal to an audible sound.

2. The method of claim 1, wherein the method further comprises the step of:

a. controlling the volume of the sound focus audio streams from the audio sources prior to the step of selecting the sound focus audio streams from the audio source that has the sound focus.

3. The method of claim 1 or 2, wherein the method further comprises the steps of:

a. designating zero or more of the audio streams as priority audio streams; and

b. mixing the priority audio streams from the audio sources with the selected sound focus audio streams to generate a composite audio output signal; and

c. converting the composite audio output signal to an audible sound.

4. The method of claim 3, wherein mixing the priority audio streams with the sound focus audio streams to generate the composite audio output signal further includes scaling the priority audio streams and the sound focus audio streams.

5. The method of claim 1, wherein the audio source with the sound focus is determined, when an audio source gains input focus, in accordance with the following logical sequence:

a. If the audio source that gained the input focus has an open sound focus audio stream, that audio source gets the sound focus;

b. If the audio source that gained the input focus has registered with the audio mixer that it gets the sound focus whenever it has the input focus even if it does not have open sound focus audio streams, that audio source gets the sound focus;

c. If the audio source losing the input focus has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, then the audio mixer leaves the sound focus with the audio source currently having the sound focus;

d. Alternatively, if the audio source losing the input focus has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer determines from an ordered list of audio sources which audio source gets the sound focus by setting a variable X equal to the identity of the audio source with the current sound focus;

e. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

f. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

g. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source;

h. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (e) above.

6. The method of claim 1, wherein the audio source with the sound focus is determined, when an audio source opens a sound focus audio input stream, in accordance with the following logical sequence:

a. If the audio source that opened the audio stream does not have the input focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

b. If the audio source that opened the sound focus audio stream already has the sound focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

c. Alternatively, the audio mixer sets the sound focus to the audio source that opened the sound focus audio stream.

7. The method of claim 1, wherein the audio source with the sound focus is determined, when an audio source closes a sound focus audio stream, in accordance with the following logical sequence:

a. If the audio source closing the sound focus audio stream does not have the current sound focus, the audio mixer does nothing, and the sound focus remains with the audio source currently having the sound focus;

b. Alternatively, if the audio source that closed the sound focus audio stream has additional open sound focus audio streams, the audio mixer does nothing, and the audio focus remains with the audio source that currently has the sound focus;

c. Alternatively, the audio mixer determines from an ordered list of audio sources which audio source gets the sound focus by setting a variable X equal to the identity of the audio source that closed the sound focus audio stream;

d. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

e. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

f. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

g. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (d) above.

8. The method of claim 1, wherein the audio source with the sound focus is determined, when an audio source registers with the audio mixer, in accordance with the following logical sequence:

a. If the registration for the registering audio source specify that it gets the sound focus only when it also has the input focus and if the audio source currently has the sound focus and if the audio source does not have the input focus, then the audio mixer goes to step (b);

b. The audio mixer determines from an ordered list of audio sources with the input focus which audio source gets the sound focus by setting a variable X equal to the identity of the audio source with the current sound focus;

c. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list, the sound focus is set to null, and no audio source has the sound focus;

d. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

e. If the audio source gaining the input focus has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

f. Alternatively, if the audio source gaining the input focus has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (c) above;

g. Alternatively, if the registration for the registering audio source specifies that it gets the sound focus whenever it has the input focus, and the registering audio source currently has the input focus, and the registering audio source currently does not have the sound focus, then the audio mixer will set the sound focus to the registering audio source;

h. If the registering audio source is changing the registration of its audio streams from priority to sound focus, the audio mixer goes to step (i) below;

i. If the registering audio source does not have the input focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

j. If the registering audio source already has the sound focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

k. Alternatively, the audio mixer sets the sound focus to the registering audio source;

l. If the registering audio source changing the registration of its audio streams from sound focus to priority, the audio mixer goes to step (m) below;

m. If the registering audio source does not have the current sound focus, the audio mixer does nothing, and the sound focus remains with the audio source currently having the sound focus;

n. Alternatively, if the registering audio source has additional open sound focus audio streams, the audio mixer does nothing, and the audio focus remains with the audio source that currently has the sound focus;

o. Alternatively, the method goes to step (p) below and examines the ordered list of audio source recently having the input focus to determine which, if any, of the audio sources on that ordered list should have the sound focus;

p. The determination of step (o) begins by setting a variable X equal to the identity of the registering audio source;

q. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

r. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

s. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

t. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (q) above;

u. Alternatively, the audio mixer does nothing to change the sound focus.

9. An audio mixer for mixing audio streams in a general purpose computing system, in which audio source can selectively have input focus, comprising:

a. an audio output;

b. audio inputs for receiving audio streams from a plurality of audio sources wherein zero or more audio streams are registered as sound focus audio streams;

c. control inputs for receiving control signals including an input focus control signal and an audio source registration signal;

d. a selection component connected to the audio inputs and the control inputs for receiving the sound focus audio streams and control signals for;

i. determining which audio source possesses sound focus based on the input focus control signal and the audio source registration signal; and

ii. connecting the sound focus audio streams from audio source possessing sound focus to the audio output to generate an audio output signal on the audio output of the audio mixer.

10. The audio mixer of claim 9, wherein the audio mixer further comprises:

a. a volume control interposed between the audio inputs and the selection component.

11. The audio mixer of claim 9 or 10, wherein the audio mixer further comprises:

a. a priority mixer for selecting audio streams from audio sources that have registered with the audio mixer as priority audio streams; and

b. an output mixer for mixing the priority audio streams with the sound focus audio streams to generate a composite audio output signal at the audio output.

12. The audio mixer of claim 11, wherein the output mixer includes a scaler for scaling the priority audio streams and the audio streams from the sound focus audio streams.

13. The audio mixer of claim 9, wherein the selection component determines the audio source with the sound focus as a result of an audio source gaining input focus in accordance with the following logical sequence:

a. If the audio source that gained the input focus has an open sound focus audio stream, that audio source gets the sound focus;

b. If the audio source that gained the input focus has registered with the audio mixer that it gets the sound focus whenever it has the input focus even if it does not have open sound focus audio streams, that audio source gets the sound focus;

c. If the audio source losing the input focus has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, then the audio mixer leaves the sound focus with the audio source currently having the sound focus;

d. Alternatively, if the audio source losing the input focus has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer determines from a ordered list of audio sources which audio source gets the sound focus by setting a variable X equal to the identity of the audio source with the current sound focus;

e. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

f. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

g. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

h. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (e) above.

14. The audio mixer of claim 9, wherein the selection component determines the audio source with the sound focus as a result of an audio source opening a sound focus audio stream in accordance with the following logical sequence:

a. If the audio source that opened the audio stream does not have the input focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

b. If the audio source that opened the sound focus audio stream already has the sound focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

c. Alternatively, the audio mixer sets the sound focus to the audio source that opened the sound focus audio stream.

15. The audio mixer of claim 9, wherein the selection component determines the audio source with the sound focus as a result of an audio source closing a sound focus audio stream in accordance with the following logical sequence:

a. If the audio source closing the sound focus audio stream does not have the current sound focus, the audio mixer does nothing, and the sound focus remains with the audio source currently having the sound focus;

b. Alternatively, if the audio source that closed the sound focus audio stream has additional open sound focus audio streams, the audio mixer does nothing, and the audio focus remains with the audio source that currently has the sound focus;

c. Alternatively, the audio mixer determines from an ordered list of audio sources which audio source gets the sound focus by setting a variable X equal to the identity of the audio source that closed the sound focus audio stream;

d. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

e. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

f. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

g. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (d) above.

16. The audio mixer of claim 9, wherein the selection component determines the audio source with the sound focus as a result of an audio source registering with the audio mixer in accordance with the following logical sequence:

a. If the registration for the registering audio source specify that it gets the sound focus only when it also has the input focus and if the audio source currently has the sound focus and if the audio source does not have the input focus, then the audio mixer goes to step (b);

b. The audio mixer determines from an ordered list of audio sources with the input focus which audio source gets the sound focus by setting a variable X equal to the identity of the audio source with the current sound focus;

c. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list, the sound focus is set to null, and no audio source has the sound focus;

d. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

e. If the audio source gaining the input focus has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

f. Alternatively, if the audio source gaining the input focus has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (c) above;

g. Alternatively, if the registration for the registering audio source specifies that it gets the sound focus whenever it has the input focus, and the registering audio source currently has the input focus, and the registering audio source currently does not have the sound focus, then the audio mixer will set the sound focus to the registering audio source;

h. If the registering audio source is changing the registration of its audio streams from priority to sound focus, the audio mixer goes to step (i) below;

i. if the registering audio source does not have the input focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

j. If the registering audio source already has the sound focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

k. Alternatively, the audio mixer sets the sound focus to the registering audio source;

l. If the registering audio source changing the registration of its audio streams from sound focus to priority, the audio mixer goes to step (m) below;

m. If the registering audio source does not have the current sound focus, the audio mixer does nothing, and the sound focus remains with the audio source currently having the sound focus;

n. Alternatively, if the registering audio source has additional open sound focus audio streams, the audio mixer does nothing, and the audio focus remains with the audio source that currently has the sound focus;

o. Alternatively, the method goes to step (p) below and examines the ordered list of audio source recently having the input focus to determine which, if any, of the audio sources on that ordered list should have the sound focus;

p. The determination of step (o) begins by setting a variable X equal to the identity of the registering audio source;

q. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

r. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

s. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

t. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (q) above;

u. Alternatively, the audio mixer does nothing to change the sound focus.

17. A computer-readable medium having computer-executable instructions for generating an audio output signal from one or more audio streams produced by one or more audio sources operating on a general purpose computer system in which each audio source can selectively have input focus, comprising the steps:

a. for each audio source, registering zero or more audio streams as sound focus audio streams;

b. determining which of the audio sources has sound focus based on the input focus and on the registration the audio streams of the audio sources as sound focus audio streams;

c. selecting the sound focus audio streams from the audio source that has the sound focus;

d. generating the audio output signal by mixing the selected sound focus audio streams of the audio source having the sound focus; and

e. converting the audio output signal to an audible sound.

18. The computer-readable medium of claim 17, having the further computer-executable instructions for performing the step of:

a. controlling the volume of the sound focus audio streams from the audio sources prior to the step of selecting the sound focus audio streams from the audio source that has the sound focus.

19. The computer-readable medium of claim 17 or 18, having the further computer-executable instructions for performing the steps of:

a. designating zero or more of the audio streams as priority audio streams; and

b. mixing the priority audio streams from the audio sources with the selected sound focus audio streams to generate a composite audio output signal; and

c. converting the composite audio output signal to an audible sound.

20. The computer-readable medium of claim 17, having the further computer-executable instructions for mixing the priority audio streams with the sound focus audio streams to generate the composite audio output signal and scaling the priority audio streams and the sound focus audio streams.

21. The computer-readable medium of claim 17, having the further computer-executable instructions for identifying the audio source with the sound focus, when an audio source gains input focus, in accordance with the following logical sequence:

a. If the audio source that gained the input focus has an open sound focus audio stream, that audio source gets the sound focus;

b. If the audio source that gained the input focus has registered with the audio mixer that it gets the sound focus whenever it has the input focus even if it does not have open sound focus audio streams, that audio source gets the sound focus;

c. If the audio source losing the input focus has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, then the audio mixer leaves the sound focus with the audio source currently having the sound focus;

d. Alternatively, if the audio source losing the input focus has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer determines from a ordered list of audio sources which audio source gets the sound focus by setting a variable X equal to the identity of the audio source with the current sound focus;

e. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

f. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

g. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

h. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (e) above.

22. The computer-readable medium of claim 17, having the further computer-executable instructions for identifying the audio source with the sound focus, when an audio source opens a sound focus audio input stream, in accordance with the following logical sequence:

a. If the audio source that opened the audio stream does not have the input focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

b. If the audio source that opened the sound focus audio stream already has the sound focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

c. Alternatively, the audio mixer sets the sound focus to the audio source that opened the sound focus audio stream.

23. The computer-readable medium of claim 17, having the further computer-executable instructions for identifying the audio source with the sound focus, when an audio source closes a sound focus audio stream, in accordance with the following logical sequence:

a. If the audio source closing the sound focus audio stream does not have the current sound focus, the audio mixer does nothing, and the sound focus remains with the audio source currently having the sound focus;

b. Alternatively, if the audio source that closed the sound focus audio stream has additional open sound focus audio streams, the audio mixer does nothing, and the audio focus remains with the audio source that currently has the sound focus;

c. Alternatively, the audio mixer determines from an ordered list of audio sources which audio source gets the sound focus by setting a variable X equal to the identity of the audio source that closed the sound focus audio stream;

d. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

e. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

f. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

g. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (d) above.

24. The computer-readable medium of claim 17, having the further computer-executable instructions for identifying the audio source with the sound focus, when an audio source registers with the audio mixer, in accordance with the following logical sequence:

a. If the registration for the registering audio source specify that it gets the sound focus only when it also has the input focus and if the audio source currently has the sound focus and if the audio source does not have the input focus, then the audio mixer goes to step (b);

b. The audio mixer determines from an ordered list of audio sources with the input focus which audio source gets the sound focus by setting a variable X equal to the identity of the audio source with the current sound focus;

c. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list, the sound focus is set to null, and no audio source has the sound focus;

d. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

e. If the audio source gaining the input focus has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

f. Alternatively, if the audio source gaining the input focus has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (c) above;

g. Alternatively, if the registration for the registering audio source specifies that it gets the sound focus whenever it has the input focus, and the registering audio source currently has the input focus, and the registering audio source currently does not have the sound focus, then the audio mixer will set the sound focus to the registering audio source;

h. If the registering audio source is changing the registration of its audio streams from priority to sound focus, the audio mixer goes to step (i) below;

i. If the registering audio source does not have the input focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

j. If the registering audio source already has the sound focus, the audio mixer does nothing, and the sound focus remains with the audio source that currently has the sound focus;

k. Alternatively, the audio mixer sets the sound focus to the registering audio source;

l. If the registering audio source changing the registration of its audio streams from sound focus to priority, the audio mixer goes to step (m) below;

m. If the registering audio source does not have the current sound focus, the audio mixer does nothing, and the sound focus remains with the audio source currently having the sound focus;

n. Alternatively, if the registering audio source has additional open sound focus audio streams, the audio mixer does nothing, and the audio focus remains with the audio source that currently has the sound focus;

o. Alternatively, the method goes to step (p) below and examines the ordered list of audio source recently having the input focus to determine which, if any, of the audio sources on that ordered list should have the sound focus;

p. The determination of step (o) begins by setting a variable X equal to the identity of the registering audio source;

q. If no audio source on the ordered list qualifies for sound focus because audio source X is the last audio source on the ordered list of audio sources, the sound focus is set to null, and no audio source has the sound focus;

r. Alternatively, replace audio source X with the audio source prior to X on the ordered list of audio sources;

s. If the audio source X has not specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the audio mixer sets the sound focus to audio source X;

t. Alternatively, if the audio source X has specified in its registration with the audio mixer that it never gets the sound focus if it does not have the input focus, the method of the audio mixer returns to step (q) above;

u. Alternatively, the audio mixer does nothing to change the sound focus.