SPATIALIZED AUDIO IN A THREE-DIMENSIONAL COMPUTER-BASED SCENE

• US 20020013813A1
• Filed: 04/30/1997
• Published: 01/31/2002
• Est. Priority Date: 04/30/1997
• Status: Active Grant

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Inventors: MATSUOKA, SHINYA
• Granted Patent: US 7,379,961 B2
• Time in Patent Office: Days
• US Class Current: 709/204
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04M 3/567 Multimedia conference systems

Abstract

A system and method for enabling an audio conference server (ACS) to provide an application program with multi-point weight controllable audio conferencing. The ACS manages a plurality of audio conferences, receives audio data from a plurality of audio clients, mixes the audio data to provide distance-based attenuation according to decay characteristics for each sound, and delivers the mixed audio data to a plurality of audio clients. Audio clients include set-top box (STB) audio clients and point source audio (PSA) audio clients. The ACS mixes the audio data by identifying a decay factor. Pre-defined decay factors include an audio big decay factor, an audio small decay factor, an audio medium decay factor, and a constant decay factor. One can also develop a customized decay factor. A weighted value for a source audio client based on the identified decay factor and the distance between the source audio client and a target audio client is determined. A mix table is generated using the weighted values for each source/target audio client pair. Then an actual mix value for each target audio client is calculated using the mix table. The present invention also includes means for refining the actual mix value.

Claims

1. An audio conference server (ACS) for enabling an application program to provide multi-point, weight controllable audio conferencing, comprising:

means for managing at least one audio conference, said at least one audio conference comprising a plurality of audio clients;

means for receiving audio data from said plurality of audio clients;

means for mixing said audio data to provide spatialized audio to said plurality of audio clients in said at least one audio conference, wherein said mixing means results in mixed audio data; and

means for delivering said mixed audio data to said plurality of audio clients in said at least one audio conference.

2. The ACS of claim 1, wherein said mixing means includes means for providing distance-based attenuation according to sound decay characteristics.

3. The ACS of claim 1, further comprising means for checking the status of a registered owner of said at least one audio conference to determine whether said at least one audio conference still exists.

4. The ACS of claim 3, wherein said checking means includes a resource audit service, said resource audit service operable when said at least one audio conference is generated by a first application and is being used by a second application.

5. The ACS of claim 1, wherein said plurality of audio clients includes set-top box (STB) audio clients and point source audio (PSA) audio clients.

6. The ACS of claim 1, wherein said managing means comprises an ACS shell to allow a user to interactively interface with said ACS, said ACS shell including:

means for providing program access to high level methods for creating and managing a proxy audio conference;

means for providing program access to methods for creating and managing a plurality of PSA audio clients; and

means for providing program access to low level methods for creating and managing said at least one audio conference.

7. The ACS of claim 2, wherein said means for providing distance-based attenuation according to sound decay characteristics comprises:

means for identifying a decay factor from one of a plurality of pre-defined decay factors and a customized decay factor for each of said plurality of audio clients, said plurality of pre-defined decay factors including

an audio big decay factor,

an audio small decay factor,

an audio medium decay factor, and

a constant decay factor;

means for determining distances between a target audio client and a plurality of source audio clients;

means for determining a plurality of weighted values for each of said source audio clients based on said identified decay factor and said distance between each of said source audio clients and said target audio client, wherein each of said weighted values corresponds to a source/target audio client pair;

means for generating a mix table for each of said source/target audio client pairs;

means for calculating an actual mix for said target audio clients; and

means for refining said actual mix for said target audio clients.

8. The ACS of claim 7, wherein said refining means comprises:

a gain control function to avoid transmitting excess energy audio data;

a fade in/fade out function to avoid the delivery of said audio data in a step-wise manner to a speaker output;

a floating point operation elimination function to avoid the performance of floating point multiplication;

a mixing adaption function to adapt the actual mix calculation for said target audio client to available CPU resources;

a mixing cut-off function to select the nearest talking audio clients for the actual mix; and

a stream audio function to prepare stream audio for playing ambient background music or using an audio source forwarded from another conference.

9. A method for enabling an audio conference server to provide an application program with multi-point, weight controllable audio conferencing, comprising the steps of:

(1) managing at least one audio conference, said at least one audio conference comprising a plurality of audio clients;

(2) receiving audio data from said plurality of audio clients;

(3) mixing said audio data to provide spatialized audio to said plurality of audio clients in said at least one audio conference, wherein said mixing means results in mixed audio data; and

(4) delivering said mixed audio data to said plurality of audio clients in said at least one audio conference.

10. The method of claim 9, wherein said mixing step includes providing distance-based attenuation according to sound decay characteristics.

11. The method of claim 9, further comprising the step of checking the status of a registered owner of said at least one audio conference to determine whether said at least one audio conference still exists.

12. The method of claim 11, wherein said checking step includes a resource audit service, said resource audit service operable when said at least one audio conference is generated by a first application and is being used by a second application.

13. The method of claim 9, wherein said plurality of audio clients includes set-top box (STB) audio clients and point source audio (PSA) audio clients.

14. The method of claim 9, wherein step (1) comprises the step of providing program access to high level methods for creating and managing a proxy audio conference using an ACS shell.

15. The method of claim 9, wherein step (1) comprises the step of providing program access to methods for creating and managing said point source audio using an ACS shell.

16. The method of claim 9, wherein step (1) comprises the step of providing program access to low level methods for creating and managing said at least one audio conference using an ACS shell.

17. The method of claim 10, wherein said step for providing distance-based attenuation according to sound decay characteristics comprises the steps of:

identifying a decay factor from one of a plurality of pre-defined decay factors and a customized decay factor for each of said plurality of audio clients, said plurality of pre-defined decay factors including

an audio big decay factor,

an audio small decay factor,

an audio medium decay factor, and

a constant decay factor;

determining distances between a target audio client and a plurality of source audio clients;

determining a plurality of weighted values for each of said source audio clients based on said identified decay factor and said distance between each of said source audio client and said target audio client, wherein each of said weighted values corresponds to a source/target audio client pair;

generating a mix table for each of said source/target audio client pairs;

calculating an actual mix for said target audio clients using said mix table; and

refining said actual mix for said target audio clients, wherein said refining step is used to avoid transmitting excess energy audio data, avoid the delivery of said audio data in a step-wise manner to a speaker output, avoid the performance of floating point multiplication, adapt the actual mix calculation for said target audio client to available CPU resources, select the nearest talking audio clients for the actual mix, and prepare stream audio for playing ambient background music or using an audio source forwarded from another conference.

18. A computer program product comprising a computer useable medium having computer program logic recorded thereon for enabling an audio conference server (ACS) to provide an application program with multi-point, weight controllable audio conferencing, said computer program logic comprising:

means for enabling the computer to manage at least one audio conference, said at least one audio conference comprising a plurality of audio clients;

means for enabling the computer to receive audio data from said plurality of audio clients;

means for enabling the computer to mix said audio data to provide spatialized audio to said plurality of audio clients in said at least one audio conferences, wherein said mixing means results in mixed audio data; and

means for enabling the computer to deliver said mixed audio data to said plurality of audio clients in said at least one audio conference.

19. The computer program product of claim 18, wherein said means for enabling the computer to mix said audio data to provide spatialized audio to said plurality of audio clients in said at least one audio conference includes means for enabling the computer to provide distance-based attenuation according to sound decay characteristics.

20. The computer program product of claim 18, further comprising means for enabling the computer to check the status of a registered owner of said at least one audio conference to determine whether said at least one audio conference still exists.

21. The computer program product of claim 20, wherein said means for enabling the computer to check the status of a registered owner of said at least one audio conference includes a resource audit service, said resource audit service operable when said at least one audio conference is generated by a first application is being used by a second application.

22. The computer program product of claim 18, wherein said plurality of audio clients includes set-top box (STB) audio clients and point source audio (PSA) audio clients.

23. The computer program product of claim 18, wherein said means for enabling the computer to manage at least one audio conference comprises means for enabling the computer to provide an ACS shell to allow a user to interactively interface with said ACS, said ACS shell including:

means for enabling the computer to provide program access to high level methods for creating and managing a proxy audio conference;

means for enabling the computer to provide program access to methods for creating and managing a plurality of point source audio (PSA) audio clients; and

means for enabling the computer to provide program access to low level methods for creating and managing said at least one audio conference.

24. The computer program product of claim 19, wherein said means for enabling the computer to provide distance-based attenuation according to sound decay characteristics comprises:

means for enabling the computer to identify a decay factor from one of a plurality of predefined decay factors and a customized decay factor for each of said plurality of audio clients, said plurality of predefined decay factors including

an audio big decay factor,

an audio small decay factor,

an audio medium decay factor, and

a constant decay factor;

means for enabling the computer to determine distances between a target audio client and a plurality of source audio clients;

means for enabling the computer to determine a plurality of weighted values for each of said source audio clients based on said identified decay factor and said distance between said source audio client and said target audio client, wherein each of said weighted values corresponds to a source/target audio client pair;

means for enabling the computer to generate a mix table for each of said source/target audio client pairs;

means for enabling the computer to calculate an actual mix for said source audio clients; and

means for enabling the computer to refine said actual mix for said source audio clients.

25. The computer program product of claim 24, wherein said means for enabling the computer to refine said actual mix for said source audio clients comprises:

means for enabling the computer to provide a gain control function to avoid transmitting excess energy audio data;

means for enabling the computer to provide a fade in/fade out function to avoid the delivery of said audio data in a step-wise manner to a speaker output;

means for enabling the computer to provide a floating point operation elimination function to avoid the performance of floating point multiplication;

means for enabling the computer to provide a mixing adaption function to adapt the actual mix calculation for said target audio client to available CPU resources;

means for enabling the computer to provide a mixing cut-off function to select the nearest talking audio clients for the actual mix; and

means for enabling the computer to provide a stream audio function to prepare stream audio for playing ambient background music or using an audio source forwarded from another conference.