Using ray tracing for real time audio synthesis

• US 8,139,780 B2
• Filed: 03/20/2007
• Issued: 03/20/2012
• Est. Priority Date: 03/20/2007
• Status: Active Grant

• Current Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Shearer, Robert Allen
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): Paul, Disler
• Application Number:
  US11/688,344
• Publication Number:
  US 20080232602A1
• Time in Patent Office: 1,827 Days
• Field of Search: 381 61- 62, 381/17, 381309-310, 381/1, 381/64, 381/63, 463 31- 32, 463/35, 455/440, 455419-421, 455/418
• US Class Current: 381/64
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

According to embodiments of the invention, a sound engine may determine a final sound at a listener location by emulating sound waves within a three-dimensional scene. The sound engine may emulate sound waves by issuing rays from a location of a sound event and tracing the rays through the three-dimensional scene. The rays may intersect objects within the three-dimensional scene which have sound modification factors. The sound modification factors and other factors (e.g., distance traveled by the ray, angle of intersection with the object, etc.) may be applied to the sound event to determine a final sound which is heard by the listener.

Claims

1. A method of simulating sound in a three-dimensional scene, comprising:

generating a sound event at a first location in the three-dimensional scene;

issuing at least two rays into the three-dimensional scene originating from the first location, the at least two rays emulating a sound wave generated by the sound event,

wherein a direct ray of the at least two rays has a trajectory directed towards a second location within the three-dimensional scene and wherein an indirect ray of the at least two rays has a trajectory different than the trajectory of the direct ray;

performing ray tracing with the at least two rays using a spatial index having branches and nodes corresponding to bounding volumes which partition the three-dimensional scene;

based on the results of ray tracing the at least two rays through the three-dimensional scene, determining a resulting sound at the second location; and

generating an output signal used to generate the resulting sound.

2. The method of claim 1, wherein performing ray tracing comprises taking branches to nodes corresponding to bounding volumes intersected by the at least two rays.

3. The method of claim 2, further comprising:

determining that the direct intersects a primitive within the three-dimensional scene; and

determining a sound modification factor associated with the intersected primitive.

4. The method of claim 3, wherein determining the resulting sound comprises modifying a prerecorded sound based at least in part on the sound modification factor.

5. The method of claim 1, further comprising:

determining that the indirect ray intersects a primitive within the three-dimensional scene;

determining a sound modification factor associated with the primitive,

wherein determining the resulting sound comprises modifying a prerecorded sound based at least in part on the sound modification factor.

6. The method of claim 1, wherein the indirect ray intersects a primitive in the three-dimensional scene further comprising:

issuing a new indirect ray that is reflected from the surface of the primitive with a trajectory towards the second location.

7. The method of claim 1, wherein the direct ray intersects a primitive in the three-dimensional scene further comprising:

issuing a new direct ray at a point different from the point of intersection with the primitive, wherein the new direct ray has a trajectory towards the second location.

8. A non-transitory computer readable medium containing a program which, when executed, performs operations comprising:

generating a sound event at a first location in a three-dimensional scene;

issuing at least two rays into the three-dimensional scene originating from the first location, the at least two rays emulating a sound wave generated by the sound event,

wherein a direct ray of the at least two rays has a trajectory directed towards a second location within the three-dimensional scene and wherein an indirect ray of the at least two rays has a trajectory different than the trajectory of the direct ray;

performing ray tracing with the at least two rays using a spatial index having branches and nodes corresponding to bounding volumes which partition the three-dimensional scene; and

based on the results of ray tracing the at least two rays through the three-dimensional scene, determining a resulting sound at the second location.

9. The non-transitory computer readable medium of claim 8, wherein performing ray tracing comprises taking branches to nodes corresponding to bounding volumes intersected by the at least two rays.

10. The non-transitory computer readable medium of claim 9, wherein the operations further comprise:

determining that the direct ray intersects a primitive within the three-dimensional scene;

determining a sound modification factor associated with the intersected primitive; and

wherein determining the resulting sound comprises modifying a prerecorded sound based at least in part on the sound modification factor.

11. The non-transitory computer readable medium of claim 8, wherein the operations further comprise:

determining that the indirect ray intersects a primitive within the three-dimensional scene;

determining a sound modification factor associated with the primitive

wherein determining the resulting sound comprises, modifying a prerecorded sound based at least in part on the sound modification factor.

12. The method of claim 8, wherein the indirect ray intersects a primitive in the three-dimensional scene further comprising:

issuing a new indirect ray that is reflected from the surface of the primitive with a trajectory towards the second location.

13. The method of claim 8, wherein the direct ray intersects a primitive in the three-dimensional scene further comprising:

issuing a new direct ray at a point different from the point of intersection with the primitive, wherein the new direct ray has a trajectory towards the second location.

14. A system, comprising:

a memory device containing a spatial index having nodes which correspond to bounding volumes which partition a three-dimensional scene; and

a processing element configured to;

issue at least one two rays into the three-dimensional scene originating from a first location corresponding to a sound event, the at least two rays emulating a sound wave generated by the sound event,

wherein a direct ray of the at least two rays has a trajectory directed towards a second location within the three-dimensional scene and wherein an indirect ray of the at least two rays has a trajectory different than the trajectory of the direct ray,

perform ray tracing with the at least two rays using a spatial index having branches and nodes corresponding to bounding volumes which partition the three-dimensional scene, and

based on the results of ray tracing the at least two rays through the three-dimensional scene, determine a resulting sound at the second location.

15. The system of claim 14, wherein the processing element is configured to perform ray tracing by:

taking branches to nodes of the spatial index corresponding to bounding volumes intersected by the at least two rays.

16. The system of claim 15, wherein the processing element is further configured to:

determine that the direct ray intersects a primitive within the three-dimensional scene; and

determine a sound modification factor associated with the intersected primitive.

17. The system of claim 16, wherein the processing element is configured to determine the resulting sound by modifying a prerecorded sound based at least in part on the sound modification factor.

18. The system of claim 14, wherein the processing element is further configured to:

determine that the indirect ray intersects a primitive within the three-dimensional scene;

determine a sound modification factor associated with the primitive,

wherein determining the resulting sound comprises, modifying a prerecorded sound based at least in part on the sound modification factor.

19. The method of claim 14, wherein the indirect ray intersects a primitive in the three-dimensional scene further comprising:

issuing a new indirect ray that is reflected from the surface of the primitive with a trajectory towards the second location.

20. The method of claim 14, wherein the direct ray intersects a primitive in the three-dimensional scene further comprising:

issuing a new direct ray at a point different from the point of intersection with the primitive, wherein the new direct ray has a trajectory towards the second location.