Voxel-based, real-time acoustic adjustment

• US 9,937,422 B2
• Filed: 12/09/2015
• Issued: 04/10/2018
• Est. Priority Date: 12/09/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ridihalgh, Robert Lucas, Ballard, Jeffrey
• Primary Examiner(s): Saunders, Jr., Joseph
• Application Number:
  US14/964,390
• Publication Number:
  US 20170165575A1
• Time in Patent Office: 853 Days
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Systems, methods, and computer media for adjusting acoustics in real time are provided herein. An acoustic adjustment system can include a processor, a memory, a voxelizer, and a pathing analyzer. The voxelizer can be configured to generate a voxel-based representation of a three-dimensional (3D) interactive media environment. The voxel-based representation includes a number of individual voxels that together represent the environment. The voxelizer can also be configured to identify a plurality of blocked voxels that intersect visual elements of the 3D interactive media environment. The pathing analyzer can be configured to determine, for the clear voxels, audio path information representing a path of clear voxels between the voxel and the location of the listener. The audio path information can be used to adjust sounds emitted by sound emitters in the media environment to realistically reflect the distances the sound travels to reach the listener.

Claims

1. A method for acoustic adjustment, the method comprising:

determining a location of a listener in a three-dimensional (3D) interactive media environment;

determining locations of one or more sound emitters in the 3D interactive media environment;

providing (i) spatial information for the 3D interactive media environment, (ii) the location of the listener, and (iii) the location of the one or more sound emitters to an acoustic adjustment system configured to generate a voxel-based representation of the 3D interactive media environment and analyze the propagation of audio signals within the 3D interactive media environment;

receiving, from the acoustic adjustment system, (i) audio path information from the locations of the one or more sound emitters to the location of the listener and (ii) reverberation information for the location of the listener, wherein the reverberation information comprises a distance from the location of the listener to a blocked voxel in multiple directions; and

adjusting audio signals produced by the one or more sound emitters based on the audio path information and the reverberation information.

2. The method of claim 1, wherein the audio path information reflects, for respective voxels, of the voxel-based representation, in which the one or more sound emitters are located, a distance of a path of clear voxels between the voxel and the location of the listener.

4. The system of claim 1, wherein the path of clear voxels indicates a path of sound, in the 3D interactive media environment, between a location corresponding to the voxel and the location of the listener.

5. The system of claim 1, wherein the 3D interactive media environment is at least one of a video game environment, a virtual reality environment, or an augmented reality environment.

6. The system of claim 1, wherein the system is part of a video game console, and wherein the pathing analyzer is configured to provide audio path information for a voxel in response to a request from a video game application.

7. The system of claim 1, wherein the path of clear voxels is a shortest clear-voxel path between the location of the listener and the voxel.

8. The system of claim 1, wherein the audio path information comprises an angle from the voxel to the next voxel of the path of clear voxels.

9. The system of claim 1, wherein the pathing analyzer is further configured to determine, by the processor and for the respective voxels of the plurality of voxels that are clear voxels, audio path information representing one or more additional paths of clear voxels between the voxel and the location of the listener.

10. The system of claim 1, further comprising an audio mixer configured to, by the processor, adjust one or more audio signals that, in the 3D interactive environment, reach the location of the listener, the adjusting based at least in part on the audio path information.

11. The system of claim 1, further comprising an environment analyzer configured to determine, by the processor, a distance, from at least one of the location of the listener or a location of a sound emitter, to a blocked voxel in respective directions of a plurality of directions, wherein the distances indicate a sound reverberation adjustment to audio reaching, in the 3D interactive media environment, the location of the listener, or to audio originating, in the 3D interactive media environment, from the sound emitter.

12. The system of claim 11, wherein the environment analyzer is further configured to determine, by the processor, a material type of the blocked voxels in the respective directions, wherein the material type indicates an additional sound reverberation adjustment to audio reaching, in the 3D interactive media environment, the location of the listener.

3. An acoustic adjustment system, comprising:

a processor;

a memory;

a voxelizer configured to;

generate, by the processor, a voxel-based representation of a three-dimensional (3D) interactive media environment, the voxel-based representation comprising a plurality of voxels,

store the voxel-based representation of the 3D interactive media environment in the memory, and

identify, by the processor, a plurality of blocked voxels, wherein blocked voxels are voxels that intersect visual elements of the 3D interactive media environment; and

a pathing analyzer configured to;

determine, by the processor and for respective voxels of the plurality of voxels that are clear voxels, audio path information representing a path of clear voxels between the voxel and a location of the listener, wherein clear voxels are voxels other than blocked voxels, and wherein the audio path information comprises a length of the path of clear voxels; and

determine a ratio of the length of the path of clear voxels to a shortest distance between the voxel and the location of the listener, wherein the shortest distance corresponds to the length of a line connecting the voxel and the location of the listener, and wherein the ratio indicates an audio adjustment that accounts for the length of the path of clear voxels.

13. One or more hardware computer-readable memory or storage devices storing computer-executable instructions for acoustic adjustment, the adjustment comprising:

receiving data representing a three-dimensional (3D) interactive media environment;

receiving a location of a listener within the 3D interactive media environment;

based on the data representing the 3D interactive media environment, generating a voxel-based representation of the 3D interactive media environment, the voxel-based representation comprising a plurality of voxels of a uniform size;

designating voxels, of the plurality of voxels, that intersect a visual element of the 3D interactive media environment as blocked voxels;

for respective voxels of the plurality of voxels that are clear voxels, wherein clear voxels are voxels other than blocked voxels, determining audio path information between the voxel and the location of the listener;

determining sound reverberation information for the location of the listener, wherein the determining comprises determining a distance from the location of the listener to a blocked voxel in respective directions of a plurality of directions, wherein the distances indicate a sound reverberation adjustment to audio reaching, in the 3D interactive media environment, the location of the listener; and

communicating (i) the audio path information for one or more of the respective clear voxels and (ii) the sound reverberation information to an application managing the 3D interactive media environment.

14. The one or more hardware computer-readable memory or storage devices of claim 13, wherein the 3D interactive media environment is a video game environment, and wherein the application is a video game.

15. The one or more hardware computer-readable memory or storage devices of claim 13, wherein the audio path information comprises at least one of an obstruction value or an occlusion value.

16. The one or more hardware computer-readable memory or storage devices of claim 13, wherein the one or more clear voxels for which the audio path information and sound reverberation information are communicated are locations of sound emitters in the 3D interactive media environment.

17. The one or more hardware computer-readable memory or storage devices of claim 13, wherein audio path information and sound reverberation information are determined for clear voxels within one or more areas and not determined for clear voxels outside of the one or more areas.