Directional audio for virtual environments

• US 10,496,358 B1
• Filed: 02/17/2017
• Issued: 12/03/2019
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hirst, Alistair Robert
• Primary Examiner(s): Elbin, Jesse A
• Application Number:
  US15/436,680
• Time in Patent Office: 1,019 Days
• Field of Search: 700 94, 381 17, 381 18, 381 22, 3484231, 348484, 348485
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10H 2210/301 Soundscape or sound field simulation, reproduction or control for musical purposes, e.g. surround or 3D sound; Granular synthesis
• CPC Class Codes: G10H 2210/305 Source positioning in a soundscape, e.g. instrument positioning on a virtual soundstage, stereo panning or related delay or reverberation changes; Changing the stereo width of a musical source
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

Technology is described for providing audio for digital content. The digital content may be provided to a plurality of devices. Each device may be associated with a profile. At least one group of profiles may be identified from the plurality of devices that share an affiliation. The group of profiles may be represented as a group of environment objects in the digital content. A location may be identified within the digital content that corresponds to the group of environment objects. Audio may be received for the devices. The audio may be received while the digital content is being transmitted to the plurality of devices. The audio may be transmitted to the plurality of devices for directional audio playback. The audio may be directed to correspond with the virtual location of the at least one group of environment objects in the digital content.

Claims

1. A computer implemented method, comprising:

identifying environment objects represented in digital content that are affiliated with one another, wherein the environment objects are virtual three-dimensional (3D) graphical models and the digital content includes an interactive, 3D virtual environment, and the environment objects are associated with users of the 3D virtual environment;

identifying a location represented in the 3D virtual environment that includes the environment objects;

obtaining audio from client devices of the users associated with the environment objects at the location represented in the 3D virtual environment; and

providing the audio with the environment objects for directional audio playback to simulate the environment objects as a source of the audio in the 3D virtual environment.

2. The method of claim 1, further comprising calculating a spread value for the audio used to determine a plurality of speakers used for playback of the audio.

3. The method of claim 2, wherein the spread value is calculated based in part on the location represented in the 3D virtual environment and a number of the environment objects.

4. The method of claim 2, wherein the spread value is calculated based in part on a distance between the environment objects.

5. The method of claim 2, further comprising transmitting the audio for directional playback on a virtualized surround sound system using the spread value.

6. The method of claim 2, further comprising providing the audio and the spread value with a sound intensity that is associated with a number of the environment objects.

7. The method of claim 1, wherein the users of the 3D virtual environment include players of an online game using the 3D virtual environment or spectators watching the online game being played.

8. The method of claim 1, further comprising processing the audio to apply directionality and sound intensity to the audio based on the location of the environment object in the 3D virtual environment.

9. A system comprising:

a processor;

a memory device including instructions that, when executed by the processor, cause the processor to;

identify environment objects represented in digital content that are affiliated with one another, wherein the environment objects are virtual 3D graphical models and the digital content includes an online game with a 3D virtual environment, and the environment objects are associated with users of the 3D virtual environment;

identify a location represented in the 3D virtual environment that includes the environment objects;

obtain audio from client devices of the users associated with the environment objects;

calculate a spread value for the audio based in part on the location and the environment objects, wherein the spread value determines a plurality of speakers used for playback of the audio; and

provide the audio and the spread value for directional audio playback using the plurality of speakers that simulate the environment objects as a source of the audio at the location in the 3D virtual environment.

10. The system of claim 9, wherein the memory device includes instructions that, when executed by the processor, cause the system to transmit the audio for directional audio playback on a surround sound system.

11. The system of claim 9, wherein the memory device includes instructions that, when executed by the processor, cause the system to:

identify an updated virtual location for the environment objects as the environment objects move within the 3D virtual environment; and

transmit the audio that is associated with the updated virtual location to enable directional audio playback.

12. The system of claim 9, wherein the memory device includes instructions that, when executed by the processor, cause the system to select the audio having a sound intensity that corresponds with a number of the environment objects and an event that is occurring within the 3D virtual environment.

13. The system of claim 9, wherein the memory device includes instructions that, when executed by the processor, cause the system to process the audio to apply directionality and sound intensity to the audio based on the location of the environment object in the 3D virtual environment.

14. A computer implemented method, comprising:

identifying 3D virtual objects represented in an interactive, 3D virtual environment that are affiliated with one another, and the 3D virtual objects are associated with users of the 3D virtual environment, wherein the 3D virtual environment is part of an online game;

identifying a location represented in the 3D virtual environment that includes the 3D virtual objects;

obtaining audio from client devices of the users associated with the 3D virtual objects at the location represented in the 3D virtual environment; and

providing the audio with the 3D virtual object for directional audio playback on a virtualized surround sound system to simulate the 3D virtual objects as a source of the audio in the 3D virtual environment.

15. The method of claim 14, further comprising calculating a spread value for the audio used to determine a plurality of speakers used for playback of the audio.

16. The method of claim 15, wherein the spread value is calculated based in part on the location represented in the 3D virtual environment and a number of the 3D virtual objects.

17. The method of claim 15, wherein the spread value is calculated based in part on a distance between the 3D virtual objects.

18. The method of claim 15, further comprising transmitting the audio for directional playback on the virtualized surround sound system using the spread value.

19. The method of claim 15, further comprising providing the audio and the spread value with a sound intensity that is associated with a number of the 3D virtual objects.