Player audio analysis in online gaming environments

• US 10,293,260 B1
• Filed: 06/05/2015
• Issued: 05/21/2019
• Est. Priority Date: 06/05/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Evans, Ethan Zane, George, Michael Martin, Frazzini, Michael Anthony, Hirst, Alistair Robert, Van Gorder, Matthew James
• Primary Examiner(s): Shah, Milap
• Application Number:
  US14/732,580
• Time in Patent Office: 1,446 Days
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

An audio analysis system that may analyze participant audio input in an online game environment. The audio analysis system may receive audio input for participants (players or spectators) in a current game session of a game executing within an online game system. The audio analysis system may analyze the audio input to generate audio analysis information for the current game session, and provide the audio analysis information to the online game system. The audio analysis information may be applied within the online game system in various ways. For example, the information may be displayed to the players and/or spectators via a user interface provided by game clients executing on respective client devices. The audio analysis system may provide an application programming interface (API) to receive audio input and provide audio analysis information to one or more game systems.

Claims

1. A system, comprising:

one or more computing devices to;

perform an electronic game;

receive audio input from one or more players of the electronic game;

analyze the audio input to determine audio analysis information including one or more audio patterns indicative of player emotion;

map the one or more audio patterns indicative of player emotion to identify corresponding one or more events that happened in the electronic game; and

cause a visualization of the one or more audio patterns to be displayed to the one or more players, wherein the visualization indicates one or more locations of the one or more events within the electronic game.

2. The system as recited in claim 1, wherein the one or more computing devices are configured to:

iteratively perform said receive, said map, and said cause during performance of the electronic game;

detect a change in state of at least one of the one or more players while iteratively performing said receive and said analyze; and

change the visualization for the at least one of the one or more players in response to said detect.

3. The system as recited in claim 1, wherein, to analyze the audio input, the one or more computing devices are configured to:

apply signal analysis to audio input signals of the audio input associated with the one or more players to determine audio signal metrics;

map the audio signal metrics to the one or more audio patterns to determine player audio analysis data;

normalize and aggregate the player audio analysis data; and

analyze the aggregated player audio analysis data to determine the audio analysis information for the electronic game.

4. The system as recited in claim 1, wherein audio analysis information indicates an emotional state of the one or more players, and wherein the visualization modifies game characters corresponding to at least one of the one or more players according to a current emotional state.

5. The system as recited in claim 1, wherein audio analysis information indicates an event or location in the electronic game within a game universe involving the one or more players.

6. The system as recited in claim 1, wherein the one or more computing devices are further configured to:

record the electronic game to generate a game session video; and

tag the game session video with metadata indicating at least one event in the game session as indicated by the audio analysis information.

7. A method, comprising:

iteratively performing, by an audio analysis module implemented on one or more computing devices;

receiving audio input for a plurality of players participating in a current game session of a game executing within a game system;

analyzing the audio input to generate audio analysis information including one or more audio patterns indicative of player emotion;

mapping the one or more audio patterns indicative of player emotion to identify corresponding one or more events that happened in the game; and

causing a visualization of the one or more audio patterns to be displayed to the one or more players, wherein the visualization indicates one or more locations of the one or more events within the game.

8. The method as recited in claim 7, wherein the one or more audio patterns affects one or both of execution of the game session within the game system or visual presentation of the current game session on client devices.

9. The method as recited in claim 7, further comprising:

mapping audio input signals of the audio input corresponding to the plurality of players to the one or more audio patterns to determine player audio analysis data; and

determining the audio analysis information for the current game session from the player audio analysis data.

10. The method as recited in claim 7, further comprising:

applying signal analysis to audio input signals of the audio input corresponding to the plurality of players to determine audio signal metrics for the players, wherein the audio signal metrics include vocal metrics and speech metrics;

applying an emotion recognition technique to the vocal metrics to determine emotion content from the audio input signals;

applying a speech recognition technique to the speech metrics to determine speech content from the audio input signals; and

comparing the determined emotion content and speech content to the one or more audio patterns to determine audio analysis information.

11. The method as recited in claim 7, wherein the current game session involves the plurality of players participating in the current game session via client devices, wherein audio analysis information generated from the one or more audio patterns indicates at least one event of the one or more events in the current game session, the method further comprising providing a visual indication of the at least one event of the one or more events for display.

12. The method as recited in claim 7, wherein the current game session involves the plurality of players participating in the current game session via game clients executing on respective client devices, wherein the audio analysis information indicates an emotional state of one or more of the plurality of players, the method further comprising providing a visual indication of the emotional state of the one or more players for display.

13. The method as recited in claim 7, wherein the audio analysis information indicates a current emotional state of two or more of the plurality of players, the method further comprising providing a visual indication of the current emotional state of the two or more of the plurality of players to a plurality of spectators of the current game session, wherein the spectators are viewing the current game session via respective spectator devices.

14. The method as recited in claim 7, wherein the audio analysis information indicates at least one event in the current game session, the method further comprising providing a visual indication of the at least one event to a plurality of spectators of the current game session, wherein the spectators are viewing the current game session via respective spectator devices.

15. The method as recited in claim 7, wherein the audio analysis information indicates at least one highlight segment from the current game session, the method further comprising providing the at least one highlight segment from the current game session to a highlight user interface on one or more client devices.

16. The method as recited in claim 7, further comprising:

receiving spectator audio input for a plurality of spectators of the current game session, wherein the spectators are viewing the current game session via respective client devices;

analyzing the spectator audio input to generate spectator audio analysis information; and

providing the spectator audio analysis information to the game system, wherein the spectator audio analysis information affects one or both of execution of the current game session within the game system or visual presentation of the current game session on the client devices.

17. The method as recited in claim 16, wherein the spectator audio analysis information indicates an emotional state of the plurality of spectators, the method further comprising providing a visual indication of the emotional state of the plurality of spectators to game clients executing on respective client devices.

18. The method as recited in claim 7, wherein the audio analysis information indicates an active region or group of players within a game universe of the game, the method further comprising providing a visual indication of the active region or group of players within the game universe to respective client devices.

19. The method as recited in claim 7, further comprising tagging a recording of the current game session with metadata indicating at least one event of the one or more events in the current game session as indicated by the audio analysis information.

20. The method as recited in claim 7, wherein the game system is an online game system, wherein the game is an online multiplayer game, and wherein the current game session involves a plurality of game characters acting within a game universe as controlled by the plurality of players via game clients executing on respective client devices.

21. The method as recited in claim 7, wherein the audio analysis module is a component of the game system.

22. The method as recited in claim 7, wherein the audio analysis module is implemented as a service on a network, and wherein the audio analysis module provides an application programming interface (API) to receive audio input from one or more game systems and to provide audio analysis information to the one or more game systems.

23. A non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to implement an audio analysis system configured to:

receive audio input for a plurality of players participating in a current game session of a multiplayer game executing within an online game system;

analyze the audio input to determine one or more audio patterns indicative of player emotion;

map the one or more audio patterns indicative of player emotion to identify corresponding one or more events that happened in the game session; and

provide the one or more audio patterns to the online game system, wherein the one or more audio patterns affects one or both of execution of the game session within the online game system or visual presentation of the game session on client devices, wherein the visual presentation indicates one or more locations of the one or more events within the multiplayer game.

24. The non-transitory computer-readable storage medium as recited in claim 23, wherein the audio analysis system is configured to:

determine audio analysis information for the plurality of players from the one or more audio patterns.

25. The non-transitory computer-readable storage medium as recited in claim 24, wherein the audio analysis information indicates one or more of an emotional state of one or more of the players, an event of the one or more events in the game session, or a group of players in the game session.