Interactive voice-controlled companion application for a video game

• US 10,449,440 B2
• Filed: 06/30/2017
• Issued: 10/22/2019
• Est. Priority Date: 06/30/2017
• Status: Active Grant

• Current Assignee: Electronic Arts Incorporated
• Original Assignee: Electronic Arts Incorporated
• Inventors: Sardari, Mohsen, Moss, Kenneth Alan, Zaman, Kazi Atif-Uz, Aghdaie, Navid, Kolen, John, Mattar, Mohamed Marwan
• Primary Examiner(s): Leichliter, Chase E
• Application Number:
  US15/639,973
• Publication Number:
  US 20190001219A1
• Time in Patent Office: 844 Days
• Field of Search: 463 36
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: A63F 2300/6027 using adaptive systems learning from user actions, e.g. for skill level adjustment
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Using voice recognition, a user can interact with a companion application to control a video game from a mobile device. Advantageously, the user can interact with the companion application when the video game is unavailable because, for example, of the user'"'"'s location. Moreover, machine learning may be used to facilitate generating voice responses to user utterances that are predicted to improve or maintain a user'"'"'s level of engagement with the companion application, or its corresponding video game.

Claims

1. A computer-implemented method comprising:

as implemented by an interactive computing system configured with specific computer-executable instructions,

receiving an utterance from a user computing device, the utterance comprising one or more spoken words associated with a video game and spoken by a user;

determining an utterance classification for the utterance; and

responsive to determining that the utterance classification corresponds to a command;

determining the command referenced by the utterance;

determining the video game associated with the command, the video game one of a plurality of video games;

identifying a user account associated with the user;

identifying a host system associated with the video game and the user account, the host system configured to host at least a first portion of an instance of the video game;

loading a user game account associated with the user account at the host system; and

executing the command on behalf of the user with respect to the user game account of the user at the host system without the user accessing the video game.

2. The computer-implemented method of claim 1, further comprising:

determining a priority of the utterance; and

processing the utterance in accordance with the priority of the utterance.

3. The computer-implemented method of claim 2, wherein processing the utterance in accordance with the priority of the utterance comprises providing the utterance to a secondary processing system to process the utterance when the priority is below a threshold priority.

4. The computer-implemented method of claim 1, wherein the utterance is received from a companion application hosted by the user computing device, the companion application distinct from but associated with the video game.

5. The computer-implemented method of claim 1, wherein executing the command on behalf of the user comprises:

instantiating a headless client configured to execute at least a second portion of the instance of the video game;

executing the second portion of the instance of the video game at the headless client; and

executing the command at the headless client.

6. The computer-implemented method of claim 1, wherein, responsive to determining that the utterance classification corresponds to a query, the method further comprises:

accessing one or more knowledge sources associated with the query, at least one of the one or more knowledge sources comprising a repository of game-state information;

generating a query response to the query based at least in part on data obtained from the one or more knowledge sources; and

transmitting the query response to the user computing device.

7. The computer-implemented method of claim 6, wherein generating the query response comprises:

accessing user interaction data for the user;

determining a plurality of eligible query responses based at least in part on the data obtained from the one or more knowledge sources;

generating a score for each of the eligible query responses based on the user interaction data and a parameter function, the parameter function generated based at least in part on a machine learning algorithm; and

selecting the query response from the plurality of eligible query responses based at least in part on the score for each of the eligible query responses.

8. The computer-implemented method of claim 7, further comprising determining one or more features of a signal corresponding to the utterance, wherein generating the score for each of the eligible query responses is further based on the one or more features of the signal corresponding to the utterance.

9. The computer-implemented method of claim 7, wherein selecting the query response from the plurality of eligible query responses comprises selecting the query response that has an engagement score that is greater than engagement scores of the remaining eligible query responses from the plurality of query responses.

10. The computer-implemented method of claim 1, wherein, responsive to determining that the utterance classification corresponds to a statement, further comprising:

applying the statement to a parameter function to determine an action, wherein the action is selected from a plurality of actions to maintain or increase a level of engagement of the user with respect to the video game; and

performing the action with respect to execution of the instance of the video game.

11. The computer-implemented method of claim 10, wherein applying the statement to the parameter function to determine the action comprises applying a plurality of statements corresponding to a plurality of utterances to the parameter function.

12. The computer-implemented method of claim 1, further comprising performing voice recognition on the utterance to determine the one or more spoken words.

13. A system comprising:

an application host system configured to host at least a first portion of an instance of a video game;

an interactive computing system comprising one or more hardware processors, the interactive computing system configured to execute specific computer-executable instructions to at least;

receive an utterance from a user computing device, the utterance associated with a video game and spoken by a user;

determine an utterance classification for the utterance;

responsive to determining that the utterance classification corresponds to a command;

determine the command referenced by the utterance;

determine the video game associated with the command, the video game one of a plurality of video games;

identify a user account associated with the user;

identify the application host system associated with the video game and the user account from a plurality of application host systems;

load a user game account associated with the user account at the application host system; and

execute the command on behalf of the user with respect to the user game account of the user at the application host system without the user accessing the video game.

14. The system of claim 13, wherein the interactive computing system is further configured to execute the command on behalf of the user by:

instantiating a headless client configured to execute at least a second portion of the instance of the video game;

executing the second portion of the instance of the video game at the headless client; and

executing the command at the headless client.

15. The system of claim 13, wherein, responsive to determining that the utterance classification corresponds to a query, the interactive computing system is further configured to execute specific computer-executable instructions to at least:

access one or more knowledge sources associated with the query;

generate a query response to the query based at least in part on data obtained from the one or more knowledge sources; and

transmit the query response to the user computing device.

16. The system of claim 15, wherein the interactive computing system is further configured to generate the query response by:

accessing user interaction data for the user;

determining a plurality of eligible query responses based at least in part on the data obtained from the one or more knowledge sources;

generating a score for each of the eligible query responses based on the user interaction data and a parameter function, the parameter function generated based at least in part on a machine learning algorithm; and

selecting the query response from the plurality of eligible query responses based at least in part on the score for each of the eligible query responses.

17. The system of claim 13, wherein, responsive to determining that the utterance classification corresponds to a statement, the interactive computing system is further configured to execute specific computer-executable instructions to at least:

apply the statement to a parameter function to determine an action; and

perform the action with respect to execution of the instance of the video game.

18. The system of claim 17, wherein the interactive computing system is further configured to apply the statement to the parameter function by:

receiving a plurality of statements, including the statement, from the user computing device;

determining a sentiment of each of the plurality of statements;

aggregating the sentiments of each of the plurality of statements to obtain an aggregated sentiment; and

providing the aggregated sentiment to the parameter function.

19. A non-transitory computer-readable storage medium storing computer executable instructions that, when executed by one or more computing devices, configure the one or more computing devices to perform operations comprising:

receiving an utterance from a user computing device, the utterance associated with a video game;

determining that an utterance classification corresponds to a command;

identifying the command referenced by the utterance;

identifying a user account of the video game that corresponds to an identifier received with the utterance;

selecting a host system for the video game based at least in part on the user account, the host system configured to host at least a first portion of an instance of the video game;

loading game data associated with the user account at the host system; and

executing the command on the instance of the video game at the host system on behalf of a user associated with the user account without the user accessing the video game.

20. The computer-readable, non-transitory storage medium of claim 19, wherein executing the command comprises:

instantiating a headless client configured to execute at least a second portion of the instance of the video game;

executing the second portion of the instance of the video game at the headless client; and

executing the command at the headless client.