Artificial intelligence (AI) model training using cloud gaming network

• US 10,576,380 B1
• Filed: 11/05/2018
• Issued: 03/03/2020
• Est. Priority Date: 11/05/2018
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Inventors: Beltran, Justin V., Butler, Dylan, Kragenbrink, Kevin
• Primary Examiner(s): Coburn, Corbett B
• Application Number:
  US16/180,787
• Time in Patent Office: 484 Days
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/77 involving data related to game devices or game servers, e.g. configuration data, software version or amount of memory
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A method for processing an artificial intelligence (AI) model for a gaming application. The method includes training the AI model from a plurality of game plays of a scenario of the gaming application using training state data collected from the plurality of game plays of the scenario and associated success criteria of each of the plurality of game plays. The method includes receiving first input state data during a first game play of the scenario. The method includes applying the first input state data to the AI model to generate an output indicating a degree of success for the scenario for the first game play. The method includes performing an analysis of the output based on a predefined objective. The method includes performing an action to achieve the predefined objective based on the output that is analyzed.

Claims

1. A method for processing an artificial intelligence (AI) model for a gaming application, comprising:

training the AI model from a plurality of game plays of a scenario of the gaming application using training state data collected from the plurality of game plays of the scenario and associated success criteria of each of the plurality of game plays;

receiving first input state data during a first game play of the scenario;

applying the first input state data to the AI model to generate an output indicating a degree of success for the scenario for the first game play;

performing an analysis of the output based on a predefined objective; and

performing an action to achieve the predefined objective based on the output that is analyzed.

2. The method of claim 1, wherein the training an AI model includes:

executing at a plurality of servers a plurality of instances of the gaming application supporting the plurality of game plays;

collecting at the plurality of servers the training state data that is associated with corresponding game plays of the scenario;

defining the success criteria; and

providing the training state data and the success criteria to a deep learning engine.

3. The method of claim 2, wherein the plurality of game plays is controlled by a plurality of players via a plurality of client devices.

4. The method of claim 2, wherein the performing an action includes:

wherein the predefined objective is to automatically train the AI model,

wherein the plurality of game plays is controlled by the AI model during training,

wherein the first input state data is generated by the AI model as a previous action taken during training, and

determining as the analysis a plurality of sets of controller inputs for the first game play to be played subsequent to the first input state data;

selecting a first set of controller inputs predicted to satisfy the success criteria and predicted to have a highest degree of success among the plurality of sets of controller inputs; and

delivering as the action the first set of controller inputs to a corresponding instance of the gaming application for execution.

5. The method of claim 2, wherein the performing an action includes:

wherein the predefined objective is to automatically test the gaming application,

wherein the first input state data is generated by the AI model as a previous action during training,

determining as the analysis different permutations for responding to the first input state data, each permutation including a unique set of controller inputs to be taken; and

executing as the action the different permutations to discover any anomaly in playing the gaming application.

6. The method of claim 2,

wherein the plurality of game plays of the gaming application is controlled by a first player through one or more client devices,

wherein the trained AI model is a virtual player that simulates the first player.

7. The method of claim 1, further comprising:

wherein the predefined objective is to provide gaming support,

determining the first input state data does not satisfy the success criteria based on the output that is analyzed;

determining a first user profile for a first player controlling the first game play based on the output that is analyzed and the first input state data; and

providing as the action a recommendation on how to play the scenario to a client device of the first player, the recommendation reflecting a skill of the first player based on the first user profile.

8. The method of claim 1, wherein performing an analysis of the output includes:

wherein the predefined objective is to provide coaching,

identifying a weakness of a first player controlling the first game play by determining that the first input state data has a lower than average degree of success; and

executing as the action a training session corresponding to the weakness for delivery to a client device of the first player.

9. The method of claim 1, further comprising:

wherein the predefined objective is to provide parity in game plays,

receiving a plurality of input state data during a second plurality of game plays of the scenario conducted after training the AI model, the second plurality of game plays being controlled by a plurality of players, the plurality of input state data including a plurality of player characteristic metrics of the plurality of players;

applying the plurality of input state data to the AI model to generate a plurality of outputs indicating a plurality of degrees of success for the second plurality of game plays of the scenario;

analyzing the plurality of outputs to determine a plurality of player profiles based on the plurality of player characteristic metrics; and

building balanced teams of players from the plurality of players based on corresponding player profiles to achieve the predefined objective.

10. The method of claim 9, wherein a player characteristic metric includes:

accuracy of a corresponding player;

or

speed of generating a sequence of controller inputs by the corresponding player;

or

reaction time of the corresponding player when responding to an event in the scenario;

or consistency of the corresponding player;

or

transition time of the corresponding player between a first target and a second target.

11. A non-transitory computer-readable medium storing a computer program for artificial intelligence (AI) training, said computer-readable medium comprising:

program instructions for training the AI model from a plurality of game plays of a scenario of the gaming application using training state data collected from the plurality of game plays of the scenario and associated success criteria of each of the plurality of game plays;

program instructions for receiving first input state data during a first game play of the scenario;

program instructions for applying the first input state data to the AI model to generate an output indicating a degree of success for the scenario for the first game play;

program instructions for performing an analysis of the output based on a predefined objective; and

program instructions for performing an action to achieve the predefined objective based on the output that is analyzed.

12. The non-transitory computer-readable medium of claim 11, wherein the program instructions for training an AI model includes:

program instructions for executing at a plurality of servers a plurality of instances of the gaming application supporting the plurality of game plays;

program instructions for collecting at the plurality of servers the training state data that is associated with corresponding game plays of the scenario;

program instructions for defining the success criteria; and

program instructions for providing the training state data and the success criteria to a deep learning engine.

13. The non-transitory computer-readable medium of claim 12, wherein the plurality of game plays is controlled by a plurality of players via a plurality of client devices.

14. The non-transitory computer-readable medium of claim 11, further comprising:

wherein the predefined objective is to provide gaming support,

program instructions for determining the first input state data does not satisfy the success criteria based on the output that is analyzed;

program instructions for determining a first user profile for a first player controlling the first game play based on the output that is analyzed and the first input state data; and

program instructions for providing as the action a recommendation on how to play the scenario to a client device of the first player, the recommendation reflecting a skill of the first player based on the first user profile.

15. The non-transitory computer-readable medium of claim 11, wherein program instructions for performing an analysis of the output includes:

wherein the predefined objective is to provide coaching,

program instructions for identifying a weakness of a first player controlling the first game play by determining that the first input state data has a lower than average degree of success; and

program instructions for executing as the action a training session corresponding to the weakness for delivery to a client device of the first player.

16. A computer system comprising:

a processor; and

memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method for artificial intelligence (AI) training comprising;

training the AI model from a plurality of game plays of a scenario of the gaming application using training state data collected from the plurality of game plays of the scenario and associated success criteria of each of the plurality of game plays;

receiving first input state data during a first game play of the scenario;

applying the first input state data to the AI model to generate an output indicating a degree of success for the scenario for the first game play;

performing an analysis of the output based on a predefined objective; and

performing an action to achieve the predefined objective based on the output that is analyzed.

17. The computer system of claim 16, wherein in the method the training an AI model includes:

executing at a plurality of servers a plurality of instances of the gaming application supporting the plurality of game plays;

collecting at the plurality of servers the training state data that is associated with corresponding game plays of the scenario;

defining the success criteria; and

providing the training state data and the success criteria to a deep learning engine.

18. The computer system of claim 17, wherein in the method the plurality of game plays is controlled by a plurality of players via a plurality of client devices.

19. The computer system of claim 16, the method further comprising:

wherein the predefined objective is to provide gaming support,

determining the first input state data does not satisfy the success criteria based on the output that is analyzed;

determining a first user profile for a first player controlling the first game play based on the output that is analyzed and the first input state data; and

providing as the action a recommendation on how to play the scenario to a client device of the first player, the recommendation reflecting a skill of the first player based on the first user profile.

20. The computer system of claim 16, wherein in the method performing an analysis of the output includes:

wherein the predefined objective is to provide coaching,

identifying a weakness of a first player controlling the first game play by determining that the first input state data has a lower than average degree of success; and

executing as the action a training session corresponding to the weakness for delivery to a client device of the first player.