Controlling behavior of entities in funnel sections of a computer-represented environment

• US 10,406,438 B2
• Filed: 07/28/2017
• Issued: 09/10/2019
• Est. Priority Date: 07/28/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Catlin, Scott C., Schlimmer, Jeffrey C.
• Primary Examiner(s): McClellan, James S.
• Assistant Examiner(s): Carter, Kevin M
• Application Number:
  US15/663,611
• Publication Number:
  US 20190030432A1
• Time in Patent Office: 774 Days
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks

Abstract

Innovations in the area of controlling behavior of computer-controlled entities such as cars in a computer-represented environment are presented herein. In various examples described herein, the behavior of computer-controlled cars is controlled by an artificial intelligence (“AI”) behavior engine. The AI behavior engine performs operations to determine if the behavior (driving pattern, etc.) of a computer-controlled car should change for a funnel section. If so, the AI behavior engine switches the computer-controlled car to a trailing mode configuration and selects a candidate car to follow in the funnel section based on various criteria. The AI behavior engine also determines how to follow the candidate car by selecting a racing path and setting rules to determine the speed to use for following the candidate car. Control values are then set. The described innovations also generally apply to other racing scenarios, self-driving/autonomous cars, and other applications with robotic equipment and devices.

Claims

1. One or more computer-readable media storing computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations to control behavior of a first virtual entity in a computer-represented environment, the behavior of the first virtual entity being controlled by an artificial intelligence (“

AI”

) behavior engine executable on the one or more processing units, the AI behavior engine being part of a gaming system, the operations comprising;

determining to change the behavior of the first virtual entity for a funnel section in the computer-represented environment, the funnel section being part of a racing course in a racing game; and

as part of switching to funnel section behavior in response to the determining to change the behavior of the first virtual entity for the funnel section in the computer-represented environment;

selecting a second virtual entity for the first virtual entity to follow in the funnel section;

determining how to follow the second virtual entity; and

updating game information for the racing game, including setting one or more control values for the behavior of the first virtual entity based at least in part on results of the determining how to follow the second virtual entity.

2. The one or more computer-readable media of claim 1, wherein:

the first virtual entity and the second virtual entity are each selected from the group consisting of car, motorcycle, bicycle, boat, ski racer, and robot;

behavior of the second virtual entity is controlled by the AI behavior engine or controlled in response to user input.

3. The one or more computer-readable media of claim 1, wherein the racing course is annotated to designate a section ahead of the funnel section.

4. The one or more computer-readable media of claim 1, wherein the funnel section is dynamically placed on the racing course in response to detection of an obstacle or congestion on the racing course.

5. The one or more computer-readable media of claim 1, wherein the determining to change the behavior of the first virtual entity includes:

detecting entry of the first virtual entity into a designated part of the funnel section;

determining a count of virtual entities in the designated part;

comparing the count to a threshold; and

determining that the count satisfies the threshold.

6. The one or more computer-readable media of claim 1, wherein the first virtual entity has an operational mode selected from among multiple available driving modes, and wherein, to change the behavior of the first virtual entity, the operational mode is changed to a trailing mode so as to follow the second virtual entity into the funnel section.

7. The one or more computer-readable media of claim 1, wherein the AI behavior engine controls the behavior of the first virtual entity based at least in part on settings, each of the settings being (a) predefined or (b) learned from user behavior and/or trial-and-error evaluation during operation.

8. The one or more computer-readable media of claim 1, wherein the selecting the second virtual entity includes:

setting a scoring function that ranks each of multiple different candidate moving paths; and

selecting one of the multiple different candidate moving paths that has a best score.

9. The one or more computer-readable media of claim 8, wherein the scoring function sets a score value for each of the multiple different candidate moving paths based on one or more of:

a penalty for an obstacle;

a penalty for deviation from an optimal moving path;

a penalty for deviation from a moving path that the first virtual entity currently follows;

a benefit for following a candidate virtual entity in a compact configuration;

a benefit for following a candidate virtual entity ahead of the first virtual entity;

a penalty for selection of a virtual entity that another virtual entity has selected to follow; and

a benefit for following a candidate virtual entity closest to the first virtual entity laterally.

10. The one or more computer-readable media of claim 1, wherein the determining how to follow the second virtual entity includes:

matching a moving path of the second virtual entity; and

matching speed of the second virtual entity.

11. The one or more computer-readable media of claim 1, wherein the one or more processing units are part of a server system for the gaming system, and wherein the operations further comprise:

sending updates to one or more local client systems, wherein the updates indicate the one or more control values for the behavior of the first virtual entity.

12. The one or more computer-readable media of claim 11, wherein the operations further comprise:

for each of the one or more local client systems, determining a level of detail suitable for the updates.

13. The one or more computer-readable media of claim 1, wherein the one or more processing units are part of a local client system for the gaming system, and wherein the operations further comprise:

sending updates to a server system for the gaming system, wherein the updates indicate the one or more control values for the behavior of the first virtual entity.

14. The one or more computer-readable media of claim 1, wherein the one or more processing units are part of a local client system for the gaming system, and wherein the operations further comprise:

receiving first updates, from a server system for the gaming system, wherein the first updates indicate control values for one or more other virtual entities whose behavior is not controlled at the local client system; and

sending second updates, to the server system, wherein the second updates indicate control values for one or more virtual entities whose behavior is controlled at the local client system, the one or more virtual entities whose behavior is controlled at the local client system including the first virtual entity.

15. The one or more computer-readable media of claim 1, wherein the operations are performed are performed as part of an iteration of a loop, and wherein the operations further comprise repeating, in one or more additional iterations of the loop, the determining to change the behavior, the selecting the second virtual entity, the determining how to follow, and the updating the game information.

16. The one or more computer-readable media of claim 15, wherein the second virtual entity is different in different iterations of the loop.

17. The one or more computer-readable media of claim 1, wherein the operations further comprise:

checking an on/off setting for controlling the funnel section behavior, wherein the determining to change the behavior, the selecting the second virtual entity, the determining how to follow, and the updating the game information are contingent on value of the on/off setting.

18. The one or more computer-readable media of claim 1, wherein the operations further comprise, for each of one or more other virtual entities whose behavior is controlled by the AI behavior engine:

repeating the determining to change the behavior, the selecting, the determining how to follow, and the updating the game information.

19. In a computer system that includes one or more processing units, a method for controlling behavior of a first virtual entity in a computer-represented environment, the behavior of the first virtual entity being controlled by an artificial intelligence (“

AI”

) behavior engine executable on the one or more processing units, the AI behavior engine being part of a gaming system, the method comprising;

determining to change the behavior of the first virtual entity for a funnel section in the computer-represented environment, the funnel section being part of a racing course in a racing game; and

as part of switching to funnel section behavior in response to the determining to change the behavior of the first virtual entity for the funnel section in the computer-represented environment;

selecting a second virtual entity for the first virtual entity to follow in the funnel section;

determining how to follow the second virtual entity; and

updating game information for the racing game, including setting one or more control values for the behavior of the first virtual entity based at least in part on results of the determining how to follow the second virtual entity.

26. The method of claim 19, wherein the determining to change the behavior of the first virtual entity includes:

detecting entry of the first virtual entity into a designated part of the funnel section;

determining a count of virtual entities in the designated part;

comparing the count to a threshold; and

determining that the count satisfies the threshold.

27. The method of claim 19, wherein the selecting the second virtual entity includes:

setting a scoring function that ranks each of multiple different candidate moving paths; and

selecting one of the multiple different candidate moving paths that has a best score.

28. The method of claim 27, wherein the scoring function sets a score value for each of the multiple different candidate moving paths based on one or more of:

a penalty for an obstacle;

a penalty for deviation from an optimal moving path;

a penalty for deviation from a moving path that the first virtual entity currently follows;

a benefit for following a candidate virtual entity in a compact configuration;

a benefit for following a candidate virtual entity ahead of the first virtual entity;

a penalty for selection of a virtual entity that another virtual entity has selected to follow; and

a benefit for following a candidate virtual entity closest to the first virtual entity laterally.

29. The method of claim 19, wherein the determining how to follow the second virtual entity includes:

matching a moving path of the second virtual entity; and

matching speed of the second virtual entity.

30. The method of claim 19, further comprising:

checking an on/off setting for controlling the funnel section behavior, wherein the determining to change the behavior, the selecting the second virtual entity, the determining how to follow, and the updating the game information are contingent on value of the on/off setting.

20. A computer system comprising one or more processing units and memory storing computer-executable instructions for an artificial intelligence (“

AI”

) behavior engine executable on the one or more processing units to perform operations for controlling behavior of a first virtual entity in a computer-represented environment, the AI behavior engine being part of a gaming system, the operations comprising;

determining to change the behavior of the first virtual entity for a funnel section in the computer-represented environment, the funnel section being part of a racing course in a racing game; and

as part of switching to funnel section behavior in response to the determining to change the behavior of the first virtual entity for the funnel section in the computer-represented environment;

selecting a second virtual entity for the first virtual entity to follow in the funnel section;

determining how to follow the second virtual entity; and

updating game information for the racing game, including setting one or more control values for the behavior of the first virtual entity based at least in part on results of the determining how to follow the second virtual entity.

21. The computer system of claim 20, wherein the determining to change the behavior of the first virtual entity includes:

detecting entry of the first virtual entity into a designated part of the funnel section;

determining a count of virtual entities in the designated part;

comparing the count to a threshold; and

determining that the count satisfies the threshold.

22. The computer system of claim 20, wherein the selecting the second virtual entity includes:

setting a scoring function that ranks each of multiple different candidate moving paths; and

selecting one of the multiple different candidate moving paths that has a best score.

23. The computer system of claim 22, wherein the scoring function sets a score value for each of the multiple different candidate moving paths based on one or more of:

a penalty for an obstacle;

a penalty for deviation from an optimal moving path;

a penalty for deviation from a moving path that the first virtual entity currently follows;

a benefit for following a candidate virtual entity in a compact configuration;

a benefit for following a candidate virtual entity ahead of the first virtual entity;

a penalty for selection of a virtual entity that another virtual entity has selected to follow; and

a benefit for following a candidate virtual entity closest to the first virtual entity laterally.

24. The computer system of claim 20, wherein the determining how to follow the second virtual entity includes:

matching a moving path of the second virtual entity; and

matching speed of the second virtual entity.

25. The computer system of claim 20, wherein the operations further comprise:

checking an on/off setting for controlling the funnel section behavior, wherein the determining to change the behavior, the selecting the second virtual entity, the determining how to follow, and the updating the game information are contingent on value of the on/off setting.