CONTROLLING BEHAVIOR OF ENTITIES IN FUNNEL SECTIONS OF A COMPUTER-REPRESENTED ENVIRONMENT

• US 20190030432A1
• Filed: 07/28/2017
• Published: 01/31/2019
• Est. Priority Date: 07/28/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Catlin, Scott C., Schlimmer, Jeffrey C.
• Granted Patent: US 10,406,438 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/40 Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks

Abstract

Innovations in the area of controlling behavior of computer-controlled entities such as cars in a computer-represented environment are presented herein. In various examples described herein, the behavior of computer-controlled cars is controlled by an artificial intelligence (“AI”) behavior engine. The AI behavior engine performs operations to determine if the behavior (driving pattern, etc.) of a computer-controlled car should change for a funnel section. If so, the AI behavior engine switches the computer-controlled car to a trailing mode configuration and selects a candidate car to follow in the funnel section based on various criteria. The AI behavior engine also determines how to follow the candidate car by selecting a racing path and setting rules to determine the speed to use for following the candidate car. Control values are then set. The described innovations also generally apply to other racing scenarios, self-driving/autonomous cars, and other applications with robotic equipment and devices.

Claims

1. One or more computer-readable media storing computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations to control behavior of a first entity in a computer-represented environment, the behavior of the first entity being controlled by a behavior engine executable on the one or more processing units, the operations comprising:

determining to change the behavior of the first entity for a funnel section in the computer-represented environment; and

as part of switching to funnel section behavior in response to the determining to change the behavior of the first entity for the funnel section in the computer-represented environment;

selecting a second entity for the first entity to follow in the funnel section;

determining how to follow the second entity; and

setting one or more control values for the behavior of the first entity based at least in part on results of the determining how to follow the second entity.

2. The one or more computer-readable media of claim 1, wherein:

the first entity and the second entity are each selected from the group consisting of car, motorcycle, bicycle, boat, ski racer, and robot;

each of the first entity and the second entity represents a real-world entity or a virtual entity; and

behavior of the second entity is controlled by the behavior engine or controlled in response to user input.

3. The one or more computer-readable media of claim 1, wherein the funnel section is part of a course, and wherein the course is annotated to designate a section ahead of the funnel section.

4. The one or more computer-readable media of claim 1, wherein the funnel section is part of a course, and wherein the funnel section is dynamically placed on the course in response to detection of an obstacle or congestion on the course.

5. The one or more computer-readable media of claim 1, wherein the determining to change the behavior of the first entity includes:

detecting entry of the first entity into a designated part of the funnel section;

determining a count of entities in the designated part;

comparing the count to a threshold; and

determining that the count satisfies the threshold.

6. The one or more computer-readable media of claim 1, wherein the first entity has an operational mode selected from among multiple available driving modes, and wherein, to change the behavior of the first entity, the operational mode is changed to a trailing mode so as to follow the second entity into the funnel section.

7. The one or more computer-readable media of claim 1, wherein the behavior engine controls the behavior of the first entity based at least in part on settings, each of the settings being (a) predefined or (b) learned from user behavior and/or trial-and-error evaluation during operation.

8. The one or more computer-readable media of claim 1, wherein the selecting the second entity includes:

setting a scoring function that ranks each of multiple different candidate moving paths; and

selecting one of the multiple different candidate moving paths that has a best score.

9. The one or more computer-readable media of claim 8, wherein the scoring function sets a score value for each of the multiple different candidate moving paths based on one or more of:

a penalty for an obstacle;

a penalty for deviation from an optimal moving path;

a penalty for deviation from a moving path that the first entity currently follows;

a benefit for following a candidate entity in a compact configuration;

a benefit for following a candidate entity ahead of the first entity;

a penalty selection of an entity that another entity has selected to follow; and

a benefit for following a candidate entity closest to the first entity laterally.

10. The one or more computer-readable media of claim 1, wherein the determining how to follow the second entity includes:

matching a moving path of the second entity; and

matching speed of the second entity.

11. The one or more computer-readable media of claim 1, wherein the one or more processing units are part of a server system, and wherein the operations further comprise:

sending updates to one or more local client systems, wherein the updates indicate the one or more control values for the behavior of the first entity.

12. The one or more computer-readable media of claim 11, wherein the operations further comprise:

for each of the one or more local client systems, determining a level of detail suitable for the updates.

13. The one or more computer-readable media of claim 1, wherein the one or more processing units are part of a local client system, and wherein the operations further comprise:

sending updates to a server system, wherein the updates indicate the one or more control values for the behavior of the first entity.

14. The one or more computer-readable media of claim 1, wherein the one or more processing units are part of a local client system, and wherein the operations further comprise:

receiving first updates, from a server system, wherein the first updates indicate control values for one or more other entities whose behavior is not controlled at the local client system; and

sending second updates, to the server system, wherein the second updates indicate control values for one or more entities whose behavior is controlled at the local client system, the one or more entities whose behavior is controlled at the local client system including the first entity.

15. The one or more computer-readable media of claim 1, wherein the operations are performed are performed as part of an iteration of a loop, and wherein the operations further comprise repeating, in one or more additional iterations of the loop, the determining to change the behavior, the selecting the second entity, the determining how to follow, and the setting the one or more control values for the behavior of the first entity.

16. The one or more computer-readable media of claim 15, wherein the second entity is different in different iterations of the loop.

17. The one or more computer-readable media of claim 1, wherein the operations further comprise:

checking an on/off setting for controlling the funnel section behavior, wherein the determining to change the behavior, the selecting the second entity, the determining how to follow, and the setting the one or more control values for the behavior of the first entity are contingent on value of the on/off setting.

18. The one or more computer-readable media of claim 1, wherein the operations further comprise, for each of one or more other entities whose behavior is controlled by the behavior engine:

repeating the determining to change the behavior, the selecting, the determining how to follow, and the setting the one or more control values.

19. In a computer system that includes one or more processing units, a method for controlling behavior of a first entity in a computer-represented environment, the behavior of the first entity being controlled by a behavior engine executable on the one or more processing units, the method comprising:

determining to change the behavior of the first entity for a funnel section in the computer-represented environment; and

as part of switching to funnel section behavior in response to the determining to change the behavior of the first entity for the funnel section in the computer-represented environment;

selecting a second entity for the first entity to follow in the funnel section;

determining how to follow the second entity; and

setting one or more control values for the behavior of the first entity based at least in part on results of the determining how to follow the second entity.

26. The method of claim 19, wherein the determining to change the behavior of the first entity includes:

detecting entry of the first entity into a designated part of the funnel section;

determining a count of entities in the designated part;

comparing the count to a threshold; and

determining that the count satisfies the threshold.

27. The method of claim 19, wherein the selecting the second entity includes:

setting a scoring function that ranks each of multiple different candidate moving paths; and

selecting one of the multiple different candidate moving paths that has a best score.

28. The method of claim 27, wherein the scoring function sets a score value for each of the multiple different candidate moving paths based on one or more of:

a penalty for an obstacle;

a penalty for deviation from an optimal moving path;

a penalty for deviation from a moving path that the first entity currently follows;

a benefit for following a candidate entity in a compact configuration;

a benefit for following a candidate entity ahead of the first entity;

a penalty selection of an entity that another entity has selected to follow; and

a benefit for following a candidate entity closest to the first entity laterally.

29. The method of claim 19, wherein the determining how to follow the second entity includes:

matching a moving path of the second entity; and

matching speed of the second entity.

30. The method of claim 19, further comprising:

checking an on/off setting for controlling the funnel section behavior, wherein the determining to change the behavior, the selecting the second entity, the determining how to follow, and the setting the one or more control values for the behavior of the first entity are contingent on value of the on/off setting.

20. A computer system comprising one or more processing units and memory storing computer-executable instructions for a behavior engine executable on the one or more processing units to perform operations for controlling behavior of a first entity in a computer-represented environment, the operations comprising:

determining to change the behavior of the first entity for a funnel section in the computer-represented environment; and

as part of switching to funnel section behavior in response to the determining to change the behavior of the first entity for the funnel section in the computer-represented environment;

selecting a second entity for the first entity to follow in the funnel section;

determining how to follow the second entity; and

setting one or more control values for the behavior of the first entity based at least in part on results of the determining how to follow the second entity.

21. The computer system of claim 20, wherein the determining to change the behavior of the first entity includes:

detecting entry of the first entity into a designated part of the funnel section;

determining a count of entities in the designated part;

comparing the count to a threshold; and

determining that the count satisfies the threshold.

22. The computer system of claim 20, wherein the selecting the second entity includes:

setting a scoring function that ranks each of multiple different candidate moving paths; and

selecting one of the multiple different candidate moving paths that has a best score.

23. The computer system of claim 22, wherein the scoring function sets a score value for each of the multiple different candidate moving paths based on one or more of:

a penalty for an obstacle;

a penalty for deviation from an optimal moving path;

a penalty for deviation from a moving path that the first entity currently follows;

a benefit for following a candidate entity in a compact configuration;

a benefit for following a candidate entity ahead of the first entity;

a penalty selection of an entity that another entity has selected to follow; and

a benefit for following a candidate entity closest to the first entity laterally.

24. The computer system of claim 20, wherein the determining how to follow the second entity includes:

matching a moving path of the second entity; and

matching speed of the second entity.

25. The computer system of claim 20, wherein the operations further comprise:

checking an on/off setting for controlling the funnel section behavior, wherein the determining to change the behavior, the selecting the second entity, the determining how to follow, and the setting the one or more control values for the behavior of the first entity are contingent on value of the on/off setting.