Real-time intelligent virtual characters with learning capabilities

• US 8,494,981 B2
• Filed: 06/21/2010
• Issued: 07/23/2013
• Est. Priority Date: 06/21/2010
• Status: Active Grant

• Current Assignee: Livermore Software Technology Corporation, Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Harrison, Gregory A., Wonneberger, Thomas, Smith, Jason Tr, Maynard, David S., Worden, Eric W.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Chubb, Mikayla
• Application Number:
  US12/819,574
• Publication Number:
  US 20110313955A1
• Time in Patent Office: 1,128 Days
• Field of Search: None
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]

Abstract

A system, method, and computer-readable instructions for real-time characters with learning capabilities. A plurality of rules are defined in a rules-based system, each of the rules defining a condition that determines a behavior of a virtual agent when the rule is triggered by the condition being satisfied so that upon triggering of multiple rules at the same time, each of the behaviors of the multiple rules whose conditions were satisfied are combined into a resultant behavior for the virtual agent. This resultant behavior is compared with a desired behavior to providing feedback in the form of rewards or punishments to each of the multiple rules based on their corresponding contribution to the resultant behavior as compared to the desired behavior.

Claims

1. A method for controlling a virtual character in a virtual space comprising:

defining a plurality of rules in a rules-based system, each of said rules having a part that defines a condition that determines a consequent behavior of a virtual agent when the rule is triggered by the condition being satisfied;

upon triggering of multiple rules at the same time, combining all of the consequent behaviors of the multiple rules whose conditions were satisfied into a resultant behavior for the virtual agent rather than selecting one of the consequent behaviors, so that the resultant behavior comprises a new combined operation for execution within the virtual space where each of the consequent behaviors partly contributes to the new combined operation;

comparing the resultant behavior with a desired behavior; and

providing feedback to each of the multiple rules based on a proportional amount of their corresponding contribution to the resultant behavior as compared to the desired behavior.

2. The method of claim 1 wherein the rules based system comprises a set of if-then rules that are triggered when the condition returns true.

3. The method of claim 1 wherein the method is implemented in a Learning Classifier System where multiple rules are adapted to fire at once.

4. The method of claim 3 wherein the multiple rules fire at once depending on a state of the virtual character and one or more states external to the virtual character.

5. The method of claim 1 wherein the consequent behavior is a movement in virtual space having a speed and direction.

6. The method of claim 5 wherein the speed and direction is defined by a two-dimensional or three-dimensional speed vector.

7. The method of claim 6 wherein the consequent behaviors are combined by a vector sum of the two-dimensional or three-dimensional speed vectors.

8. The method of claim 1 wherein the comparison with the desired behavior is vector based.

9. The method of claim 1 wherein the feedback is provided during a real-time epoch when the multiple rules triggered as a reward or punishment based on the rule'"'"'s corresponding contribution to the resultant behavior as compared to the desired behavior.

10. The method of claim 1 further comprising implementing the resultant behavior for the virtual character.

11. The method of claim 10 wherein the implemented resultant behavior is detected to determine its effectiveness in the virtual space when subjected to environmental influences of the virtual space so that further feedback is provided in the form of a reward or punishment.

12. The method of claim 11 wherein the further feedback is issued at the end of a real-time period before a next epoch starts.

13. The method of claim 12 further comprising a behavior reward for the actual behavior that was implemented.

14. The method of claim 1 wherein the feedback provides for a rule fitness adapted to be considered in a genetic operation with respect to how each rule contributed to the resultant behavior.

15. A non-transitory tangible computer readable medium having instructions thereon for implementing a method for controlling a virtual character in a virtual space comprising:

defining a plurality of rules in a rules-based system, each of said rules defining a condition that determines a consequent behavior of a virtual agent when the rule is triggered by the condition being satisfied;

upon triggering of multiple rules at the same time, combining all of the behaviors of the multiple rules whose conditions were satisfied into a resultant behavior for the virtual agent rather than selecting one of the consequent behaviors, so that the resultant behavior comprises a new combined operation for execution within the virtual space where each of the consequent behaviors partly contributes to the new combined operation;

comparing the resultant behavior with a desired behavior; and

providing feedback to each of the multiple rules based on a proportional amount of their corresponding contribution to the resultant behavior as compared to the desired behavior.

16. The non-transitory tangible computer readable medium of claim 15 wherein the consequent behavior is a movement in virtual space having a speed and direction defined by a two-dimensional or three-dimensional speed vector, wherein the consequent behaviors are combined by a vector sum of the two-dimensional or three-dimensional speed vectors.

17. The non-transitory tangible computer readable medium of claim 15 wherein the feedback is provided during a real-time epoch when the multiple rules triggered as a reward or punishment based on the rule'"'"'s corresponding contribution to the resultant behavior as compared to the desired behavior.

18. A virtual reality system having a virtual character in a virtual space, comprising:

a processor;

a memory comprising a plurality of rules in a rules-based system, each of said rules defining a condition that determines a consequent behavior of a virtual agent when the rule is triggered by the condition being satisfied, wherein the program instructions are executable by the processor to;

upon triggering of multiple rules at the same time, combine all of the consequent behaviors of the multiple rules whose conditions were satisfied into a resultant behavior for the virtual agent rather than selecting one of the consequent behaviors, so that the resultant behavior comprises a new combined operation for execution within the virtual space where each of the consequent behaviors partly contributes to the new combined operation;

compare the resultant behavior with a desired behavior; and

provide feedback to each of the multiple rules based on a proportional amount of their corresponding contribution to the resultant behavior as compared to the desired behavior.

19. The system of claim 18 wherein the consequent behavior is a movement in virtual space having a speed and direction defined by a two-dimensional or three-dimensional speed vector, wherein the consequent behaviors are combined by a vector sum of the two-dimensional or three-dimensional speed vectors.

20. The system of claim 18 wherein the feedback is provided during a real-time epoch when the multiple rules triggered as a reward or punishment based on the rule'"'"'s corresponding contribution to the resultant behavior as compared to the desired behavior.