Robotics Systems

• US 20090254217A1
• Filed: 04/10/2008
• Published: 10/08/2009
• Est. Priority Date: 04/02/2008
• Status: Active Grant

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Pack, Robert T., Vale, Marshall J., Kearns, Justin H.
• Granted Patent: US 8,452,448 B2
• Time in Patent Office: Days
• US Class Current: 700/246
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Abstract

A method of controlling a robot includes running multiple applications on a processor, where each application has a robot controller and an action selection engine. Each application is in communication with at least one behavior and at least one action model of at least part of the robot. The method includes running periodic action selection cycles on each action selection engine. Each action selection cycle includes selecting a command for each action space of each action model, generating a single overall command based on the accumulated commands for each action model, and sending the overall command to the robot controller for execution on the robot.

Claims

1. A method of generating a command for a robot controller, the method comprising:

calling at least one action model having at least one action space to generate feasible commands for each action space;

calling each action model to generate an outcome for each command;

sending each outcome to at least one behavior for evaluation, each behavior providing an outcome evaluation for each outcome;

selecting a winning outcome for each action space based on the at least one outcome evaluation;

selecting the command corresponding to the winning outcome for each action space;

generating a single overall command based on the accumulated commands for each action space; and

sending the overall command to the robot controller.

2. The method of claim 1, further comprising:

obtaining a system state from the robot controller; and

informing each action model and each behavior of the system state.

3. The method of claim 1, wherein calling the action model to generate feasible commands comprises executing a heuristic search on the respective action space to find the feasible commands.

4. The method of claim 3, wherein the search comprises a hill climbing search technique to search the action space near a last known command.

5. The method of claim 3, wherein the search comprises a search technique utilizing a command history of the action space.

6. The method of claim 3, wherein the search comprises a randomized search based on a current command.

7. The method of claim 1, wherein each action model is sequentially called in a predetermined order and each action space within each action model is sequentially called in a predetermined order.

8. The method of claim 7, further comprising considering the winning outcomes of any preceding action spaces when selecting the winning outcome for each action space.

9. The method of claim 1, wherein the outcome evaluations are weighted according to weights associated with each behavior.

10. The method of claim 1, further comprising generating an overall outcome for the overall command, and sending the overall outcome to each behavior as feedback.

11. A method of controlling a robot, the method comprising:

running multiple applications on a processor, each application having a robot controller and an action selection engine, each application being in communication with at least one behavior and at least one action model of at least part of the robot; and

running periodic action selection cycles on each action selection engine, each action selection cycle comprising;

selecting a command for each action space of each action model;

generating a single overall command based on the accumulated commands for each action model; and

sending the overall command to the robot controller for execution on the robot.

12. The method of claim 11, wherein the action selection cycle further comprises:

obtaining a system state from the robot controller;

informing each action model and each behavior of the system state; and

informing each action model and each behavior of the start of the action selection cycle.

13. The method of claim 11, wherein selecting a command for each action space comprises:

calling the corresponding action model to generate feasible commands for the action space;

calling the corresponding action model to generate outcomes for the feasible commands; and

calling each behavior to evaluate and provide an outcome evaluation for each outcome;

accumulating the outcome evaluations of each behavior;

selecting a winning outcome for the action space; and

selecting the command corresponding to the winning outcome.

14. The method of claim 13, wherein calling the action model to generate feasible commands for the action space comprises executing a heuristic search on the action space to find the feasible commands.

15. The method of claim 14, wherein the search comprises a hill climbing search technique to search the action space near a last known command.

16. The method of claim 14, wherein the search comprises a search technique utilizing a command history of the action space.

17. The method of claim 14, wherein the search comprises a randomized search based on a current command.

18. The method of claim 13, wherein each action model is sequentially called in a predetermined order and each action space within each action model is sequentially called in a predetermined order.

19. The method of claim 18, further comprising considering the winning outcomes of any preceding action spaces when selecting the winning outcome for each action space.

20. The method of claim 13, wherein the outcome evaluations are weighted according to weights associated with each behavior.

21. The method of claim 11, further comprising generating an overall outcome for the overall command, and sending the overall outcome to each behavior as feedback.

22. The method of claim 11, further comprising implementing an application priority policy that determines which application has exclusive control of resources of the robot required by that application at a given time.

23. A robotics system comprising:

multiple robot resources;

a control arbiter for each robot resource, the control arbiter controlling its associated robot resource;

multiple applications in communication with the control arbiters, each application comprising;

a robot controller in communication with the control arbiters;

an action selection engine in communication with robot controller, the action selection engine periodically executing an action selection cycle to generate an overall command which is sent to the robot controller for execution on the robot resources;

at least one behavior in communication with the action selection engine; and

at least one action model in communication with the action selection engine, each action model modeling at least one of the robot resources and having at least one action space; and

a robot manager in communication with the applications and the control arbiters, the robot manager implementing an application priority policy for determining which application has exclusive control of any one or more of the robot resources at a given time;

wherein the action selection cycle comprises;

selecting a command for each action space of each action model;

generating the single overall command based on the accumulated commands for each action model; and

sending the overall command to the robot controller.

24. The robotics system of claim 23, wherein each action model is independently removable from the robotics system and communicates with the action selection engine through an action model application programming interface.

25. The robotics system of claim 24, wherein the action model application programming interface comprises a get action spaces function configured to provide a resource and state space structure of the action model.

26. The robotics system of claim 24, wherein the action model application programming interface comprises:

a begin cycle function configured to begin the action selection cycle;

a generate commands function configured to generate commands for a given action space;

a simulate command function configured to simulate outcomes for given commands; and

a notify command function configured to notify the action model of a command chosen for a particular action space.

27. The robotics system of claim 26, wherein the action model application programming interface further comprises an event handler function configured to handle events within the robotics system.

28. The robotics system of claim 23, wherein each behavior is independently removable from the robotics system and communicates with the action selection engine through a behavior application programming interface.

29. The robotics system of claim 28, wherein the behavior application programming interface comprises:

an initialization function configured to initialize the behavior for use;

a begin cycle function configured to begin the action selection cycle;

an evaluate function configured to evaluate a collection of outcomes; and

an end cycle function configured to notify the respective behavior that the action selection cycle is complete.

30. The robotics system of claim 23, wherein the action selection cycle further comprises:

obtaining a system state from the robot controller;

informing each action model and each behavior of the system state; and

informing each action model and each behavior of the start of the action selection cycle.

31. The robotics system of claim 23, wherein selecting a command for each action space comprises:

calling the corresponding action model to generate feasible commands for the action space;

calling the corresponding action model to generate outcomes for the feasible commands;

calling each behavior to evaluate and provide an outcome evaluation for each outcome;

accumulating the outcome evaluations of each behavior;

selecting a winning outcome for the action space; and

selecting the command corresponding to the winning outcome.

32. The robotics system of claim 23, further comprising a publish-subscribe system configured to provide asynchronous messaging between each robot controller, each control arbiter, and the robot manager.

33. A robotics system comprising:

multiple robot resources;

a control arbiter for each robot resource, the control arbiter controlling its associated robot resource;

multiple applications in communication with the control arbiters, each application comprising;

a robot controller in communication with the control arbiters; and

an action selection engine in communication with the robot controller;

a robot manager in communication with the applications and the control arbiters,

the robot manager implementing an application priority policy for determining which application has exclusive control of robot resources required by that application at a given time;

wherein the action selection engine executes a heuristic search on each action space of each action model, which models one or more of the robot resources, to identify feasible commands, the action model providing an outcome for each command;

wherein the action selection engine selects one of the commands for each action space based on the outcome evaluations provided by each behavior in communication with the action selection engine; and

wherein the action selection engine generates an overall command for execution by the robot controller on the robot resources, through the control arbiters, based on the commands selected for each action space.

34. The robotics system of claim 33, wherein the action selection engine accumulates the outcome evaluations for each action space and selects a winning outcome for each action space, the action selection engine selecting a command corresponding to the winning outcome for each action space.

35. The robotics system of claim 33, wherein the action model provides the heuristic search.

36. The robotics system of claim 33, wherein the action selection engine sequentially processes each action model in a predetermined order and each action space within each action model in a predetermined order.

37. The robotics system of claim 36, wherein the action selection engine select a command for each action space by selecting a corresponding winning outcome based on the outcome evaluations.

38. The robotics system of claim 37, wherein the action selection engine uses the winning outcomes of any preceding action spaces when selecting the winning outcome for each action space.

39. The robotics system of claim 37, wherein the outcome evaluations are weighted according to weights associated with each behavior.

40. The robotics system of claim 33, wherein the action selection engine generates an overall outcome for the overall command and sends the overall outcome to each behavior as feedback.

41. A action selection system for robotics control, the action selection system comprising:

one or more action models, each action model comprising;

at least one action space model defining a simulated state propagation for commands for a physical resource;

a command generating routine that generates a predetermined limited number of feasible commands for the physical resource; and

a command simulating routine that generates simulated outcomes, each simulated outcome corresponding to one feasible command, using a simulated state propagation of a corresponding action space model;

one or more behaviors, each behavior comprising;

a routine for collecting sensor data; and

a routine assigning scores to simulated outcomes using an evaluation routine that considers sensor data, current resource state data, and predetermined goals associated with the behavior; and

one or more action selection engines, each action selection engine comprising a routine for sequentially

obtaining simulated outcomes from each action space model of each action model;

then

providing the simulated outcomes to each behavior for assigning scores;

then

weighting the scores according to a predetermined weighting among behaviors;

then

comparing the weighted scores to determine one winning outcome for each action space model; and

then

sending the one feasible command corresponding to the one winning outcome for each action space model to the physical resource corresponding to that one feasible command, one winning outcome, and one action space model.

42. The action selection system of claim 41, wherein the command generating routine generates commands throughout the action space model, and the command simulating routine generates simulated outcomes from commands distributed throughout the action space model.

43. The action selection system of claim 42, wherein the command generating routine generates random commands throughout the action space model.

44. The action selection system of claim 41, wherein the command generating routine generates commands in proximity to a current command in the action space model, and the command simulating routine generates simulated outcomes from commands distributed in proximity to a current command in the action space model.

45. The action selection system of claim 44, wherein the command generating routine generates random commands in proximity to a current command in the action space model.

46. The action selection system of claim 41, wherein the command generating routine generates commands in proximity to one or more previous commands in the action space model and the command simulating routine generates simulated outcomes from commands distributed in proximity to one or more previous commands in the action space model.

47. The action selection system of claim 46, wherein the command generating routine generates random commands in proximity to one or more previous commands in the action space model.

48. An action selection engine for robotics control, the action selection engine comprising:

a routine for sequentially

(i) obtaining simulated outcomes from an action space model of an action model associated with the action selection engine, associated action models being characterized by;

at least one action space model defining a simulated state propagation for commands for a physical resource;

a command generating routine that generates a predetermined limited number of feasible commands for the physical resource; and

a command simulating routine that generates simulated outcomes, each simulated outcome corresponding to one feasible command, using a simulated state propagation of a corresponding action space model;

(ii) providing the simulated outcomes to behaviors associated with the action selection engine, associated behaviors being characterized by

a routine for collecting sensor data; and

a routine assigning scores to simulated outcomes using an evaluation routine that considers sensor data, current resource state data, and predetermined goals associated with the behavior;

(iii) weighting the scores according to a predetermined weighting among behaviors;

then

(iv) comparing the weighted scores to determine one winning outcome for each action space model; and

then

(v) sending the one feasible command corresponding to the one winning outcome for each action space model to the physical resource corresponding to that one feasible command, one winning outcome, and one action space model.

49. A behavior for robotics control, the behavior comprising:

(i) a routine that collects sensor data; and

(ii) a routine that receives a set of simulated outcomes from a connected object, assigns scores to the simulated outcomes using an evaluation routine that considers sensor data, current resource state data, a cascade of previously determined winning outcomes from earlier cycles, and predetermined goals associated with the behavior, wherein each simulated outcome in the set was provided as a simulated state propagation corresponding to one feasible command for a physical resource.

50. The behavior of claim 49, further comprising a behavior application programming interface comprising:

an initialization function configured to initialize the behavior for use;

a begin cycle function configured to begin an action selection cycle;

an evaluate function configured to evaluate the set of outcomes; and

an end cycle function configured to notify the respective behavior that the action selection cycle is complete.

51. An action model for robotics control, the action model comprising:

(i) at least one action space model defining a simulated state propagation for commands for a physical resource;

(ii) a command generating routine that generates a predetermined limited number of feasible commands for the physical resource;

(iii) a command simulating routine that generates simulated outcomes, each simulated outcome corresponding to one feasible command, using a simulated state propagation of a corresponding action space model; and

(iv) a routine that responds to messages from a connected object to provide at least a set of simulated outcomes for each action space model of each action model.

52. The action model of claim 51, further comprising an action model application programming interface comprising:

a begin cycle function configured to begin an action selection cycle;

a generate commands function configured to generate commands for a given action space;

a simulate command function configured to simulate outcomes for given commands; and

a notify command function configured to notify the action model of a command chosen for a particular action space.

53. The action model of claim 52, wherein the action model application programming interface further comprises an event handler function configured to handle events.