Interactive systems employing robotic companions

• US 20090055019A1
• Filed: 05/08/2008
• Published: 02/26/2009
• Est. Priority Date: 05/08/2007
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Lee, Jun Ki, Maymin, Allan Z., Cheung, Iris M., Knight, Heather, Toscano, Robert L., Stiehl, Walter Dan, Breazeal, Cynthia
• Granted Patent: US 8,909,370 B2
• Time in Patent Office: Days
• US Class Current: 700/249
• CPC Class Codes: A63H 2200/00 Computerized interactive toys, e.g. dolls
• CPC Class Codes: B25J 13/00 Controls for manipulators programme controls B25J9/16; control in general G05
• CPC Class Codes: B25J 13/081 Touching devices, e.g. pressure-sensitive
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: G05B 2219/33051 BBC behavior based control, stand alone module, cognitive, independent agent
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Abstract

An interactive system for interacting with a sentient being. The system includes a robotic companion of which the sentient being may be a user and an entity which employs the robot as a participant in an activity involving the user. The robotic companion responds to inputs from an environment that includes the user during the activity. The robotic companion is capable of social and affective behavior either under control of the entity or in response to the environment. The entity may provide an interface by which an operator may control the robotic companion. Example applications for the interactive system include as a system for communicating with patients that have difficulties communicating verbally, a system for teaching remotely-located students or students with communication difficulties, a system for facilitating social interaction between a remotely-located relative and a child, and systems in which the user and the robot interact with an entity such as a smart book. Also disclosed are control interfaces for the robotic companion, techniques for rendering the robotic companion sensitive to touch and responding to those touches, and techniques for providing quiet, back-drivable motion to components of the robotic companion.

Claims

1. An interactive system for interacting with a sentient being comprising:

a robot of which the sentient being may be a user; and

an entity which employs the robot as a participant in an activity involving the user,

the robot responding to inputs received from an environment that includes the user and inputs received from the entity during the activity.

2. The interactive system set forth in claim 1 wherein:

the robot receives inputs from the environment through user interaction with the robot.

3. The interactive system set forth in claim 2 wherein:

the user interaction with the robot includes direct manipulation of the robot by the user.

4. The interactive system set forth in claim 2 wherein:

the interactive system further includes a user control interface for the user; and

the user interaction with the robot includes manipulation of the robot using the user control interface.

5. The interactive system set forth in claim 4 wherein:

the user control interface is wearable by the user.

6. The interactive system set forth in claim 1 wherein:

the robot further provides information about the activity to the entity.

7. The interactive system set forth in claim 6 wherein:

the entity uses the provided information in employing the robot.

8. The interactive system set forth in claim 6 wherein:

the entity uses the provided information to evaluate the user of the robot.

9. The interactive system set forth in claim 8 wherein:

the entity further uses the provided information in employing the robot.

10. The interactive system set forth in claim 1 wherein:

the entity operates during the activity substantially without human intervention.

11. The interactive system set forth in claim 1 wherein:

the entity is controlled at least in part during the activity by a human operator.

12. The interactive system set forth in claim 1 wherein:

The entity includes an operator interface whereby a human operator determines how the robot is participating in the activity.

13. The interactive system set forth in claim 12 wherein

the robot includes a video system and

the operator interface comprises:

a panorama produced by the video system which shows the environment; and

a detailed view produced by the video system of a location of interest in the environment.

14. The interactive system set forth in claim 13 wherein

the robot further includes a motion measurement system;

and

the video system responds to the motion measurement system by counteracting motion of the robot in producing the detailed view.

15. The interactive system set forth in claim 13 wherein:

the entity is controlled at least in part by the operator using the operator interface;

the operator employs the operator interface to select the location of interest, the video system responding to the selection by providing the detailed view.

16. The interactive system set forth in claim 12 wherein

the robot includes a motion measurement system;

and

the operator interface displays a representation of the robot'"'"'s motion.

17. The interactive system set forth in claim 12 wherein:

the robot includes a motion measurement system; and

the operator interface displays a representation of the robot'"'"'s orientation.

18. The interactive system set forth in claim 12 wherein:

the entity is controlled at least in part by the operator using the operator interface;

the operator interface includes a list of robot behaviors;

the operator selects a robot behavior from the list; and

the entity causes the robot to perform the robot behavior.

19. The interactive system set forth in claim 12 wherein:

the entity is controlled at least in part by the operator using the operator interface;

the operator interface includes a field for input of text;

the robot includes an audio output device; and

when the human operator inputs the text to the field, the entity causes the robot to output speech corresponding to the text from the audio output device.

20. The interactive system set forth in claim 11 wherein:

the entity includes an interface that is wearable by the human operator, the human operator using the wearable interface to control the entity.

21. The interactive system set forth in claim 11 wherein:

the interactive system further includes an operator robot; and

the operator controls the entity by directly manipulating the operator robot.

22. The interactive system set forth in claim 11 wherein:

the human operator is located remotely to the robot and the user of the robot.

23. The interactive system set forth in claim 11 wherein:

the human operator determines a degree to which the robot is controlled by the inputs received from the user through user interaction and by the inputs received from the entity.

24. The interactive system set forth in claim 1 wherein:

the user of the robot is under the care of a care giver;

the activity is care of the user; and

the entity is controlled by the care giver.

25. The interactive system set forth in claim 1 wherein:

the user of the robot has a relationship with a remotely-located person;

the activity is related to the relationship, and

the entity is controlled by the remotely-located person.

26. A robot for communication between a sentient being which is a user of the robot and a human operator of the robot,

the robot comprising:

a sensor that responds to a manipulation of the robot by the user by providing a signal to a controller operated by the operator; and

an actuator that responds to a control signal from the controller.

27. The robot set forth in claim 26 further comprising:

a local controller that responds to the sensor by providing a local control signal to which the actuator responds.

28. The robot set forth in claim 27 wherein:

the robot includes an operator interface for the controller; and

the operator employs the operator interface to determine which of the local control signal and the control signal the actuator responds to.

29. A robot which is employed by an entity as a participant in an activity involving a sentient being, the sentient being a user of the robot and the robot being characterized in that:

the robot responds to inputs from the entity and from the sentient being; and

in responding to an input from the sentient being, the robot carries out a behavior which the sentient being regards as affective and/or social.

30. A graphical user interface for a controller of a robot, the robot having an operator and interacting with a sentient being that is a user of the robot and the graphical user interface comprising:

a display that displays a current status of the robot; and

a control interface whereby the operator indicates a behavior of the robot.

31. The graphical user interface set forth in claim 30 wherein:

the robot includes a video camera; and

the display includes a panorama of the robot'"'"'s environment and a specification of a location in the panorama at which the video camera is aimed.

32. The graphical user interface set forth in claim 30 wherein:

the robot includes a motion measurement system which measures motion of the robot, and

the display includes a representation of the robot which shows a motion of the robot which is currently indicated by measurements from the motion measurement system.

33. The graphical user interface set forth in claim 30 wherein:

the robot includes a motion measurement system which measures motion of the robot; and

the display includes a representation of the robot which shows an orientation of the robot which is currently indicated by measurements from the motion measurement system.

34. The graphical user interface set forth in claim 30 wherein:

the control interface includes a list of behaviors of the robot in the display; and

the operator selects a behavior from the list.

35. The graphical user interface set forth in claim 30 wherein:

the control interface includes a sensor for sensing a gesture of the operator and interprets the sensed gesture as indicating a behavior of the robot.

36. The graphical user interface set forth in claim 30 wherein:

the control interface includes an operator robot; and

the operator indicates the behavior of the robot by directly manipulating the operator robot.

37. The graphical user interface set forth in claim 30 wherein:

a further behavior of the robot is determined by interactions between the user and the robot; and

the operator further indicates in the control interface a degree to which the robot'"'"'s behavior is determined by the control interface and the interactions between the user and the robot.

38. A system for rendering an object responsive to touch, the system comprising:

a first set of sensors that sense pressure at first points on the object;

a second set of sensors that senses proximity to the object at second points on the object;

a third set of sensors which senses temperatures at third points on the object; and

a processor which uses inputs from the sensors to determine a kind of touch that is being applied to the object.

39. The system for rendering an object responsive to touch set forth in claim 38 further comprising:

calibration devices that calibrate the sensors in the sets thereof, the calibration devices being settable in response to digital values; and

a calibration controller that provides the digital values to the to the sensors, the digital values being determined by an autocalibration routine executed by the calibration controller.

40. The system for rendering an object responsive to touch set forth in claim 38 wherein:

the object is a robot that includes an actuator that actuates a behavior of the robot; and

having determined the kind of touch, the processor causes the actuator to actuate a behavior of the robot in response thereto.

41. The system for rendering an object responsive to touch set forth in claim 40 wherein:

the behavior includes a plurality of actions of the robot.

42. The system for rendering an object responsive to touch set forth in claim 40 wherein:

a source of the touch is a user of the robot who is directly manipulating the robot; and

the behavior appears to the user to be affective.

43. The system for rendering an object responsive to touch set forth in claim 40 wherein:

a source of the touch is a user of the robot who is directly manipulating the robot; and

the behavior appears to the user to be social.

44. Apparatus for rendering an object sensitive to a touch by a user of the object,

the apparatus comprising:

in the object,

an air bladder; and

a pressure sensor that is responsive to changes in pressure of the air in the air bladder; and

a processor that receives a pressure indication from the pressure sensor and provides a pressure signal to apparatus that is external to the object,

whereby the pressure signal indicates whether the user is touching the object.

45. The apparatus for rendering an object sensitive to a touch set forth in claim 44 further comprising:

a plurality of the air bladders;

a plurality of the pressure sensors; and

a location indication indicating a location of an air bladder,

the processor receiving the pressure indication and the location indication and providing a location signal for a bladder in addition to the pressure signal for the bladder.

46. The apparatus for rendering an object sensitive to a touch set forth in claim 45 wherein:

the object has a humanoid form;

the user of the object is suffering pain at a particular location in the user'"'"'s body; and

the user of the object employs the object to indicate the particular location and an intensity of the pain, the user indicating the location by touching a corresponding location on the object and indicating the intensity by an intensity with which the user touches the corresponding location.

47. Apparatus for moving a component of an object and permitting the component also to be moved by a user of the object,

the apparatus comprising:

a source of rotary motion for moving the component;

a friction-driven first reduction stage for reducing the speed of the rotary motion; and

a non-slipping second reduction stage that is coupled to the component, the second reduction stage further reducing the speed of the rotary motion and providing a high-torque response to movement of the component by the user.

48. The apparatus for moving the component of the object set forth in claim 47 further comprising:

a controller for providing a motion control input to the source of rotary motion; and

a motion-measuring device coupled to the component for measuring motion of the component, the controller receiving a measurement of the amount of motion indicated by the motion measuring device.

49. The apparatus for moving the component of the object set forth in claim 48 further comprising:

a control interface to which the controller responds by producing the control input, the control interface also receiving the measurement of the amount of motion.