User interfaces for robot training

• US 8,996,174 B2
• Filed: 09/17/2012
• Issued: 03/31/2015
• Est. Priority Date: 06/21/2012
• Status: Active Grant

• Current Assignee: Rethink Robotics
• Original Assignee: Rethink Robotics
• Inventors: Brooks, Rodney, Blumberg, Bruce, Dye, Noelle, Long, Paula
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US13/621,519
• Publication Number:
  US 20130345872A1
• Time in Patent Office: 925 Days
• Field of Search: 700257-259, 700245-250, 700/264, 318/567, 318/568.1, 318/568.2, 318/568.11, 318/568.12, 318/568.16, 318/568.25, 318/568.21
• US Class Current: 700/259
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 13/084 Tactile sensors in general G01L5/16, G01L5/22
• CPC Class Codes: B25J 9/0087 Dual arms double SCARA arms B25J9/043
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1682 Dual arm manipulator; Coordination of several manipulators
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/42 Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
• CPC Class Codes: G05B 19/423 Teaching successive positions by walk-through, i.e. the tool head or end effector being grasped and guided directly, with or without servo-assistance, to follow a path
• CPC Class Codes: G05B 2219/39451 Augmented reality for robot programming
• CPC Class Codes: G05B 2219/39466 Hand, gripper, end effector of manipulator
• CPC Class Codes: G05B 2219/40116 Learn by operator observation, symbiosis, show, watch
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/194 : References adjustable by an adaptive method, e.g. learning
• CPC Class Codes: Y02P 90/80 : Management or planning
• CPC Class Codes: Y10S 901/01 : Mobile robot
• CPC Class Codes: Y10S 901/03 : Teaching system
• CPC Class Codes: Y10S 901/09 : Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/31 : Gripping jaw
• CPC Class Codes: Y10S 901/46 : Sensing device
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

In accordance with various embodiments, a user interface embedded into a robot facilitates robot training via direct and intuitive physical interactions.

Claims

1. A robot capable of direct interaction with a user, comprising:

a robot body comprising a torso and, attached thereto, a head and at least one appendage; and

integrated into the robot body, a user interface comprising (i) a visual display for conveying information to the user;

(ii) at least one camera for acquiring images of an environment of the robot; and

(iii) at least one mechanical input device responsive to at least one of touch or mechanical actuation; and

a processor for processing input acquired by the user interface and, based thereon, controlling the at least one appendage or the visual display, wherein the user interface is operable in a training mode whereby, based on user operation of the interface, the processor identifies and stores parameters relating to autonomous performance of a sequence of robot movements for repeated execution.

2. The robot of claim 1, wherein the user interface is configured to facilitate programming or the robot.

3. The robot of claim 1, wherein the visual display is configured to display an image captured by the at least one camera, a graphical user interface, and computer graphics showing a facial expression.

4. The robot of claim 3, wherein the visual display is configured to display the image, the graphical user interface, and the facial expression are displayed alternately.

5. The robot of claim 1, wherein the at least one mechanical input device comprises identical navigators placed on two sides of the torso and on each of the at least one appendage.

6. The robot of claim 5, wherein the navigators are collectively configured such that a series of user inputs provided via multiple of the navigators causes the same response as the series of user inputs provided via a single one of the navigators.

7. The robot of claim 5, wherein each navigator comprises a plurality of mechanically actuated elements.

8. The robot of claim 1, wherein the at least one mechanical input device comprises a plurality of mechanical input devices disposed in different locations on the robot, the processor being configured to cause the robot to turn toward an estimated location of a user in response to user manipulation of a mechanical input device and based on the location of the manipulated input device.

9. The robot of claim 8, wherein the plurality of mechanical input devices comprises touch pads disposed on shoulders of the robot.

10. The robot of claim 8, wherein the plurality of mechanical input devices comprises at least one wrist cuff attached to the at least one appendage at a distal end thereof.

11. The robot of claim 8, wherein causing the robot to turn toward the user comprises causing the visual display to face the user.

12. The robot of claim 8, wherein causing the robot to turn toward the user comprises moving one of the at least one appendages toward the user.

13. The robot of claim 1, further comprising a base to which the body is mounted, the base comprising a horizontal portion flippable between an upward orientation that contributes a first height to the base and a downward orientation that contributes a second height to the base, the first height being different from the second height.

14. A robot configured for interaction with a user, comprising:

a plurality of user input devices disposed in different locations on the robot; and

a processor for causing the robot, in response to input provided via any of the user input devices and based on a location thereof, to (i) estimate a location of the user and (ii) turn toward the estimated location.

15. A robot configured for interaction with a user, comprising:

a robot body comprising a torso and, attached thereto, a head and at least one appendage;

a plurality of identical navigators, disposed in different locations on the robot, for receiving input from a user; and

a processor for processing the input and, based thereon, controlling an action of the at least one appendage,

wherein the navigators and processor are collectively configured such that a series of user inputs provided via multiple of the navigators causes the same action as the series of user inputs provided via a single one of the navigators.