TRAINING AND OPERATING INDUSTRIAL ROBOTS

• US 20130345873A1
• Filed: 09/17/2012
• Published: 12/26/2013
• Est. Priority Date: 06/21/2012
• Status: Active Grant

• Current Assignee: Rethink Robotics
• Original Assignee: Rethink Robotics
• Inventors: Blumberg, Bruce, Brooks, Rodney, Buehler, Christopher J., Dye, Noelle, Sussman, Michael, Deegan, Patrick A., DiCicco, Matthew, Ens, Gerry, Linder, Natan, Siracusa, Michael, Williamson, Matthew M.
• Granted Patent: US 8,958,912 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 13/084 Tactile sensors in general G01L5/16, G01L5/22
• CPC Class Codes: B25J 9/0087 Dual arms double SCARA arms B25J9/043
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1682 Dual arm manipulator; Coordination of several manipulators
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/42 Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
• CPC Class Codes: G05B 19/423 Teaching successive positions by walk-through, i.e. the tool head or end effector being grasped and guided directly, with or without servo-assistance, to follow a path
• CPC Class Codes: G05B 2219/39451 Augmented reality for robot programming
• CPC Class Codes: G05B 2219/39466 Hand, gripper, end effector of manipulator
• CPC Class Codes: G05B 2219/40116 Learn by operator observation, symbiosis, show, watch
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/194 : References adjustable by an adaptive method, e.g. learning
• CPC Class Codes: Y02P 90/80 : Management or planning
• CPC Class Codes: Y10S 901/01 : Mobile robot
• CPC Class Codes: Y10S 901/03 : Teaching system
• CPC Class Codes: Y10S 901/09 : Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/31 : Gripping jaw
• CPC Class Codes: Y10S 901/46 : Sensing device
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

Robots may manipulate objects based on sensor input about the objects and/or the environment in conjunction with data structures representing primitive tasks and, in some embodiments, objects and/or locations associated therewith. The data structures may be created by instantiating respective prototypes during training by a human trainer.

Claims

1. A robot trainable, via interactions with a human trainer, to manipulate objects, the robot comprising:

at least one appendage for manipulating an object;

a controller for operating the at least one appendage;

a computer memory for storing a library of prototypes, at least one of the prototypes defining a type of task to be performed by the appendage, the task comprising at least one of a movement of the appendage or a manipulation of the object;

a perception system for receiving physical input from the human trainer related to the task; and

a training module for creating an instance of the at least one prototype by specifying at least one attribute thereof based on the physical input; and

a task-execution module for executing the instance of the prototype via commands to the controller, thereby causing the appendage to perform the task.

2. The robot of claim 1, wherein the instance defines the task in terms of generic, spatially unspecified motions of the appendage.

3. The robot of claim 2, wherein the instance defines the task in terms of motions of the appendage relative to at least one of the object or an environment thereof.

4. The robot of claim 2, wherein the motions are defined without specifying spatial trajectories.

5. The robot of claim 1, wherein the physical input comprises at least one of mechanical input or visual input.

6. The robot of claim 1, further comprising at least one output device responsive to the training module for providing task-related feedback to the trainer.

7. The robot of claim 6, wherein the feedback comprises at least one of an indication of robot status, a request for further input, or an error alert.

8. The robot of claim 6, wherein the feedback comprises graphics overlaid onto a robot camera view.

9. The robot of claim 1, wherein the perception system comprises at least one of a pressable button, a knob, a touch-sensitive pad, a touch-sensitive cuff, or a camera.

10. The robot of claim 1, wherein the library of prototypes further comprises prototypes defining at least one of an object class, a type of equipment, or a location.

11. The robot of claim 10, wherein the training module creates an instance of the object class based on visual input about a member of the class.

12. The robot of claim 10, wherein the object subject to the manipulation is a member of a class of objects referenced by the instance.

13. The robot of claim 1, wherein the perception system is further configured to receive physical information about the object or an environment thereof.

14. The robot of claim 1, wherein the controller is configured to operate in zero-force gravity-compensated mode in response to a physical input.

15. A method of robot learning based on interactions with a human trainer, the method comprising:

receiving physical input from the trainer regarding a task to be performed by the robot;

in response to and based at least in part on the input, selecting a prototype associated with the task from a library of stored prototypes, and creating and storing, in a database, an instance of the prototype, the instance specifying at least one of a robot movement or a robot manipulation to be performed on the object.

16. The method of claim 15, further comprising specifying at least one attribute of the instance based on the physical input.

17. The method of claim 15, wherein the instance defines the task in generic, spatially unspecified terms.

18. The method of claim 15, wherein the physical input comprises at least one of mechanical input or visual input.

19. The method of claim 15, wherein receiving physical input comprises sensing physical contact with at least one of the trainer or an object.

20. The method of claim 15, wherein receiving physical input comprises tracking a position of a robot appendage as the appendage is being moved by the trainer.

21. The method of claim 15, wherein receiving the physical input comprises acquiring an image of at least one of the trainer, the object, or an environment.

22. The method of claim 15, further comprising providing feedback to the trainer.

23. The method of claim 22, wherein the feedback comprises at least one of indicating a status of the robot, requesting additional input from the trainer, or alerting the trainer to an error.

24. The method of claim 22, wherein providing feedback comprises overlaying graphics onto a robot camera view.

25. The method of claim 22, wherein the feedback comprises haptic feedback.

26. The method of claim 22, further comprising, in response to and based at least in part on the input, selecting a prototype defining an object class and instantiating the prototype so as to create a representation of the object in memory.