NATURAL MACHINE INTERFACE SYSTEM

• US 20140012415A1
• Filed: 07/09/2013
• Published: 01/09/2014
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: Deep Learning Robotics LTD
• Original Assignee: TECHNION RESEARCH AND DEVELOPMENT FOUNDATION, LTD. (Technion-Israel Institute Of Technology)
• Inventors: BENAIM, Carlos, REINER, Miriam
• Granted Patent: US 9,753,453 B2
• Time in Patent Office: Days
• US Class Current: 700/248
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A method for defining a robotic machine task. The method comprises a) collecting a sequence of a plurality of images showing at least one demonstrator performing at least one manipulation of at least one object, b) performing an analysis of said sequence of a plurality of images to identify demonstrator body parts manipulating said at least one object and said at least one manipulation of said at least one object, c) determining at least one robotic machine movement to perform said task, and d) generating at least one motion command for instructing said robotic machine to perform said task.

Claims

1. A method for defining a robotic machine task, comprising:

collecting a sequence of a plurality of images showing at least one demonstrator performing at least one manipulation of at least one object;

performing an analysis of said sequence of a plurality of images to identify demonstrator body parts manipulating said at least one object and said at least one manipulation of said at least one object;

determining at least one robotic machine movement to perform said task; and

generating at least one motion command for instructing said robotic machine to perform said task.

2. The method of claim 1, wherein said sequence of a plurality of images is collected by at least one depth map sensor.

3. The method of claim 1, wherein said sequence of a plurality of images is collected by at least one digital camera.

4. The method of claim 1, where said robotic machine task is to perform said object manipulation modified by any combination of scale, translation, rotation, symmetry, skew, or shear transformations in the space or time dimensions resulting in a modified object manipulation task by said robotic machine.

5. The method of claim 1, wherein said motion commands are adjusted to take into account robotic machine capabilities of different types, makes, and models.

6. The method of claim 1, wherein said demonstration is performed by any type of demonstrator:

human, animal or machine.

7. A method for communications between a human operator and at least one controller associated with at least one robotic machine comprising:

receiving at least one sequence of a plurality of images imaging a human operator performing at least one gesture by moving at least one body part;

performing an analysis of said sequence of a plurality of images to identify said at least one gesture; and

identifying an association between said at least one gesture and at least one command to be executed; and

instructing a robotic machine with said at least one command

8. The method of claim 7, wherein said at least one sequence of a plurality of images is collected by at least one depth map sensor.

9. The method of claim 7, wherein said at least one sequence of a plurality of images is collected at least one at least one digital camera.

10. The method of claim 7, wherein said at least one command to be executed comprises associating a subset of the identified at least one body part movement with another at least one command to be executed.

11. The method of claim 7, wherein said at least one command to be executed is assignment of at least one message from said at least one controller to human operator by movement of at least one robotic machine part to mimic intuitively understood human body language.

12. A method for enhancing at least one robotic machine'"'"'s design, comprising:

gathering usage data of a plurality of technical elements from at least one robotic machine while said at least one robotic machine performs at least one task;

statistically analyzing said usage data to determine statistically at least one desired parameter for each of said plurality of said technical elements; and

using said at least one desired parameter of each of said plurality of technical element usage to generate a new design defining at least one change to a current design of said at least one robotic machine.

13. The method of claim 12, wherein said at least one change is a change to robotic machine'"'"'s hardware.

14. The method of claim 12, wherein said at least one change is a change to robotic machine'"'"'s software.

15. The method of claim 12, wherein said at least one change is a change to robotic machine'"'"'s processing algorithm.

16. The method of claim 12, wherein said usage data is acquired by at least one associated controller through analysis of at least one sequence of a plurality of images collected by at least one spatial sensor.

17. The method of claim 12, wherein said usage data is acquired by at least one associated controller through robotic machine movement history.

18. The method of claim 12, wherein said usage data is software usage.

19. The method of claim 12, wherein said performance at least one desired parameter is any combination of robotic machine'"'"'s cost, maintenance cost, speed of performing task, precision of performing task, weight, dimensions, strength, power, throughput, and other unspecified production relevant parameter.

20. The method of claim 16, wherein said at least one spatial sensor is at least one depth map sensor.

21. The method of claim 16, wherein said at least one spatial sensor is at least one digital camera.

22. The method of claim 12, wherein said robotic machine design'"'"'s is software design.

23. The method of claim 12, wherein said robotic machine design'"'"'s is hardware design.

24. A method to reduce the risk of collision between at least one first robotic machine and at least one second robotic machine in the surrounding environment, comprising:

collecting scene and motion data imaging said at least one second robotic machine performing at least one robotic machine task;

performing an analysis of said scene and motion data to identify at least one location of at least one part of each of said at least one second robotic machine;

identifying at least one task being performed by said at least one second robotic machine; and

using said identified at least one location of said at least one part and said identified task to determine at least one movement limit for at least one part of said at least one first robotic machine to reduce a risk of collision between said at least one second robotic machine and said at least one first robotic machine.

25. The method of claim 24, wherein said motion data is acquired through analysis of at least one sequence of a plurality of images collected by at least one spatial sensor showing said at least one robotic machine in motion.

26. The method of claim 25, wherein said at least one spatial sensor is at least one depth map sensor.

27. The method of claim 25, wherein said at least one spatial sensor is at least one digital camera.

28. The method of claim 24, wherein additional motion data is acquired through analysis of movement history data transferred from other robotic machine'"'"'s controller.

29. The method of claim 24, wherein said identified locations includes compensation for at least one object being manipulate by said robotic machines.

30. The method of claim 24, wherein said motion data includes compensation for at least one object being manipulated by said robotic machines.

31. An apparatus associated with at least one robotic machine comprising:

at least one spatial sensor for collecting a sequence of plurality of images;

at least one processing unit which analyses a sequence of a plurality of images to identify at least one task objective, determining at least one robotic machine movement to perform, and generating at least one motion command for instructing said robotic machine to perform;

at least one processing unit which analyses a sequence of a plurality of images to identify at least one technical usage characteristic of said at least one associated robotic machine;

at least one storage unit to save said sequence of a plurality of images, said at least one processed robotic machine motion, at least one processing algorithm, said at least one motion command, and said at least one technical usage characteristic;

at least one digital communication method to communicate said at least one motion command to said at least one associated robotic machine;

at least one digital communication method to communicate said at least one technical usage characteristic to at least one statistical analysis system; and

housing for containing said at least one spatial sensor, said at least one processing unit, said storage unit, and said communication method, said housing being configured for being suitable for the environment of said robotic machine.

32. The apparatus of claim 31, wherein said at least one spatial sensor consists of at least one depth map sensor.

33. The apparatus of claim 31, wherein said at least one spatial sensor consists of at least one digital camera.

34. The apparatus of claim 31, wherein said at least one digital communication method is any digital communication method typically or atypically used in the working environment of the associated at least one robotic machine, including, but not limited to, wired Ethernet, wireless Ethernet, serial communication standards, parallel communications standards, infrared communications, wired universal serial bus, wireless universal serial bus, Bluetooth communication, or any type of typical or atypical industrial automation protocol.