Natural machine interface system

• US 9,753,453 B2
• Filed: 07/09/2013
• Issued: 09/05/2017
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: Deep Learning Robotics LTD
• Original Assignee: Deep Learning Robotics LTD
• Inventors: Benaim, Carlos, Reiner, Miriam
• Primary Examiner(s): OH, HARRY Y
• Application Number:
  US13/937,397
• Publication Number:
  US 20140012415A1
• Time in Patent Office: 1,519 Days
• Field of Search: 700248
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A method for defining a robotic machine task. The method comprises a) collecting a sequence of a plurality of images showing at least one demonstrator performing at least one manipulation of at least one object, b) performing an analysis of said sequence of a plurality of images to identify demonstrator body parts manipulating said at least one object and said at least one manipulation of said at least one object, c) determining at least one robotic machine movement to perform said task, and d) generating at least one motion command for instructing said robotic machine to perform said task.

Claims

1. A method for defining a robotic machine task, comprising:

using at least one processor to execute code instructions for;

collecting a sequence of a plurality of images captured by at least one sensor showing at least one demonstrator performing a task and operating in a work environment of said performed task, wherein said task comprises at least one manipulation of at least one object;

performing an analysis of said sequence of a plurality of images to identify demonstrator body parts manipulating said at least one object and to identify changes to said at least one object by said at least one manipulation, wherein said changes to said at least one object are at least one of changes of relative positions, changes of orientations, changes of assembly, changes of content and changes of shape;

determining according to said analysis at least one robotic machine movement to perform said task;

calculating a modified task based on said task, by applying at least one time dimension transformation on said at least one robotic machine movement, said at least one time dimension transformation is a member of a group consisting of accelerated pace of said at least one robotic machine movement, decelerated pace of said at least one robotic machine movement and performing said at least one robotic machine movement in reversed manner; and

generating at least one motion command for instructing said robotic machine to perform said modified task.

2. The method of claim 1, wherein said sequence of a plurality of images is captured by at least one depth map sensor.

3. The method of claim 1, wherein said sequence of a plurality of images is captured by at least one digital camera.

4. The method of claim 1, wherein said modified task is further calculated by applying at least one space dimension transformation on said at least one robotic machine movement, said at least one space dimension transformation is a member of a group consisting of scaling of an extent of said at least one robotic machine movement and rotation of a direction of said at least one robotic machine movement.

5. The method of claim 1, wherein said motion commands are adjusted to take into account robotic machine capabilities of different types, makes, and models, by processing said at least one robotic machine movement to be within said robotic machine capabilities.

6. The method of claim 1, wherein said demonstrator is one of at least one of a human, an animal and a machine.

12. The method of claim 1, wherein said at least one processor is further used to execute code instructions for recording a plurality of three dimensional positions in space associated with said demonstrator body parts manipulating said at least one object and with said changes to said at least one object;

and selecting a type of movement path between said plurality of three dimensional positions in space for said at least one robotic machine movement, wherein said type of movement path is at least one of a straight line, a circle and an ellipse and wherein said selecting is done either automatically or by operator guidance.

13. The method of claim 1 wherein said at least one processor is further used to execute code instructions for identifying a first gesture performed by said at least one demonstrator, wherein said first gesture signals a command to start a learning mode.

14. The method of claim 13 wherein said at least one processor is further used to execute code instructions for identifying a second gesture performed by said at least one demonstrator, wherein said second gesture signals a command to stop said learning mode.

7. A method for communications between a controller associated with a robotic machine and at least one controller associated with at least one other robotic machine comprising:

using at least one processor included in a controller associated with a robotic machine to execute code instructions for;

receiving at least one sequence of a plurality of images captured by at least one sensor associated with said robotic machine, said at least one sequence of a plurality of images depicts one other robotic machine performing at least one gesture by moving at least one robotic machine part of said one other robotic machine;

performing an analysis of said sequence of a plurality of images to identify said at least one gesture, by comparing said sequence of a plurality of images to a plurality of images of previously demonstrated plurality of gestures;

identifying at least one command associated with said at least one gesture, wherein said at least one command is for executing a pre-defined task which is different than said at least one gesture; and

instructing said robotic machine to execute said pre-defined task.

8. The method of claim 7, wherein said at least one sequence of a plurality of images is captured by at least one depth map sensor.

9. The method of claim 7, wherein said at least one sequence of a plurality of images is captured by at least one digital camera.

10. The method of claim 7, wherein said at least one command comprises associating a subset of the identified at least one robotic machine part movement with another at least one command.

11. The method of claim 7, wherein said at least one command to be executed by said robotic machine is for assigning at least one message from said robotic machine to a human operator, by movement of at least one robotic machine part, of said robotic machine to mimic intuitively understood human body language.