System and method for instructing a device

• US 9,211,644 B1
• Filed: 10/25/2013
• Issued: 12/15/2015
• Est. Priority Date: 10/25/2013
• Status: Active Grant

• Current Assignee: Vecna Robotics, Inc.
• Original Assignee: Vecna Technologies, Inc
• Inventors: Checka, Neal
• Primary Examiner(s): Holloway, Jason
• Application Number:
  US14/062,973
• Time in Patent Office: 781 Days
• Field of Search: 901/1, 901/3, 901/47
• US Class Current: 1/1
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 13/00 Controls for manipulators programme controls B25J9/16; control in general G05
• CPC Class Codes: B25J 13/06 Control stands, e.g. consoles, switchboards
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65G 1/137 with arrangements or automatic control means for selecting which articles are to be removed
• CPC Class Codes: G05B 2219/33051 BBC behavior based control, stand alone module, cognitive, independent agent
• CPC Class Codes: G05B 2219/33056 Reinforcement learning, agent acts, receives reward, emotion, action selective
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G05B 2219/40391 Human to robot skill transfer
• CPC Class Codes: G05B 2219/40413 Robot has multisensors surrounding operator, to understand intention of operator
• CPC Class Codes: G05B 2219/40414 Man robot interface, exchange of information between operator and robot
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06N 3/00 Computing arrangements based on biological models
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: Y10S 901/06 Communication with another machine

Abstract

A system and method of instructing a device is disclosed. The system includes a signal source for providing at least one visual signal where the at least one visual signal is substantially indicative of at least one activity to be performed by the device. A visual signal capturing element captures the at least one visual signal and communicates the at least one visual signal to the device where the device interprets the at least one visual signal and performs the activity autonomously and without requiring any additional signals or other information from the signal source.

Claims

1. A system for instructing a device, comprising:

a signal source configured to provide at least one visual signal where the at least one visual signal is substantially indicative of at least one activity to be performed by the device;

a visual signal capturing element configured to capture the at least one visual signal and to communicate the at least one captured visual signal to the device, where the device is configured to interpret the at least one visual signal and perform the at least one activity autonomously and without requiring any additional signals or other information from the signal source; and

a display configured to assist in providing the visual signal, where the display is configured to show a cursor that substantially mimics in substantially real time the visual signal in a simulated environment with virtual objects, and where the cursor'"'"'s interactions with the virtual objects indicate the at least one activity to be performed.

2. The system as defined in claim 1 where the visual signal capturing element communicates the at least one visual signal to the device in a format in which the device recognizes the activity to be performed.

3. The system as defined in claim 1 further comprising at least one interpreter, where the at least one interpreter translates the at least one visual to the device in a format in which the device recognizes the activity to be performed.

4. The system as defined in claim 1 where the device is a robot.

5. The system as defined in claim 4 where the robot includes at least one actuator that performs the at least one activity.

6. The system as defined in claim 1 where the device performs the at least one activity without mimicking the visual signal exactly.

7. The system as defined in claim 1 where the device performs the activity efficiently by determining and following feasible trajectories.

8. The system as defined in claim 1 where the at least one activity to be performed by the device includes lifting, moving, and placing an object.

9. The system as defined in claim 1 where the visual signal capturing element includes at least one camera.

10. The system as defined in claim 9 where the at least one camera is capable of capturing visual signals in three dimensions.

11. The system as defined in claim 1 where the device begins to perform the activity while the visual signal is being provided by the signal source.

12. The system as defined in claim 1 where the device performs the at least one activity at a location remote from the signal source.

13. A method of instructing a device, comprising:

providing at least one visual signal by a signal source, where the at least one visual signal is substantially indicative of at least one activity to be performed by the device;

providing a visual signal capturing element configured to capture the at least one visual signal;

communicating the at least one visual signal from the visual signal capturing element to the device, where the device interprets the at least one visual signal and performs the at least one activity autonomously and without requiring any additional signals or other information from the signal source; and

providing a display configured to assist in providing the visual signal, where the display is configured to show a cursor that substantially mimics in substantially real time the visual signal in a simulated environment with virtual objects, and where the cursor'"'"'s interactions with the virtual objects indicate the at least one activity to be performed.

14. The method as defined in claim 13 where the device is a robot.

15. The method as defined in claim 13 further comprising determining and following feasible trajectories to perform the activity efficiently.