ROBOT CONTROL USING GESTURES

• US 20180001480A1
• Filed: 06/20/2017
• Published: 01/04/2018
• Est. Priority Date: 06/30/2016
• Status: Active Grant

• Current Assignee: Beijing Airlango Technology Company Limited
• Original Assignee: Beijing Airlango Technology Company Limited
• Inventors: MAO, Yinian, LIU, Xinmin
• Granted Patent: US 10,710,244 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/23021 Gesture programming, camera sees hand, displays it on screen, grasp buttons
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A method and a device for operating a robot are provided. According to an example of the method, information of a first gesture is acquired from a group of gestures of an operator, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions. A first operation instruction from the group of operation instructions is obtained based on the acquired information of the first gesture, the first operation corresponding to the first gesture. The first operation instruction is executed.

Claims

1. A method of operating a robot, comprising:

acquiring information of a first gesture from a group of gestures of an operator, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtaining, based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

executing the first operation instruction.

2. The method according to claim 1, wherein acquiring the information of the first gesture of the operator comprises:

capturing an image using a camera;

identifying a Region Of Interest (ROI) from the captured image;

determining whether the ROI indicates the operator; and

acquiring, based on the captured image, the information of the first gesture if the ROI indicates the operator.

3. The method according to claim 2, wherein determining whether the ROI indicates the operator comprises:

calculating a similarity between face feature information of the ROI and pre-configured face feature information of the operator; and

determining that the ROI indicates the operator if the calculated similarity is greater than a predetermined threshold.

4. The method according to claim 2, wherein the image is a first image, the method further comprises capturing a second image using the camera, the determining whether the ROI indicates the operator comprises:

acquiring, based on the second image, information of a second gesture;

determining whether the second gesture matches a pre-configured gesture password; and

determining that the ROI of the first image indicates the operator if the second gesture matches the pre-configured gesture password.

5. The method according to claim 4, wherein the pre-configured gesture password is associated with a posture or gesture of the operator, the method further comprises:

executing the first operation instruction from the group of operation instructions if the second gesture matches the pre-configured gesture password.

6. The method according to claim 2, wherein acquiring the information of the first gesture comprises:

identifying, from the captured image, a region associated with the operator;

analyzing the identified region to obtain a first feature vector;

calculating, for each gesture from the group of gestures, a similarity between the first feature vector and a feature vector of that gesture; and

selecting the gesture having the highest similarity as the first gesture to be acquired.

7. The method according to claim 1, wherein

the first gesture is a static gesture;

and

the information of the first gesture comprises information of a status of a limb.

8. The method according to claim 1, wherein

the first gesture is a dynamic gesture;

and

the information of the first gesture comprises information of a movement of a limb.

9. The method according to claim 1, wherein the group of operation instructions comprise an action instruction, a photographing instruction, an automatic-charging instruction, a return-to-origin instruction, a take-off instruction, and a landing instruction.

10. A robot, comprising:

a processor; and

a non-transitory storage medium storing machine-executable instructions,

wherein by executing the machine-executable instructions, the processor is caused to;

acquire information of a first gesture from a group of gestures of an operator, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtain, based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

execute the first operation instruction.

11. The robot according to claim 10, wherein when the processor is caused to acquire the information of the first gesture of the operator, the machine-executable instructions further cause the processor to:

capture an image using a camera;

identify a Region Of Interest (ROI) from the captured image;

determine whether the ROI indicates the operator; and

acquire, based on the captured image, the information of the first gesture if the ROI indicates the operator.

12. The robot according to claim 11, wherein when the processor is caused to determine whether the ROI indicates the operator, the machine-executable instructions further cause the processor to:

calculate a similarity between face feature information of the ROI and pre-configured face feature information of the operator; and

determine that the ROI indicates the operator if the calculated similarity is greater than a predetermined threshold.

13. The robot according to claim 11, wherein the image is a first image, when the processor is caused to determine whether the ROI indicates the operator, the machine-executable instructions further cause the processor to:

capture a second image using the camera;

acquire, based on the second image, information of a second gesture;

determine whether the second gesture matches a pre-configured gesture password; and

determine that the ROI of the first image indicates the operator if the second gesture matches the pre-configured gesture password.

14. The robot according to claim 13, wherein the pre-configured gesture password indicates a posture or gesture of the operator, the machine-executable instructions further cause the processor to:

execute the first operation instruction from the group of operation instructions if the second gesture matches the pre-configured gesture password.

15. The robot according to claim 11, wherein when the processor is caused to acquire the information of the first gesture, the machine-executable instructions further cause the processor to:

identify, from the captured image, a region associated with the operator;

analyze the identified region to obtain a first feature vector;

calculate, for each gesture from the group of gestures, a similarity between the first feature vector and a feature vector of that gesture; and

select the gesture having the highest similarity as the first gesture to be acquired.

16. The robot according to claim 10, wherein

the first gesture is a static gesture;

and

the information of the first gesture comprises information of a status of a limb.

17. The robot according to claim 10, wherein

the first gesture is a dynamic gesture;

and

the information of the first gesture comprises information of a movement of a limb.

18. The robot according to claim 10, wherein the group of operation instructions comprise an action instruction, a photographing instruction, an automatic-charging instruction, a return-to-origin instruction, a take-off instruction, and a landing instruction.

19. An apparatus for operating the robot as claimed in claim 10, comprising:

a processor; and

a non-transitory storage medium storing machine-executable instructions,

wherein by executing the machine-executable instructions, the processor is caused to;

receive an image captured using a camera of the robot;

acquire, based on the received image, information of a first gesture from a group of gestures of an operator, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtain, based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

transmit the first operation instruction to the robot such that the robot executes the first operation instruction.

20. A method of operating a robot, comprising:

receiving, at a server, an image captured using a camera of the robot;

acquiring, at the server and based on the received image, information of a first gesture from a group of gestures of an operator, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtaining, at the server and based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

transmitting, from the server to the robot, the first operation instruction such that the robot executes the first operation instruction.