Method of controlling robotic arm with human computer interaction, and terminal and system for same

• US 10,513,032 B2
• Filed: 08/04/2017
• Issued: 12/24/2019
• Est. Priority Date: 11/21/2016
• Status: Active Grant

• Current Assignee: Shenzhen XiaoR Geek Technology Co., Ltd.
• Original Assignee: Shenzhen XiaoR Geek Technology Co., Ltd.
• Inventors: Liu, Hui, Huang, Xing
• Primary Examiner(s): Holloway, Jason
• Application Number:
  US15/668,804
• Publication Number:
  US 20180141212A1
• Time in Patent Office: 872 Days
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 9/046 Revolute coordinate type
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/36168 Touchscreen
• CPC Class Codes: G05B 2219/40099 Graphical user interface for robotics, visual robot user interface
• CPC Class Codes: G05B 2219/40522 Display of workpiece, workspace, locus of robot tip in different planes, xy xz yz

Abstract

A method of controlling a robotic arm with human-computer interaction, and terminal and system for the same are provided. The method of controlling the robotic arm with human-computer interaction includes: virtualizing a robotic arm to provide a virtual robotic arm having at least two movable nodes on a screen, and designating a distal movable node of the at least two movable nodes as a target node; when the target node is triggered, responding to a drag-and-drop operation by a user and generating a moving path according to a path of the drag-and-drop operation on the target node; and generating a controlling signal for controlling a motion of the robotic arm based on the moving path, and controlling the robotic arm to move according to a motion of the virtual robotic arm based on control of the controlling signal.

Claims

1. A method of controlling a robotic arm with human-computer interaction, comprising:

virtualizing, by a processor, a robotic arm to provide a virtual robotic arm having at least two movable nodes on a screen, and designating a distal movable node of the at least two movable nodes as a target node;

when the target node is triggered by a user in the screen, responding to a drag-and-drop operation by the user, generating, by the processor, a moving path according to a path of the drag-and-drop operation on the target node; and

generating, by the processor, a controlling signal for controlling a motion of the robotic arm based on the moving path, and controlling the robotic arm to move according to a motion of the virtual robotic arm based on control of the controlling signal;

wherein the step of generating the controlling signal for controlling the motion of the robotic arm based on the moving path, and controlling the robotic arm to move according to the motion of the virtual robotic arm based on control of the controlling signal comprises;

within a set period, based on the moving path, recording a first initial angle between a first reference line and a line connecting the target node at an initial position and an adjacent node, and a first end angle between the first reference line and a line connecting the target node at an end position and the adjacent node;

calculating a first angle change of the lines connecting the adjacent node and the target node based on the first initial angle and the first end angle; and

generating a first synchronized controlling signal for controlling the motion of the robotic arm based on the first angle change, thereby allowing the robotic arm to move in synchronization with the motion of the virtual robotic arm.

2. The method according to claim 1, further comprising:

determining whether the moving path of the target node lies on a first motion path; and

when the moving path of the target node lies on the first motion path, performing the step of recording the first initial angle between the first reference line and the line connecting the target node at the initial position and the adjacent node, and the first end angle between the first reference line and the line connecting the target node at the end position and the adjacent node.

3. The method according to claim 2, wherein the at least two movable nodes include a third movable node designated as a proximal node, the movable nodes are connected in the order of proximal node-adjacent node-target node;

wherein the method further comprises;

when the moving path of the target node does not lie on the first motion path, then recording a second initial angle between a second reference line and a line connecting the adjacent node at an initial position and the proximal node, and a second end angle between the second reference line and a line connecting the adjacent node at an end position and the proximal node, and recording a finishing position of the target node;

calculating a second angle change of the lines connecting the proximal node and the adjacent node based on the second initial angle and the second end angle; and

generating a second synchronized controlling signal for controlling the motion of the robotic arm based on the second angle change and the finishing position of the target node, thereby allowing the robotic arm to move in synchronization with the motion of the virtual robotic arm.

4. A terminal for controlling a robotic arm with human-computer interaction, the terminal comprising a processor comprising:

a virtualization module for virtualizing a robotic arm to provide a virtual robotic arm having at least two movable nodes on a screen, and designating a distal movable node of the at least two movable nodes as a target node;

a response module for, when the target node is triggered by a user in the screen, responding to a drag-and-drop operation by the user, generating a moving path according to a path of the drag-and-drop operation on the target node; and

a process module for generating a controlling signal for controlling a motion of the robotic arm based on the moving path, and controlling the robotic arm to move according to a motion of the virtual robotic arm based on control of the controlling signal;

wherein the process module comprises;

a record unit for, within a set period, based on the moving path, recording a first initial angle between a first reference line and a line connecting the target node at an initial position and an adjacent node, and a first end angle between the first reference line and a line connecting the target node at an end position and the adjacent node;

a calculation unit for calculating a first angle change of the lines connecting the adjacent node and the target node based on the first initial angle and the first end angle; and

a control unit for generating a first synchronized controlling signal for controlling the motion of the robotic arm based on the first angle change, thereby allowing the robotic arm to move in synchronization with the motion of the virtual robotic arm.

5. The terminal according to claim 4, wherein the process module further comprises:

a determination unit for determining whether the moving path of the target node lies on a first motion path; and

when the moving path of the target node lies on the first motion path, the record unit records the first initial angle between the first reference line and the line connecting the target node at the initial position and the adjacent node, and the first end angle between the first reference line and the line connecting the target node at the end position and the adjacent node.

6. The terminal according to claim 5, wherein the at least two movable nodes include a third movable node designated as a proximal node, the movable nodes are connected in the order of proximal node-adjacent node-target node;

the process module is further configured for;

when the moving path of the target node does not lie on the first motion path, then recording a second initial angle between a second reference line and a line connecting the adjacent node at an initial position and the proximal node, and a second end angle between the second reference line and a line connecting the adjacent node at an end position and the proximal node, and recording a finishing position of the target node;

calculating a second angle change of the lines connecting the proximal node and the adjacent node based on the second initial angle and the second end angle; and

generating a second synchronized controlling signal for controlling the motion of the robotic arm based on the second angle change and the finishing position of the target node, thereby allowing the robotic arm to move in synchronization with the motion of the virtual robotic arm.

7. A system for controlling a robotic arm with human-computer interaction, comprising a robotic arm, a driving structure for driving a motion of the robotic arm, and a control terminal for providing a controlling signal for the driving structure, wherein the control terminal comprises a terminal comprising a processor comprising:

a virtualization module for virtualizing a robotic arm to provide a virtual robotic arm having at least two movable nodes on a screen, and designating a distal movable node of the at least two movable nodes as a target node;

a response module for, when the target node is triggered by a user in the screen, responding to a drag-and-drop operation by the user, generating a moving path according to a path of the drag-and-drop operation on the target node; and

a process module for generating a controlling signal for controlling a motion of the robotic arm based on the moving path, and controlling the robotic arm to move according to a motion of the virtual robotic arm based on control of the controlling signal;

wherein the process module comprises;

a record unit for, within a set period, based on the moving path, recording a first initial angle between a first reference line and a line connecting the target node at an initial position and an adjacent node, and a first end angle between the first reference line and a line connecting the target node at an end position and the adjacent node;

a calculation unit for calculating a first angle change of the lines connecting the adjacent node and the target node based on the first initial angle and the first end angle; and

a control unit for generating a first synchronized controlling signal for controlling the motion of the robotic arm based on the first angle change, thereby allowing the robotic arm to move in synchronization with the motion of the virtual robotic arm.

8. The system according to claim 7, wherein the process module further comprises:

a determination unit for determining whether the moving path of the target node lies on a first motion path; and

when the moving path of the target node lies on the first motion path, the record unit records the first initial angle between the first reference line and the line connecting the target node at the initial position and the adjacent node, and the first end angle between the first reference line and the line connecting the target node at the end position and the adjacent node.

9. The system according to claim 8, wherein the at least two movable nodes include a third movable node designated as a proximal node, the movable nodes are connected in the order of proximal node-adjacent node-target node;

the process module is further configured for;

when the moving path of the target node does not lie on the first motion path, then recording a second initial angle between a second reference line and a line connecting the adjacent node at an initial position and the proximal node, and a second end angle between the second reference line and a line connecting the adjacent node at an end position and the proximal node, and recording a finishing position of the target node;

calculating a second angle change of the lines connecting the proximal node and the adjacent node based on the second initial angle and the second end angle; and

generating a second synchronized controlling signal for controlling the motion of the robotic arm based on the second angle change and the finishing position of the target node, thereby allowing the robotic arm to move in synchronization with the motion of the virtual robotic arm.