SYSTEMS AND METHODS FOR CONTROL OF ROBOTIC MANIPULATION

• US 20170341231A1
• Filed: 10/14/2016
• Published: 11/30/2017
• Est. Priority Date: 05/31/2016
• Status: Active Grant

• Current Assignee: GE Global Sourcing LLC (Westinghouse Air Brake Technologies Corporation)
• Original Assignee: General Electric Company
• Inventors: Tan, Huan, Kannan, Balajee, Gefen, Yonatan, Patrick, Romano, Al Assad, Omar, Forman, Douglas, Theurer, Charles, Lizzi, John
• Granted Patent: US 10,471,595 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39543 Recognize object and plan hand shapes in grasping movements
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05B 2219/40609 Camera to monitor end effector as well as object to be handled
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593

Abstract

A robot system is provided that includes a base, an articulable arm, a visual acquisition unit, and at least one processor. The articulable arm extends from the base and is configured to be moved toward a target. The visual acquisition unit is mounted to the arm or the base, and acquires environmental information. The at least one processor is operably coupled to the arm and the visual acquisition unit, the at least one processor configured to: generate an environmental model using the environmental information; select, from a plurality of planning schemes, using the environmental model, at least one planning scheme to translate the arm toward the target; plan movement of the arm toward the target using the selected at least one planning scheme; and control movement of the arm toward the target using the at least one selected planning scheme.

Claims

1. A robot system comprising:

a base;

an articulable arm extending from the base and configured to be moved toward a target;

a visual acquisition unit configured to be mounted to the arm or the base and to acquire environmental information corresponding to at least one of the arm or the target;

at least one processor operably coupled to the arm and the visual acquisition unit, the at least one processor configured to;

generate an environmental model using the environmental information;

select, from a plurality of planning schemes, using the environmental model, at least one planning scheme to translate the arm toward the target;

plan movement of the arm toward the target using the selected at least one planning scheme; and

control movement of the arm toward the target using the at least one selected planning scheme.

2. The robot system of claim 1, wherein at least one of the planning schemes utilizes a first coordinate system, and at least one other of the planning schemes utilizes a second coordinate system that is different from the first coordinate system.

3. The robot system of claim 2, wherein the first coordinate system is a Cartesian coordinate system, and the second coordinate system is a joint space coordinate system.

4. The robot system of claim 1, wherein the at least one processor is configured to:

control the visual acquisition unit to acquire additional environmental information during movement of the arm; and

dynamically re-plan movement of the arm using the additional environmental information.

5. The robot system of claim 4, wherein the at least one processor is configured to use a first planning scheme for an initial planned movement using the environmental information, and to use a second planning scheme that is different from the first planning scheme for a revised planned movement using the additional environmental information.

6. The robot system of claim 4, wherein the visual acquisition unit comprises an arm-mounted visual acquisition unit.

7. The robot system of claim 1, wherein the at least one processor is configured to control the movement of the arm in a series of stages, wherein the selected at least one planning scheme includes a first planning scheme for at least one of the stages and a second planning scheme that is different from the first planning scheme for at least one other of the stages.

8. The robot system of claim 1, wherein the plurality of planning schemes comprise:

a first planning scheme that utilizes linear trajectory planning in a joint space coordinate system;

a second planning scheme that utilizes linear trajectory planning in a Cartesian coordinate system; and

a third planning scheme that utilizes point-to-point trajectory planning in the joint space coordinate system.

12. The method of claim 1, further comprising:

controlling the visual acquisition unit to acquire additional environmental information during movement of the arm; and

dynamically re-planning movement of the arm using the additional environmental information.

13. The method of claim 12, further comprising:

using a first planning scheme for an initial planned movement using the environmental information; and

using a second planning scheme that is different from the first planning scheme for a revised planned movement using the additional environmental information.

14. The method of claim 12, wherein the visual acquisition unit comprises an arm-mounted visual acquisition unit.

15. The method of claim 1, wherein moving the arm toward the target comprises moving the arm in a series of stages, wherein the selected at least one planning scheme includes a first planning scheme for at least one of the stages and a second planning scheme that is different from the first planning scheme for at least one other of the stages.

9. A method for controlling a robot system, the method comprising:

acquiring, with a visual acquisition unit, environmental information corresponding to at least one of an arm or a target to which the arm is configured to be moved toward;

generating, with at least one processor, an environmental model using the environmental information;

selecting, from a plurality of planning schemes, using the environmental model, at least one planning scheme to translate the arm toward the target;

planning movement of the arm toward the target using the selected at least one planning scheme; and

moving the arm toward the target using that at least one selected planning scheme.

10. The method of claim 9, wherein at least one of the planning schemes utilizes a first coordinate system, and at least one other of the planning schemes utilizes a second coordinate system that is different from the first coordinate system.

11. The method of claim 10, wherein the first coordinate system is a Cartesian coordinate system, and the second coordinate system is a joint space coordinate system.

16. A tangible and non-transitory computer readable medium comprising one or more computer software modules configured to direct one or more processors to:

acquire, with a visual acquisition unit, environmental information corresponding to at least one of an arm or a target to which the arm is configured to be moved toward;

generate, with at least one processor, an environmental model using the environmental information;

select, from a plurality of planning schemes, using the environmental model, at least one planning scheme to translate the arm toward the target;

plan movement of the arm toward the target using the selected at least one planning scheme; and

move the arm toward the target using that at least one selected planning scheme.

17. The tangible and non-transitory computer readable medium of claim 16, wherein at least one of the planning schemes utilizes a first coordinate system, and at least one other of the planning schemes utilizes a second coordinate system that is different from the first coordinate system.

18. The tangible and non-transitory computer readable medium of claim 16, wherein the computer readable medium is further configured to direct the one or more processors to:

control the visual acquisition unit to acquire additional environmental information during movement of the arm; and

dynamically re-plan movement of the arm using the additional environmental information

19. The tangible and non-transitory computer readable medium of claim 18, wherein the computer readable medium is further configured to direct the one or more processors to use a first planning scheme for an initial planned movement using the environmental information, and use a second planning scheme that is different from the first planning scheme for a revised planned movement using the additional environmental information.

20. The tangible and non-transitory computer readable medium of claim 16, wherein the computer readable medium is further configured to direct the one or more processors to move the arm in a series of stages, wherein the selected at least one planning scheme includes a first planning scheme for at least one of the stages and a second planning scheme that is different from the first planning scheme for at least one other of the stages.