Systems and methods for control of robotic manipulation

• US 10,471,595 B2
• Filed: 10/14/2016
• Issued: 11/12/2019
• Est. Priority Date: 05/31/2016
• Status: Active Grant

• Current Assignee: GE Global Sourcing LLC (Westinghouse Air Brake Technologies Corporation)
• Original Assignee: GE Global Sourcing LLC (Westinghouse Air Brake Technologies Corporation)
• Inventors: Tan, Huan, Kannan, Balajee, Gefen, Yonatan, Patrick, Romano, Al Assad, Omar, Forman, Douglas, Theurer, Charles, Lizzi, John
• Primary Examiner(s): Holloway, Jason
• Application Number:
  US15/293,905
• Publication Number:
  US 20170341231A1
• Time in Patent Office: 1,124 Days
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39543 Recognize object and plan hand shapes in grasping movements
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05B 2219/40609 Camera to monitor end effector as well as object to be handled
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593

Abstract

A robot system is provided that includes a base, an articulable arm, a visual acquisition unit, and at least one processor. The articulable arm extends from the base and is configured to be moved toward a target. The visual acquisition unit is mounted to the arm or the base, and acquires environmental information. The at least one processor is operably coupled to the arm and the visual acquisition unit, the at least one processor configured to: generate an environmental model using the environmental information; select, from a plurality of planning schemes, using the environmental model, at least one planning scheme to translate the arm toward the target; plan movement of the arm toward the target using the selected at least one planning scheme; and control movement of the arm toward the target using the at least one selected planning scheme.

Claims

1. A robot system comprising:

a base;

an articulable arm extending from the base and configured to be moved toward a target;

a visual acquisition unit configured to be mounted to the arm or the base and to acquire environmental information corresponding to at least one of the arm or the target;

at least one processor operably coupled to the arm and the visual acquisition unit, the at least one processor configured to;

generate an environmental model using the environmental information;

select, from a plurality of planning schemes, at least one planning scheme based on the environmental model generated using the environmental information to translate the arm toward the target, wherein each planning scheme is defined by at least one of path shape or path type, and wherein the at least one planning scheme selected from the plurality of planning schemes is selected to provide movement of the articulable arm within at Least one of a desired time frame or at a desired speed based on the environmental model generated using the environmental information;

plan movement of the arm toward the target using the selected at least one planning scheme; and

control movement of the arm toward the target using the at least one selected planning scheme.

2. The robot system of claim 1, wherein at least one of the planning schemes utilizes a first coordinate system, and at least one other of the planning schemes utilizes a second coordinate system that is different from the first coordinate system.

3. The robot system of claim 2, wherein the first coordinate system is a Cartesian coordinate system, and the second coordinate system is a joint space coordinate system, wherein the at least one processor is configured to select between the planning schemes based on an amount of openness determined using the environmental model generated using the environmental information.

4. The robot system of claim 1, wherein the at least one processor is configured to:

control the visual acquisition unit to acquire additional environmental information during movement of the arm; and

dynamically re-plan movement of the arm using the additional environmental information.

5. The robot system of claim 4, wherein the at least one processor is configured to use a first planning scheme for an initial planned movement using the environmental information, and to use a second planning scheme that is different from the first planning scheme for a revised planned movement using the additional environmental information.

6. The robot system of claim 4, wherein the visual acquisition unit comprises an arm-mounted visual acquisition unit.

7. The robot system of claim 1, wherein the at least one processor is configured to control the movement of the arm in a series of stages, Wherein the selected at least one planning scheme includes a first planning scheme for at least one of the stages and a second planning scheme that is different from the first planning scheme for at least one other of the stages.

8. The robot system of claim 1, wherein the plurality of planning schemes comprise:

a first planning scheme that utilizes linear trajectory planning in a joint space coordinate system;

a second planning scheme that utilizes linear trajectory planning in a Cartesian coordinate system; and

a third planning scheme that utilizes point-to-point trajectory planning in the joint space coordinate system.

12. The method of claim 1, further comprising:

controlling the visual acquisition unit to acquire additional environmental information during movement of the arm; and

dynamically re-planning movement of the arm using the additional environmental information.

13. The method of claim 12, further comprising:

using a first planning scheme for an initial planned movement using the environmental information; and

using a second planning scheme that is different from the first planning scheme for a revised planned movement using the additional environmental information.

14. The method of claim 12, wherein he visual acquisition unit comprises an arm-mounted visual acquisition unit.

15. The method of claim 1, wherein moving the arm toward the target comprises moving the arm in a series of stages, wherein the selected at least one planning scheme includes a first planning scheme for at least one of the stages and a second planning scheme that is different from the first planning scheme for at least one other of the stages.

9. A method for controlling a robot system, the method comprising:

acquiring, with a visual acquisition unit, environmental information corresponding to at least one of an arm or a target to which the arm is configured to he moved toward;

generating, with at least one processor, an environmental model using the environmental information;

selecting, from a plurality of planning schemes, at least one planning scheme based on the environmental model generated using the environmental information to translate the arm toward the target, wherein planning scheme is defined by at least one of path shape or path type, and wherein the at least one planning scheme selected from the plurality of planning schemes is selected to provide movement of the articulable arm within at least one of a desired time frame or at a desired speed based on the environmental model generated using the environmental information;

planning movement of the arm toward the target using the selected at least one planning scheme; and

moving the arm toward the target using that at least one selected planning scheme.

10. The method of claim 9, wherein at least one of the planning schemes utilizes a first coordinate system, and at least one other of the planning schemes utilizes a second coordinate system that is different from the first coordinate system.

11. The method of claim 10, wherein the first coordinate system is a Cartesian coordinate system, and the second coordinate system is a joint space coordinate system, wherein the method includes selecting between the planning schemes based on an amount of openness determined using the environmental model generated using the environmental information.

16. A tangible and non-transitory computer readable medium comprising one or more computer software modules configured to direct one or more processors to:

acquire, with a visual acquisition unit, environmental information corresponding to at least one of an arm or a target to whiCh the arm is configured to be moved toward;

generate, with at least one processor, an environmental model using the environmental information;

select, from a plurality of planning schemes, at least one planning scheme based on the environmental model generated using the environmental information to translate the arm toward the target wherein each planning scheme is defined by at least one of path shape or path type, and wherein the at least one planning scheme selected from the plurality of planning schemes is selected to provide movement of the articulable arm within at least one of a desired time frame or at a desired speed based on the environmental model generated using the environmental information;

plan movement of the area toward the target using the selected at least one planning scheme; and

move the arm toward the target using that at least one selected planning scheme.

17. The tangible and non-transitory computer readable medium of claim 16, wherein at least one of the planning schemes utilizes a first coordinate system, and at least one other of the planning schemes utilizes a second coordinate system that is different from the first coordinate system.

18. The tangible and non-transitory computer readable medium of claim 16, wherein the computer readable medium is further configured to direct the one or more processors to:

control the visual acquisition unit to acquire additional environmental information during movement of the arm; and

dynamically re-plan movement of the arm using the additional environmental information.

19. The tangible and non-transitory computer readable medium of claim 18, wherein the computer readable medium is further configured to direct the one or more processors to use a first planning scheme for an initial planned movement using the environmental information, and use a second planning scheme that is different from the first planning scheme for a revised planned movement using the additional environmental information.

20. The tangible and non-transitory computer readable medium of claim 16, wherein the computer readable medium is further configured to direct the one or more processors to move the arm in a series of stages, wherein the selected at least one planning scheme includes a first planning scheme for at least one of the stages and a second planning scheme that is different from the first planning scheme for at least one other of the stages.