METHOD FOR THE SELECTION OF PHYSICAL OBJECTS IN A ROBOT SYSTEM

• US 20130151007A1
• Filed: 05/05/2011
• Published: 06/13/2013
• Est. Priority Date: 06/24/2010
• Status: Active Grant

• Current Assignee: MP Zenrobotics OY
• Original Assignee: Zenrobotics Oy (Terex Corporation)
• Inventors: Valpola, Harri, Lukka, Tuomas
• Granted Patent: US 9,050,719 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 13/08 by means of sensing devices, e.g. viewing or touching devices
• CPC Class Codes: B25J 13/085 Force or torque sensors B25J13/082, B25J13/084 take precedence
• CPC Class Codes: B25J 9/00 Programme-controlled manipulators
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39528 Measuring, gripping force sensor build into hand
• CPC Class Codes: G05B 2219/40053 Pick 3-D object from pile of objects
• CPC Class Codes: G05B 2219/40571 Camera, vision combined with force sensor
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

A method in which an apparatus receives first sensor data from first sensors and determines a target position from the data, the target position may be a position in space or an orientation of a gripper in a robot arm First instructions are issued to the robot arm or the gripper in order to move a gripper to the target position. Force feedback sensor data is received from force feedback sensors associated with either the robot arm or the gripper or from the first sensors. A failure in carrying out the first instructions is determined Second sensor data is received from the at least one first sensor. Successful gripping of an object is determined from the second sensor data.

Claims

1. A method, comprising:

receiving first sensor data from at least one first sensor;

determining a target position from the first sensor data, said target position comprising at least one of a position in space and an orientation for a gripper in association with a robot arm;

starting to issue at least one first instruction to at least one of the robot arm and the gripper in order to move the gripper to the target position;

receiving force feedback sensor data from at least one of the at least one first sensor and at least one force feedback sensor, associated with at least one of the robot arm and the gripper, to determine a failure in carrying out the at least one instruction;

receiving second sensor data from the at least one first sensor;

determining a successful gripping of an object from the second sensor data;

receiving verification sensor data from at least one second sensor, in response to the determining of the successful gripping; and

issuing at least one second instruction to the robot arm in order to move the arm to a predetermined position to release the grip of the gripper.

2. The method according to claim 1, the method further comprising:

registering the successful gripping and the first sensor data to a first learning entity.

3. The method according to claim 2, the method further comprising:

registering an unsuccessful gripping and the first sensor data to the first learning entity, in response to a determined failure in carrying out at least one of the at least one instruction.

4. The method according to claim 2, the method further comprising:

predicting a likelihood for the successful gripping from the first sensor data using the information registered in the first learning entity; and

issuing the at least one first instruction to the robot arm in response to the likelihood.

5. The method according to claim 3, the method further comprising:

issuing at least one third instruction to the robot arm to move the gripper to a verification position.

6. The method according to claim 5, wherein the verification position corresponds to a verification environment where the at least one second sensor provide may be applied to the object without interfering objects.

7. The method according to claim 6, wherein the verification environment has a constant background.

8. The method according to claim 5, the method further comprising:

determining a type of the object from the verification sensor data from the at least one second sensor.

9. The method according to claim 1, wherein the step of starting to issue the at least one first instruction to the robot arm further comprises:

providing first coordinates to the robot arm;

receiving first intermediate sensor data from the at least one first sensor; and

determining an actual position of the gripper in a sensor coordinate system using the first intermediate sensor data.

10. The method according to claim 9, wherein the step of starting to issue the at least one first instruction to the robot arm further comprises:

determining an error between the actual position of the gripper and the first coordinates;

providing second coordinates to the robot arm;

receiving second intermediate sensor data from the at least one first sensor;

determining a second actual position of the gripper in a sensor coordinate system using the second intermediate sensor data; and

checking whether the error is smaller than a predetermined threshold.

11. The method according to claim 9, wherein the step of starting to issue the at least one first instruction to the robot arm further comprises:

registering an association between coordinates of the robot arm and the position of the gripper as determined in the sensor coordinate system in a second learning entity.

12. The method according to claim 11, wherein the step of starting issue the at least one first instruction to the robot arm further comprises:

constructing a conversion function between coordinates of the robot arm and coordinates in the sensor coordinates system, using the association registered.

13. The method according to claim 11, wherein the step of determining a target position from the first sensor data further comprises:

determining an initial target position in the sensor coordinate system; and

computing the target position from the initial target position using the conversion function.

14. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:

receiving first sensor data from at least one first sensor;

determining a target position from the first sensor data, said target position comprising at least one of a position in space and an orientation for a gripper in association with a robot arm;

starting to issue at least one first instruction to at least one of the robot arm and the gripper in order to move the gripper to the target position;

receiving force feedback sensor data from at least one of at least one force feedback sensor, associated with at least one of the robot arm and the gripper, and the at least one first sensor, to determine a failure in carrying out the at least one instruction;

receiving second sensor data from the at least one first sensor;

determining a successful gripping of an object from the second sensor data;

receiving verification sensor data from at least one second sensor, in response to the determining of the successful gripping; and

issuing at least one second instruction to the robot arm in order to move the arm to a predetermined position to release the grip of the gripper.

15. An apparatus, comprising:

means for receiving first sensor data from at least one first sensor;

means for determining a target position from the first sensor data, said target position comprising at least one of a position in space and an orientation for a gripper in association with a robot arm;

means for starting to issue at least one first instruction to at least one of the robot arm and the gripper in order to move the gripper to the target position;

means for receiving force feedback sensor data from at least one of at least one force feedback sensor, associated with at least one of the robot arm and the gripper, and the at least one first sensor, to determine a failure in carrying out the at least one instruction;

means for receiving second sensor data from the at least one first sensor;

means for determining a successful gripping of an object from the second sensor data;

means for receiving verification sensor data from at least one second sensor, in response to the determining of the successful gripping; and

means for issuing at least one second instruction to the robot arm in order to move the arm to a predetermined position to release the grip of the gripper.

16. A computer program embodied on a computer readable medium, the computer program comprising code for controlling a processor to execute a method comprising:

receiving first sensor data from at least one first sensor;

determining a target position from the first sensor data, said target position comprising at least one of a position in space and an orientation for a gripper in association with a robot arm;

starting to issue at least one first instruction to at least one of the robot arm and the gripper in order to move the gripper to the target position;

receiving force feedback sensor data from at least one of at least one force feedback sensor, associated with at least one of the robot arm and the gripper, and the at least one first sensor, to determine a failure in carrying out the at least one instruction;

receiving second sensor data from the at least one first sensor;

determining a successful gripping of an object from the second sensor data;

receiving verification sensor data from at least one second sensor, in response to the determining of the successful gripping; and

issuing at least one second instruction to the robot arm in order to move the arm to a predetermined position to release the grip of the gripper.

17. The computer program according to claim 16, wherein said computer program is stored on a computer readable medium.

18. The computer program according to claim 17, wherein said computer readable medium is a removable memory card, a holographic memory, a magnetic disk or an optical disk.