Information processing apparatus and information processing method

• US 9,102,053 B2
• Filed: 03/04/2013
• Issued: 08/11/2015
• Est. Priority Date: 03/09/2012
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Suzuki, Masahiro
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US13/784,684
• Publication Number:
  US 20130238125A1
• Time in Patent Office: 890 Days
• Field of Search: 700/245, 700/253, 700/258, 700/259, 318/568.1, 318/568.11, 318/568.12, 318/568.16, 318/568.21, 901/2, 901/14, 901/31, 901/46
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1612 characterised by the hand, wrist, grip control
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39508 Reorientation of object, orient, regrasp object
• CPC Class Codes: G05B 2219/40053 Pick 3-D object from pile of objects

Abstract

The state of an area where one or more target objects are arranged is determined based on the result of measurement by a first sensor. A robot is controlled to grip one of the target objects by the grip unit of the robot. The gripping state of the gripping target object gripped by the grip unit is determined from the result of measuring the gripping target object by using a second sensor. When it is determined that gripping has failed, a rearrangement area where the gripping target object is to be rearranged is decided using the result of measurement by the first sensor or that of measurement by the second sensor. The robot is controlled to rearrange the gripping target object in the rearrangement area by the grip unit.

Claims

1. An information processing apparatus comprising:

an area state determination unit which determines, based on a result of measurement by a first sensor, a state of an area where a plurality of target objects are arranged;

a control unit which controls a robot to hold one target object among the plurality of target objects by a holding unit of the robot;

a holding state determination unit which determines a holding state of a holding target object held by the holding unit from a result of measuring the holding target object by using a second sensor; and

an area decision unit which, when holding is determined based on the holding state determined by said holding state determination unit to have failed, decides, by using at least one of the result of measurement by the first sensor and the result of measurement by the second sensor, a return area where the holding target object is to be returned,

wherein said control unit controls the robot to return the holding target object in the return area.

2. The apparatus according to claim 1, wherein

said area state determination unit acquires a sensed image of the plurality of target objects from the first sensor, and determines a state of an area including at least position and orientation information of the plurality of target objects by using the sensed image, and

said holding state determination unit acquires a sensed image of the holding target object from the second sensor, and determines a position and orientation of the holding target object by using the sensed image of the holding target object.

3. The apparatus according to claim 2, wherein said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects which is specified based on the position and orientation obtained by said area state determination unit is placed, that corresponds to an area excluding an area of the plurality of target objects in the sensed image of the set.

4. The apparatus according to claim 1, wherein

said area state determination unit determines a state of an area including a two-dimensional image feature of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area having two-dimensional image features in an amount falling outside a predetermined range in the sensed image of the set.

5. The apparatus according to claim 1, wherein

said area state determination unit determines a distance measurement state of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area where the distance measurement state is poor in the area where the plurality of target objects are placed.

6. The apparatus according to claim 1, wherein

said area state determination unit determines a distribution of distances of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area where a variance of distances is smaller than a predetermined value in the area where the plurality of target objects are placed.

7. The apparatus according to claim 1, wherein

said area state determination unit determines a range of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area excluding a range within a predetermined distance from a boundary in the area where the plurality of target objects are placed.

8. The apparatus according to claim 1, wherein said control unit controls the robot to return the holding target object in a predetermined orientation by the holding unit.

9. The apparatus according to claim 1, wherein said control unit controls the robot to return the holding target object by the holding unit in an orientation different from an orientation before holding the holding target object.

10. The apparatus according to claim 1, wherein said control unit controls the robot to return the holding target object by the holding unit in an orientation different from orientations of target objects presenting around the holding target object before holding the holding target object.

11. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as each unit of an information processing apparatus defined in claim 1.

12. The apparatus according to claim 1, wherein the return area is a part of a whole area in a container where the plurality of target objects are contained.

13. An information processing apparatus comprising:

an area state determination unit which determines, based on a result of measurement by a first sensor, a state of an area where a plurality of target objects are arranged;

a control unit which controls a robot to hold one target object among the plurality of target objects by a holding unit of the robot;

an assembly state determination unit which determines an assembly state of a holding target object held by the holding unit from a result of measuring the holding target object by using a second sensor; and

an area decision unit which, when assembly is determined based on the assembly state determined by said assembly state determination unit to have failed, decides, by using the result of measurement by the first sensor, a return area where the holding target object is to be returned,

wherein said control unit controls the robot to return the holding target object in the return area.

14. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as each unit of an information processing apparatus defined in claim 13.

15. The apparatus according to claim 13, wherein

said area state determination unit acquires a sensed image of the plurality of target objects from the first sensor, and determines a state of an area including at least position and orientation information of the plurality of target objects by using the sensed image, and

said assembly state determination unit acquires a sensed image of the holding target object from the second sensor, and determines a position and orientation of the holding target object by using the sensed image of the holding target object.

16. The apparatus according to claim 15, wherein said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects which is specified based on the position and orientation obtained by said area state determination unit is placed, that corresponds to an area excluding an area of the plurality of target objects in the sensed image of the set.

17. The apparatus according to claim 13, wherein

said area state determination unit determines a state of an area including a two-dimensional image feature of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area having two-dimensional image features in an amount falling outside a predetermined range in the sensed image of the set.

18. The apparatus according to claim 13, wherein

said area state determination unit determines a distance measurement state of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area where the distance measurement state is poor in the area where the plurality of target objects are placed.

19. The apparatus according to claim 13, wherein

said area state determination unit determines a distribution of distances of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area where a variance of distances is smaller than a predetermined value in the area where the plurality of target objects are placed.

20. The apparatus according to claim 13, wherein

said area state determination unit determines a range of the area where the plurality of target objects are placed, and

said area decision unit decides, as the return area, an area on a surface on which a set of the plurality of target objects is placed, that corresponds to an area excluding a range within a predetermined distance from a boundary in the area where the plurality of target objects are placed.

21. The apparatus according to claim 13, wherein said control unit controls the robot to return the holding target object in a predetermined orientation by the holding unit.

22. The apparatus according to claim 13, wherein said control unit controls the robot to return the holding target object by the holding unit in an orientation different from an orientation before holding the holding target object.

23. The apparatus according to claim 13, wherein said control unit controls the robot to return the holding target object by the holding unit in an orientation different from orientations of target objects presenting around the holding target object before holding the holding target object.

24. The apparatus according to claim 13, wherein the return area is a part of a whole area in a container where the plurality of target objects are contained.

25. An information processing apparatus comprising:

a robot including a holding unit;

a first sensor;

a second sensor;

an area state determination unit which determines, based on a result of measurement by a first sensor, a state of an area where a plurality of target objects are arranged;

a control unit which controls said robot to hold one target object among the plurality of target objects by the holding unit;

a holding state determination unit which determines a holding state of a holding target object held by the holding unit from a result of measuring the holding target object by using a second sensor; and

an area decision unit which, when holding is determined based on the holding state determined by said holding state determination unit to have failed, decides, by using at least one of the result of measurement by the first sensor and the result of measurement by the second sensor, a return area where the holding target object is to be returned,

wherein said control unit controls said robot to return the holding target object in the return area.

26. An information processing apparatus comprising:

a robot including a holding unit;

a first sensor;

a second sensor;

an area state determination unit which determines, by using a first sensor, a state of an area where plurality of target objects that can be held by the holding unit exists;

a control unit which controls said robot to hold one target object among the plurality of target objects by the holding unit;

an assembly state determination unit which determines an assembly state of a holding target object held by the holding unit from a result of measuring the holding target object by using a second sensor; and

an area decision unit which, when assembly is determined based on the assembly state determined by said assembly state determination unit to have failed, decides, by using the result of measurement by the first sensor, a return area where the holding target object is to be returned,

wherein said control unit controls said robot to return the holding target object in the return area.

27. An information processing method comprising:

an area state determination step of determining, based on a result of measurement by a first sensor, a state of an area where plurality of target objects are arranged;

a control step of controlling a robot to hold one target object among the plurality of target objects by a holding unit of the robot;

a holding state determination step of determining a holding state of a holding target object held by the holding unit from a result of measuring the holding target object by using a second sensor; and

an area decision step of, when holding is determined based on the holding state determined in the holding state determination step to have failed, deciding, by using at least one of the result of measurement by the first sensor and the result of measurement by the second sensor, a return area where the holding target object is to be returned,

wherein in the control step, the robot is controlled to return the holding target object in the return area.

28. An information processing method comprising:

an area state determination step of determining, based on a result of measurement by a first sensor, a state of an area where plurality of target objects are arranged;

a control step of controlling a robot to hold one target object among the plurality of target objects by a holding unit of the robot;

an assembly state determination step of determining an assembly state of a holding target object held by the holding unit from a result of measuring the holding target object by using a second sensor; and

an area decision step of, when assembly is determined based on the assembly state determined in the assembly state determination step to have failed, deciding, by using the result of measurement by the first sensor, a return area where the holding target object is to be returned,

wherein in the control step, the robot is controlled to return the holding target object in the return area.

29. An information processing apparatus comprising:

a control unit which controls a robot to hold one target object among a plurality of target objects by a holding unit of the robot;

a holding state determination unit which determines a holding state of a holding target object held by the holding unit from a result of measuring the holding target object by using a sensor; and

an area decision unit which, when holding is determined based on the holding state determined by said holding state determination unit to have failed, decides a return area where the holding target object is to be returned,

wherein said control unit controls the robot to return the holding target object in the return area.

30. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as each unit of an information processing apparatus defined in claim 29.

31. The apparatus according to claim 29, wherein the return area is a part of a whole area in a container where the plurality of target objects are contained.

32. An information processing method comprising:

a control step of controlling a robot to hold one target object among a plurality of target objects by a holding unit of the robot;

a holding state determination step of determining a holding state of a holding target object held by the holding unit from a result of measuring the holding target object by using a sensor; and

an area decision step of, when holding is determined based on the holding state determined by said holding state determination step to have failed, deciding a return area where the holding target object is to be returned,

wherein said control step controls the robot to return the holding target object in the return area.

33. An information processing apparatus comprising:

a control unit which controls a robot to hold one target object among a plurality of target objects by a holding unit of the robot;

a holding state determination unit which determines whether a holding state of a holding target object held by the holding unit is a specific state based on a result of measuring the holding target object by using a sensor; and

an area decision unit which, when said holding state determination unit determines that the holding state of the holding target object is the specific state, decides a return area where the holding target object is to be returned,

wherein said control unit controls the robot to return the holding target object in the return area.

34. A non-transitory computer-readable storage medium storing a computer program for causing a computer to function as each unit of an information processing apparatus defined in claim 33.

35. The apparatus according to claim 33, wherein the return area is a part of a whole area in a container where the plurality of target objects are contained.

36. An information processing method comprising:

a control step of controlling a robot to hold one target object among a plurality of target objects by a holding unit of the robot;

a holding state determination step of determining whether a holding state of a holding target object held by the holding unit is a specific state based on a result of measuring the holding target object by using a sensor; and

an area decision step of, when said holding state determination step determines that the holding state of the holding target object is the specific state, deciding a return area where the holding target object is to be returned,

wherein said control step controls the robot to return the holding target object in the return area.