Robot apparatus, method for controlling the same, and computer program

• US 8,996,176 B2
• Filed: 02/26/2013
• Issued: 03/31/2015
• Est. Priority Date: 03/08/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Genta, Miyamoto, Atsushi, Shimizu, Satoru
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US13/776,763
• Publication Number:
  US 20130238131A1
• Time in Patent Office: 763 Days
• Field of Search: 700/245, 700257-259, 382/103, 382153-155, 382/159, 382173-174, 382/190, 348/169, 901/47, 901 1- 4, 901/14, 901/30, 901/31, 901/46
• US Class Current: 700/259
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/39449 Pendant, pda displaying camera images overlayed with graphics, augmented reality
• CPC Class Codes: G05B 2219/40099 Graphical user interface for robotics, visual robot user interface
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05B 2219/45108 Aid, robot for aid to, assist human disabled

Abstract

A robot apparatus includes an output unit that displays an image including an object on a screen, an input unit that receives an operation performed by a user for specifying information relating to an approximate range including the object in the image, an object extraction unit that extracts information regarding a two-dimensional contour of the object on the basis of the specification received by the input unit, and a position and attitude estimation unit that estimates information regarding a three-dimensional position and attitude of the object on the basis of the information regarding the two-dimensional contour.

Claims

1. A robot apparatus comprising:

an output unit configured to display an image including an object on a screen;

an input unit configured to receive, based on an operation performed by a user, a first specification relating to an approximate range including the object in the image;

an object extraction unit configured to extract information regarding a two-dimensional contour of the object based on the first specification received by the input unit;

a position and attitude estimation unit configured to estimate information regarding a three-dimensional position and attitude of the object based on the information regarding the two-dimensional contour;

a holding unit; and

a holding planning unit configured to plan, without receiving a stored shape model from a database, a trajectory of the holding unit for holding the object based on the estimated information regarding the three-dimensional position and attitude of the object.

2. The robot apparatus according to claim 1, further comprising:

an image pickup unit,

wherein the output unit is configured to display an image captured by the image pickup unit on the screen.

3. The robot apparatus according to claim 1, further comprising:

a holding control unit configured to control a holding operation for the object performed by the holding unit in accordance with the planned trajectory.

4. The robot apparatus according to claim 1,

wherein a terminal device including the output unit and the input unit is configured to perform wireless communication with the robot apparatus including the holding unit.

5. The robot apparatus according to claim 1,

wherein the input unit is a touch panel integrated into the screen of the output unit, and

wherein the operation performed by the user comprises touching the touch panel.

6. The robot apparatus according to claim 5,

wherein the operation performed by the user comprises drawing a region surrounding the object in the captured image displayed on the touch panel.

7. The robot apparatus according to claim 5,

wherein the operation performed by the user comprises touching substantially a center of the approximate range including the object in the captured image displayed on the touch panel, while providing a radius of the approximate range using a period for which the user touches substantially the center of the approximate range.

8. The robot apparatus according to claim 5,

wherein the operation performed by the user comprises drawing a diagonal of a rectangular region surrounding the object in the captured image displayed on the touch panel.

9. The robot apparatus according to claim 1,

wherein the output unit is configured to display the information regarding the two-dimensional contour of the object extracted by the object extraction unit on the screen, and

wherein a holding operation for the object is performed when the user has approved the information regarding the two-dimensional contour through the input unit.

10. The robot apparatus according to claim 9,

wherein, when the user has not approved the information regarding the two-dimensional contour, the input unit is configured to receive a second specification from the user, and

wherein the object extraction unit is configured to extract the information regarding the two-dimensional contour of the object again on the basis of the second specification.

11. The robot apparatus according to claim 10,

wherein the input unit is configured to receive the information relating to the approximate range including the object again as the second specification.

12. The robot apparatus according to claim 10,

wherein the input unit is configured to receive, as the second specification, additional specification for the approximate range including the object specified first.

13. The robot apparatus according to claim 10,

wherein the input unit is configured to receive, as the second specification, specification of a boundary line between a plurality of objects included in the approximate range including the object specified first,

wherein the object extraction unit is configured to extract information regarding two-dimensional contours of objects from both sides of the boundary line and the output unit is configured to display the extracted information regarding the two-dimensional contours of the objects on the screen, and

wherein the holding operation for the object is performed when the user has selected an object to be held and approved the information regarding the two-dimensional contour through the input unit.

14. A method for controlling a robot apparatus, the method comprising:

displaying, using an output unit of the robot apparatus, an image including an object on a screen;

receiving, using an input unit of the robot apparatus, based on an operation performed by a user, a first specification relating to an approximate range including the object in the image;

extracting, using an object extraction unit of the robot apparatus, information regarding a two-dimensional contour of the object based on the first specification received using the input unit;

estimating, using a position and attitude estimation unit of the robot apparatus, information regarding a three-dimensional position and attitude of the object based on the information regarding the two-dimensional contour; and

planning, using a holding planning unit of the robot apparatus, without receiving a stored shape model from a database, a trajectory of a holding unit of the robot apparatus for holding the object based on the estimated information regarding the three-dimensional position and attitude of the object.

15. The method according to claim 14, further comprising:

controlling, using a holding control unit, a holding operation for the object performed by the holding unit in accordance with the planned trajectory.

16. The method according to claim 14, further comprising:

displaying, using the output unit, the information regarding the two-dimensional contour of the object extracted by the object extraction unit on the screen, and

performing, using the holding unit, a holding operation for the object when the user has approved the information regarding the two-dimensional contour through the input unit.

17. A computer-readable storage medium having stored thereon instructions that, when executed by a processor, cause the processor to function as an apparatus, the apparatus comprising:

an output unit configured to display an image including an object on a screen;

an input unit configured to receive, based on an operation performed by a user, a first specification relating to an approximate range including the object in the image;

an object extraction unit configured to extract information regarding a two-dimensional contour of the object based on the first specification received by the input unit;

a position and attitude estimation unit configured to estimate information regarding a three-dimensional position and attitude of the object based on the information regarding the two-dimensional contour;

a holding unit; and

a holding planning unit configured to plan, without receiving a stored shape model from a database, a trajectory of the holding unit for holding the object based on the estimated information regarding the three-dimensional position and attitude of the object.

18. The computer-readable storage medium according to claim 17, the apparatus further comprising:

a holding control unit configured to control a holding operation for the object performed by the holding unit in accordance with the planned trajectory.

19. The computer-readable storage medium according to claim 17,

wherein the output unit is configured to display the information regarding the two-dimensional contour of the object extracted by the object extraction unit on the screen, and

wherein a holding operation for the object is performed when the user has approved the information regarding the two-dimensional contour through the input unit.