ROBOT APPARATUS AND POSITION AND ORIENTATION DETECTING METHOD

• US 20130182903A1
• Filed: 01/17/2013
• Published: 07/18/2013
• Est. Priority Date: 01/18/2012
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: NAMMOTO, Takashi, HASHIMOTO, Koichi, ZANG, Chuantao
• Granted Patent: US 9,082,017 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A robot apparatus includes a reference-model storing unit configured to store a reference model of an object, a feature-value-table storing unit configured to store a feature value table that associates position data and orientation data of the reference model and a feature value, a photographed-image acquiring unit configured to capture a photographed image of the object, a detecting unit configured to calculate a photographed image feature value from the photographed image, and a driving control unit configured to control a robot main body on the basis of the position data and the orientation data to change the position and the orientation of a gripping unit.

Claims

1. A robot apparatus comprising:

a robot main body movably including a hand;

a reference-model storing unit configured to store a reference model obtained by converting an object into a three-dimensional model;

a feature-value-table storing unit configured to store a feature value table that associates position data and orientation data of the reference model, which are acquired by changing, at an interval set in advance, a position and a orientation of the reference model in a three-dimensional space, and a reference model image feature value, which is a feature value of a two-dimensional reference model image;

a photographed-image acquiring unit configured to capture a photographed image of the object;

a detecting unit configured to calculate a photographed image feature value, which is a feature value of the photographed image captured by the photographed-image acquiring unit, detect, from the feature value table stored in the feature-value-table storing unit, a reference model image feature value coinciding with the photographed image feature value and extract a plurality of sets of position data and orientation data associated with the reference model image feature value, set, according to the plurality of sets of position data and orientation data, a position and a orientation in the three-dimensional space of the reference model stored in the reference-model storing unit and acquire two-dimensional images for the plurality of sets, and detect position data and orientation data corresponding to a two-dimensional image having a highest correlation between the photographed image and each of the two-dimensional images for the plurality of sets; and

a driving control unit configured to control the robot main body on the basis of the position data and the orientation data detected by the detecting unit to update a position and a orientation of the hand.

2. The robot apparatus according to claim 1, further comprising a detailed detection unit configured to capture the position data and the orientation data detected by the detecting unit, set, according to the position data and the orientation data, a position and a orientation in the three-dimensional space of the reference model stored in the reference-model storing unit and acquire a two-dimensional image, and calculate a two-dimensional image having a highest correlation between the photographed image and the two-dimensional image according to a nonlinear minimization method and detect position data and orientation data corresponding to the two-dimensional image, wherein

the driving control unit controls the robot main body on the basis of the position data and the orientation data detected by the detailed detection unit to change the position and the orientation of the hand.

3. The robot apparatus according to claim 1, further comprising a detailed detection unit configured to capture the position data and the orientation data detected by the detecting unit, set, according to the position data and the orientation data, a position and a orientation in the three-dimensional space of the reference model stored in the reference-model storing unit and acquire a two-dimensional image, and calculate a two-dimensional image having a highest correlation between the photographed image and the two-dimensional image according to a nonlinear minimization method and detect position data and orientation data corresponding to the two-dimensional image, wherein

the driving control unit controls the robot main body on the basis of the position data and the orientation data detected by the detailed detection unit to change the position and the orientation of the hand, and

the nonlinear minimization method is a steepest descent method.

4. The robot apparatus according to claim 1, further comprising a detailed detection unit configured to capture the position data and the orientation data detected by the detecting unit, set, according to the position data and the orientation data, a position and a orientation in the three-dimensional space of the reference model stored in the reference-model storing unit and acquire a two-dimensional image, and calculate a two-dimensional image having a highest correlation between the photographed image and the two-dimensional image according to a nonlinear minimization method and detect position data and orientation data corresponding to the two-dimensional image, wherein

the driving control unit controls the robot main body on the basis of the position data and the orientation data detected by the detailed detection unit to change the position and the orientation of the hand, and

the nonlinear minimization method is a Levenberg-Marquardt algorithm.

5. The robot apparatus according to claim 1, wherein the reference model is three-dimensional computer graphics.

6. The robot apparatus according to claim 1, wherein

the reference model image feature value is a coordinate value indicating a center of gravity position of the reference model image, and

the photographed image feature value is a coordinate value indicating a center of gravity position of the photographed image.

7. The robot apparatus according to claim 1, wherein

the reference model image feature value is an image moment of the reference model image, and

the photographed image feature value is an image moment of the photographed image.

8. A robot apparatus comprising:

a robot main body movably including hands of two systems;

a conveying unit configured to support the robot main body to be capable of moving;

an imaging unit attached to one hand of the hands of the two systems and configured to photograph an object to generate a photographed image;

a reference-model storing unit configured to store a reference model obtained by converting the object into a three-dimensional model;

a feature-value-table storing unit configured to store a feature value table that associates position data and orientation data of the reference model, which are acquired by changing, at an interval set in advance, a position and a orientation of the reference model in a three-dimensional space, and a reference model image feature value, which is a feature value of a two-dimensional reference model image;

a photographed-image acquiring unit configured to capture the photographed image generated by the photographing unit;

a detecting unit configured to calculate a photographed image feature value, which is a feature value of the photographed image captured by the photographed-image acquiring unit, detect, from the feature value table stored in the feature-value-table storing unit, a reference model image feature value coinciding with the photographed image feature value and extract a plurality of sets of position data and orientation data associated with the reference model image feature value, set, according to the plurality of sets of position data and orientation data, a position and a orientation in the three-dimensional space of the reference model stored in the reference-model storing unit and acquire two-dimensional images for the plurality of sets, and detect position data and orientation data corresponding to a two-dimensional image having a highest correlation between the photographed image and each of the two-dimensional images for the plurality of sets; and

a driving control unit configured to control the robot main body on the basis of the position data and the orientation data detected by the detecting unit to change a position and a orientation of the other hand of the hands of the two systems.

9. A position and orientation detecting method comprising:

a photographed-image acquiring unit capturing a photographed image of an object;

a detecting unit calculating a photographed image feature value, which is a feature value of the photographed image captured from the photographed-image acquiring unit;

the detecting unit detecting the reference model image feature value which coincides with the photographed image feature value, from a feature value table stored in a feature-value-table storing unit having stored therein the feature value table that associates position data and orientation data of the reference model, which are acquired by changing, at an interval set in advance, a position and a orientation in a three-dimensional space of a reference model obtained by converting the object into a three-dimensional model, and a reference model image feature value, which is a feature value of a two-dimensional reference model image, and extracting a plurality of sets of position data and orientation data associated with the reference model image feature value;

the detecting unit setting, according to the plurality of sets of position data and orientation data, a position and a orientation in the three-dimensional space of the reference model stored in the reference-model storing unit having stored therein the reference model and acquiring two-dimensional images for the plurality of sets; and

the detecting unit detecting position data and orientation data corresponding to a two-dimensional image having a highest correlation between the photographed image and each of the two-dimensional images for the plurality of sets.