Position-and-attitude recognition method and apparatus by use of image pickup means

• US 5,499,306 A
• Filed: 03/07/1994
• Issued: 03/12/1996
• Est. Priority Date: 03/08/1993
• Status: Expired due to Term

• Current Assignee: Nippondenso Co., Ltd. (DENSO Corporation)
• Original Assignee: Nippondenso Co., Ltd. (DENSO Corporation)
• Inventors: Saitou, Hideki, Kominami, Tetsuya, Kanayama, Yukihiro, Sasaki, Mikio
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Tran, Phuoc
• Application Number:
  US08/206,828
• Time in Patent Office: 736 Days
• Field of Search: 382/48, 382/49, 382/1, 382/8, 382/100, 382/153, 382/203, 382/291, 382/302, 348/94, 348/95, 395/85, 395/86, 395/89, 395/94
• US Class Current: 382/291
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors

Abstract

A positional relationship between an autonomous mobile robot and a workpiece to be handled by the robot is calculated promptly and accurately on the basis of the image information obtained from a monocular vision camera mounted on the robot. Extracting the image feature points of the workpiece taken by the camera, a search region with a plurality of discrete space points is set on the basis of the estimated position and attitude of the camera. Then, the estimated image feature points of the workpiece, to be obtained when the workpiece is taken a view from these space points, are obtained. Then an error between the image feature point and the estimated feature point is calculated, and the position and attitude of the camera can be determined by finding the space point where the error is minimized.

Claims

1. A position and attitude recognition method for determining the position and attitude of an image pickup means with respect to an object having three-dimensional shape, the method comprising:

an image pickup step using the image pickup means for taking two-dimensional image information of the object;

an image processing step for extracting actual image feature representing the shape of an object, on the basis of said two-dimensional image information of said object taken by said image pickup means;

a search region setting step for setting a search region with a plurality of discrete space points having a required accuracy in discrete value and allocated in the vicinity of said image pickup means whose position and attitude are estimated based on external information;

perspective transformation equation calculating step for obtaining a perspective transformation equation for calculating two-dimensional estimated image feature of said object to be obtained when a view of said object is taken by said image pickup means located at each of said discrete space points of said search region, derived from the three-dimensional shape of said object;

an error calculating step for calculating an error between said actual image feature extracted by said image processing step and said estimated image feature calculated by said perspective transformation calculating step for each of said discrete space points; and

a searching step for searching each of said discrete space points to find a space point which has a minimum error between said actual image feature and said estimated image feature, thereby determining the position and attitude of said image pickup means with respect to said object.

2. A position and attitude recognition method in accordance with claim 1, wherein said actual image feature and said estimated image feature are represented by actual image feature points and estimated image feature points, respectively.

3. A position and attitude recognition method in accordance with claim 2, wherein said error is calculated as an error based on a distance between an actual image feature point and an estimated image feature point.

4. A position and attitude recognition method in accordance with claim 2, wherein said error is calculated as an error between a line connecting said actual image feature points and a line connecting said estimated image feature points.

5. A position and attitude recognition method in accordance with claim 2, wherein said error is calculated as an area error between a shape whose contour connecting said actual image feature points and a shape whose contour connecting said estimated image feature points.

6. A position and attitude recognition method in accordance with claim 2, wherein said search region setting step sets said search region on the basis of an edge error between edge pattern formed by said actual image feature points and edge pattern formed by said estimated image feature points.

7. A position and attitude recognition method in accordance with claim 6, further comprising a judging step for judging error condition of said attitude based on said edge error, and a search region for attitude is determined on the basis of the result of said judging step.

8. A position and attitude recognition method in accordance with claim 1, wherein said error is calculated as a pixel value error between said actual image feature and said estimated image feature.

9. A position and attitude recognition method in accordance with claim 1, wherein said image processing step corrects said actual image feature on the basis of color information involved in said image information.

10. A position and attitude recognition method in accordance with claim 1, wherein said image pickup means is a single image pickup means.

11. A position and attitude recognition method in accordance with claim 1, further comprising steps of:

obtaining said perspective transformation equation between said actual image feature and absolute values of said object according to least-square estimation method;

obtaining an estimated perspective transformation equation based on said estimated image feature,

wherein said error is obtained as error between components between said perspective transformation equation and said estimated perspective transformation equation.

12. A position and attitude recognition method in accordance with claim 1, wherein said search region setting step sets a plurality of search regions for renewal of said search region so as to repeatedly execute said search region setting step, said estimated image feature calculating step, and said error calculating step, thereby allowing said searching step to find a space point which reduces said error within a desired region.

13. A position and attitude recognition method in accordance with claim 12, wherein said search region setting step sets a new search region which is smaller than an older search region in said renewal of said search region.

14. A position and attitude recognition method in accordance with claim 1, wherein said estimated image feature is selected from perspective patterns memorized in memory means.

15. A position and attitude recognition method in accordance with claim 1, wherein said position and attitude are defined by a plurality of parameters, and said searching step is executed independently for each of said parameters.

16. A position and attitude recognition method in accordance with claim 15, wherein a parameter defining position is searched after a specific parameter defining attitude is fixed as a predetermined value.

17. A position and attitude recognition apparatus in accordance with claim 1, wherein said attitude, estimated by the external information, is estimated based on a deviation amount from a central position of an image plane of said two-dimensional image information.

18. A position and attitude recognition apparatus for determining the position and attitude of image pickup means with respect to an object having three-dimensional shape, said apparatus comprising:

image pickup means for taking two-dimensional image information of the object having three-dimensional shape;

image processing means for extracting actual image feature representing the shape of an object, on the basis of said two-dimensional image information of said object taken by said image pickup means;

search region setting means for setting a search region with a plurality of discrete space points having a required accuracy in discrete value allocated in the vicinity of said image pickup means whose position and attitude are estimated based on external information;

perspective transformation equation calculating means for obtaining a perspective transformation equation for calculating two-dimensional image feature of said object to be obtained when a view of said object is taken by said image pickup means located at each of discrete space points of said search region, derived from the three-dimensional shape of said object;

perspective transformation calculating means for obtaining estimated image feature of said object to be obtained when view of said object is taken by said image pickup means located at each of said discrete space point of said search region, based on said perspective transformation equation;

error calculating means for calculating an error between said actual image feature extracted by said image processing means and said estimated image feature calculated by said perspective transformation calculating means for each of said discrete space points; and

position and attitude determining means for searching each of said discrete space points to find a space point which has a minimum error between the actual image feature and the estimated image feature, thereby determining the position and attitude of said image pickup means with respect to said object.

19. A position and attitude recognition apparatus in accordance with claim 18, wherein said search region setting point found by said position and attitude determining means, so as to execute search until said error becomes smaller that a predetermined value.