Pose estimation

• US 8,639,035 B2
• Filed: 01/30/2013
• Issued: 01/28/2014
• Est. Priority Date: 02/07/2008
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Shiba, Hisashi
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US13/754,084
• Publication Number:
  US 20130142437A1
• Time in Patent Office: 363 Days
• Field of Search: 382/100, 382/103, 382/104, 382/154, 382/128, 382/131, 382/254, 382/115, 382/118, 382/168, 382/151, 382/155, 382/173, 382/181, 382/190, 382/201, 382/203, 382/195, 382/199, 382/232, 382/276, 382/312, 382/197, 345/418, 345/419, 345/421, 345/420, 345/426, 345/582, 702/1, 702/19, 435/7.1, 250/332, 250/341.3, 422/420, 422/408, 422/401
• US Class Current: 382/197
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation

Abstract

In a pose estimation for estimating the pose of an object of pose estimation with respect to a reference surface that serves as a reference for estimating a pose, a data processing device: extracts pose parameters from a binarized image; identifies a combination of pose parameters for which the number of cross surfaces of parameter surfaces that accord with surface parameter formulas, which are numerical formulas for expressing a reference surface, is a maximum; finds a slope weighting for each of cross pixels, which are pixels on each candidate surface and which are pixels within a prescribed range, that is identified based on the angles of the tangent plane at the cross pixel and based on planes formed by each of the axes of parameter space; and identifies the significant candidate surface for which a number, which is the sum of slope weightings, is a maximum, as the actual surface that is the reference surface that actually exists in the image.

Claims

1. A pose estimation method including a data processing switching process for, based on the results of comparing a parameter space calculation amount, which is a calculation amount indicating the volume of arithmetic processing carried out by using a first pose estimation method to identify an actual surface as a reference surface, and an image space calculation amount, which is a calculation amount indicating the volume of arithmetic processing carried out by using a second pose estimation method to identify an actual surface, selecting one of the pose estimation methods, among the first pose estimation method and the second pose estimation methods and using the selected pose estimation method to identify said actual surface:

the first pose estimation method for, based on an image that is received as input, estimating the pose of an object of pose estimation with respect to said reference surface that serves as a reference for estimating the pose, said pose estimation method comprising;

a binarization process for dividing said image into a candidate region that is a candidate for said reference surface and a background region that is a region other than the candidate region;

a surface parameter formula expression process for extracting pose parameters that indicate the pose of an object of pose estimation with respect to said reference surface that appears in said image that is received as input and, based on the combination of values obtained by applying a transform by a prescribed function upon, of pose parameters that have been extracted, parameters that indicate direction in which an object of pose estimation is directed and pose parameters other than the parameters that indicate direction, finding surface parameter formulas that are numerical formulas that express said reference surface;

a parameter computation process for, based on pose parameters that were extracted in said surface parameter formula expression process, computing a combination of pose parameters that make up said surface parameter formulas that express all surfaces that pass through each pixel contained in a candidate region that was divided in said binarization process;

a parameter surface drawing process for, based on the combination of pose parameters computed in said parameter computation process, drawing parameter surfaces that accord with said surface parameter formulas on parameter space, which is space that includes each of the parameters belonging to said pose parameters as the axes of base vectors;

an intersection detection process for finding cross point coordinates, which are the coordinates of intersections through which pass a plurality of said parameter surfaces that were drawn in said parameter surface drawing process, and cross surface numbers, which are the numbers of parameter surfaces that pass through the cross point coordinates;

a significant candidate extraction process for, when the cross surface number at each cross point that was found in said intersection detection process is compared with the cross surface numbers at other cross point coordinates in prescribed neighboring regions that take the cross point coordinates corresponding to the cross surface number as a reference and is thus determined to be a maximum, identifying the combinations of pose parameters that make up the parameter surface that corresponds to the determined cross surface number as significant candidates, which are optimum combinations of pose parameters for expressing said reference surface;

a significant candidate surface drawing process for, based on a significant candidate identified in said significant candidate extraction process and a combination of pose parameters that were computed in said parameter computation process, for which a distance to said significant candidate in said parameter space in prescribed neighboring regions that take the significant candidate as a reference is no greater than a prescribed value, drawing a significant candidate surfaces on said image;

a cross determination process for, regarding each significant candidate surface that was drawn in said significant candidate surface drawing process, identifying pixels that are located on, of said candidate region, the significant candidate surface, and pixels that are located within a prescribed range from the significant candidate surface as cross pixels and finding coordinates of the cross pixels and tangent planes at said cross pixels;

a weighting computation process for, based on angles that are formed by tangent planes at the cross pixels that were found in said cross determination process and planes that are formed by the axes of each base vector included in said parameter space, finding a slope weighting for each cross pixel that was identified in said cross determination process;

a number measurement process for computing for each of said significant candidate surfaces a number that is the value obtained by adding up the slope weightings of cross pixels that were found in said weighting computation process and that were contained in each of the significant candidates; and

an actual surface determination process for, when the number computed in said number measurement process is compared with a number belonging to other significant candidate surfaces in neighboring regions that take as a reference the significant candidate surface that has this number and thus is determined to be a maximum, identifying the significant candidate surface having the determined number as an actual surface that is said reference surface that actually exists in said image;

the second pose estimation method for, based on an image that is received as input, estimating the pose of an object of pose estimation with respect to said reference surface that serves as the reference for estimating pose, said pose estimation method comprising;

a binarization process for dividing said image into a candidate region that is a candidate for said reference surface and a background region that is a region other than the candidate region;

a surface parameter formula expression process for extracting pose parameters that indicate the pose of an object of pose estimation with respect to said reference surface that appears in said image received as input and, based on the combination of values obtained by implementing a transform by a prescribed function upon, of the pose parameters that were extracted, parameters that indicate a direction in which the object of pose estimation is directed and pose parameters other than the parameters that indicate direction, finding surface parameter formulas that are numerical formulas that express said reference surface;

an all-candidate surface drawing process for, based on the combination of all pose parameters in parameter space, which is space having, as the axes of base vectors, each of the parameters belonging to said pose parameters, drawing candidate surfaces that accord with said surface parameter formulas on the image in said parameter space;

a cross determination process for, as regards each candidate surface that was drawn in said all-candidate surface drawing process, identifying pixels that are located on, of said candidate region, the candidate surface, and pixels that are located within a prescribed range from the candidate surface as cross pixels and finding coordinates of the cross pixels and tangent planes at said cross pixels;

a weighting computation process for, based on angles that are formed by tangent planes at cross pixels that were found in said cross determination process and planes that are formed by the axes of each base vector included in said parameter space, finding a slope weighting for each cross pixel that was identified in said cross determination process;

a number measurement process for computing for each of said candidate surfaces a number that is a value obtained by adding up the slope weightings of cross pixels that were found in said weighting computation process and that were contained in each of the candidate surfaces; and

an actual surface determination process for, based on the numbers computed in said number measurement process, identifying an actual surface that is said reference surface that actually exists in said image from among candidate surfaces drawn in said all-candidate surface drawing process.

2. The pose estimation method according to claim 1, wherein said data processing switching process comprises:

a binarization process for dividing said image into a candidate region that is a candidate for said reference surface and a background region that is a region other than the candidate region;

a surface parameter formula expression process for extracting pose parameters that indicate the pose of an object of pose estimation with respect to a reference surface that appears in said image that was received as input and, based on a combination of values obtained by implementing a transform by a prescribed function upon, of the pose parameters that were extracted, parameters that indicate direction in which the object of pose estimation is directed and pose parameters other than the parameters that indicate direction, finding surface parameter formulas that are numerical formulas that express said reference surface;

a parameter computation calculation amount estimation process for estimating a parameter computation calculation amount, which is an amount that indicates the volume of arithmetic processing for using said first pose estimation method to compute the combinations of pose parameters that make up said surface parameter formulas that express all reference surfaces that pass through each pixel contained in a candidate region that was divided in said binarization process;

a parameter surface drawing calculation amount estimation process for estimating a parameter surface drawing calculation amount for using said first pose estimation method to draw parameter surfaces that accord with said surface parameter formulas;

an intersection detection calculation amount estimation process for estimating an intersection detection calculation amount for using said first pose estimation method to find said cross point coordinates and said cross surface numbers;

a significant candidate extraction calculation amount estimation process for estimating a significant candidate extraction calculation amount for using said first pose estimation method to identify said significant candidates;

a significant candidate surface drawing calculation amount estimation process for estimating a significant candidate surface drawing calculation amount for using said first pose estimation method to draw said significant candidate surfaces;

a cross determination calculation amount estimation process for estimating a cross determination calculation amount for using said first pose estimation method to identify said cross pixels and for using said first pose estimation method to find the coordinates of the cross pixels and tangent planes at said cross pixels;

a weighting computation calculation amount estimation process for estimating a weighting computation calculation amount for using said first pose estimation method to find said slope weightings;

a number measurement calculation amount estimation process for estimating a number measurement calculation amount for using said first pose estimation method to compute said numbers for each said significant candidate surface;

an actual surface determination calculation amount estimation process for estimating an actual surface determination calculation amount for using said first pose estimation method to identify said actual surfaces from among said significant candidate surfaces;

an all-candidate surface drawing calculation amount estimation process for estimating an all-candidate surface drawing calculation amount for using said second pose estimation method to draw candidate surfaces that accord with said surface parameter formulas;

an all-cross determination calculation amount estimation process for estimating an all-cross determination calculation amount for using said second pose estimation method to identify said cross pixels and for using said first pose estimation method to find coordinates of the cross pixels and tangent planes at said cross pixels;

an all-weighting computation calculation amount estimation process for estimating an all-weighting computation calculation amount for using said second pose estimation method to find said slope weightings;

an all-number measurement calculation amount estimation process for estimating an all-number measurement calculation amount for using said second pose estimation method to compute said numbers;

an all-actual surface determination calculation amount estimation process for estimating an all-actual surface determination calculation amount for using said second pose estimation method to identify said actual surfaces from among said candidate surfaces; and

said method selection process for;

finding said parameter space calculation amount by adding up said parameter computation calculation amount, said parameter surface drawing calculation amount, said intersection detection calculation amount, said significant candidate extraction calculation amount, said significant candidate surface drawing calculation amount, said cross determination calculation amount, said weighting computation calculation amount, said number measurement calculation amount, and said actual surface determination calculation amount;

finding said image space calculation amount by adding up said all-candidate surface drawing calculation amount, said all-cross determination calculation amount, said all weighting computation calculation amount, said all-number measurement calculation amount, and said all-actual surface determination calculation amount; and

when, as a result of comparing said parameter space calculation amount that was found and said image space calculation amount that was found, said parameter space calculation amount is determined to be less than said image space calculation amount, selecting said first pose estimation method to identify said actual surface, and when said parameter space calculation amount is determined to be greater than said image space calculation amount, selecting said second pose estimation method to identify said actual surface.

3. The pose estimation method according to claim 2, wherein,

the first pose estimation method, further comprising:

a pixel value-weighted parameter surface drawing process for, based on a pixel value weighting found by multiplication of a combination of pose parameters computed in said parameter computation process and a value that was computed based on pixel values of an image before binarization in said binarization process, drawing on said parameter space parameter surfaces that accord with said surface parameter formulas;

a pixel value-weighted significant candidate extraction process for, when at said cross point coordinates, the sum of the number of cross surfaces at which said parameter surfaces intersect each other and pixel value weightings is greater than a prescribed value, and moreover, when the number of cross surfaces is a maximum in neighboring regions that take the cross point coordinates as a reference, identifying the combination of parameters at the cross point coordinates as a significant candidate of a combination of parameters that form a reference surface that is the object of detection; and

a pixel value-weighted weighting computation process for, based on angles formed by tangent planes at cross pixels that were found in said cross determination process and surfaces formed by axes of each base vector included in said parameter space, finding a slope weighting for each cross pixel that was identified in said cross determination process, and, by multiplying these slope weightings that were found and values computed based on pixel values of an image before binarization in said binarization process, finding pixel value-weighted slope weightings;

said data processing switching process comprises;

a pixel value-weighted parameter surface drawing process for, when drawing parameter surfaces on said parameter space, finding pixel value weightings based on pixel values of an image before binarization in said binarization process;

a pixel value-weighted significant candidate extraction process for, when at said intersections, the sum of the number of cross surfaces at which parameter surfaces intersect each other and pixel value weightings is greater than a prescribed value, and moreover, when the number of cross surfaces is a maximum in neighboring regions that take the cross point coordinates as a reference, identifying the combination of parameters at the cross point coordinates as a significant candidate of a combination of parameters that form a reference surface that is the object of detection;

a pixel value-weighted weighting computation process for further multiplying said slope weightings by values that are computed based on pixel values of a three-dimensional image before binarization and then taking the result as new slope weightings;

a pixel value-weighted parameter surface drawing calculation amount estimation process for estimating the calculation amount necessary for execution of said weighted parameter surface drawing process;

a pixel value-weighted significant candidate extraction calculation amount estimation process for estimating the calculation amount necessary for execution of said pixel value-weighted significant candidate extraction process; and

a pixel value-weighted weighting computation calculation amount estimation process for estimating the calculation amount necessary for execution of said pixel value-weighted weighting computation process.

4. The pose estimation method according to claim 1, comprising a communication process for, when an actual surface is determined in said actual surface determination process, transmitting to the outside pose parameters that make up the actual surface that was determined.