Pose estimation

• US 8,401,295 B2
• Filed: 02/06/2009
• Issued: 03/19/2013
• Est. Priority Date: 02/07/2008
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Shiba, Hisashi
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US12/367,178
• Publication Number:
  US 20090202174A1
• Time in Patent Office: 1,502 Days
• Field of Search: 382/100, 382/118, 382/154, 382/104, 382/128, 382/131, 382/115, 382/103, 382/254, 382/168, 382/151, 382/155, 382/173, 382/181, 382/190, 382/201, 382/203, 382/195, 382/199, 382/232, 382/276, 382/312, 345/419, 345/418, 345/421, 345/420, 345/582, 345/426, 702/1, 702/19, 435/7.1, 250/332, 250/341.3, 422/420, 422/408, 422/401
• US Class Current: 382/181
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation

Abstract

In a pose estimation for estimating the pose of an object of pose estimation with respect to a reference surface that serves as a reference for estimating a pose, a data processing device: extracts pose parameters from a binarized image; identifies a combination of pose parameters for which the number of cross surfaces of parameter surfaces that accord with surface parameter formulas, which are numerical formulas for expressing a reference surface, is a maximum; finds a slope weighting for each of cross pixels, which are pixels on each candidate surface and which are pixels within a prescribed range, that is identified based on the angles of the tangent plane at the cross pixel and based on planes formed by each of the axes of parameter space; and identifies the significant candidate surface for which a number, which is the sum of slope weightings, is a maximum, as the actual surface that is the reference surface that actually exists in the image.

Claims

1. A data processing device comprising:

a binarization unit for dividing an image received as input into a candidate region that is a candidate for a reference surface that serves as a reference for estimating pose and a background region that is a region other than the candidate region;

a surface parameter formula expression unit for extracting pose parameters that indicate pose of an object of pose estimation with respect to a reference surface that appears in said image that was received as input and, based on a combination of values obtained by implementing a transform by a prescribed function upon, of the pose parameters that were extracted, parameters that indicate the direction in which the object of pose estimation is directed and pose parameters other than the parameters that indicate direction, finding surface parameter formulas that are numerical formulas that express said reference surface;

a parameter computation unit for, based on pose parameters that were extracted by said surface parameter formula expression unit, computing the combination of pose parameters that make up said surface parameter formulas that express all surfaces that pass through each pixel contained in the candidate region that was divided by said binarization unit;

a parameter surface drawing unit for, based on the combination of pose parameters computed by said parameter computation unit, drawing parameter surfaces in accordance with said surface parameter formulas on parameter space, which is space that includes each of the parameters belonging to said pose parameters as the axes of base vectors;

an intersection detection unit for finding cross point coordinates, which are the coordinates of intersections through which pass a plurality of said parameter surfaces that were drawn by said parameter surface drawing unit, and a cross surface number, which is the number of parameter surfaces that pass through the cross point coordinates;

a significant candidate extraction unit for, when the cross surface number at each cross point that was found by said intersection detection unit is compared with cross surface numbers at other cross point coordinates in prescribed neighboring regions that take as a reference the cross point coordinate corresponding to the cross surface number and is thus determined to be a maximum, identifying the combination of pose parameters that make up the parameter surface that corresponds to the determined cross surface number as a significant candidate that is an optimum combination of pose parameters for expressing said reference surface;

a significant candidate surface drawing unit for, based on a significant candidate identified by said significant candidate extraction unit and a combination of pose parameters that were computed by said parameter computation unit for which distance to said significant candidate in parameter space in prescribed neighboring regions that take the significant candidate as a reference is no greater than a prescribed value, drawing significant candidate surfaces on said image;

a cross determination unit for, regarding each significant candidate surface that was drawn by said significant candidate surface drawing unit, identifying pixels that are located on, of said candidate region, the significant candidate surface, and pixels that are located within a prescribed range from the significant candidate surface as cross pixels and finding coordinates of the cross pixels and tangent planes at said cross pixels;

a weighting computation unit for, based on angles that are formed by tangent planes in cross pixels that were found by said cross determination unit and planes that are formed by the axes of each base vector included in said parameter space, finding a slope weighting for each cross pixel that was identified by said cross determination unit;

a number measurement unit for computing for each of said significant candidate surfaces a number that is the value obtained by adding up the slope weightings of cross pixels that were found by said weighting computation unit and that were contained in each of the significant candidates; and

an actual surface determination unit for, when the number computed by said number measurement unit is compared with a number belonging to other significant candidate surfaces in neighboring regions that take as a reference the significant candidate surface that has this number and that thus is determined to be a maximum, identifying the significant candidate surface having said determined number as an actual surface that is a reference surface that actually exists in said image.

2. The data processing device according to claim 1, wherein, of said pose parameters, pose parameters that indicate direction are angles.

3. The data processing device according to claim 2, wherein:

said prescribed function for implementing a transform upon said angles is a trigonometric function; and

said surface parameter formula expression unit finds said surface parameter formulas based on values obtained by implementing a transform by said trigonometric function upon said angles and pose parameters other than said angles.

4. The data processing device according to claim 1, wherein said significant candidate extraction unit compares the cross surface number at each cross point found by said intersection detection unit and a first threshold value and identifies said significant candidates from among a combination of said pose parameters that corresponds to cross surface numbers determined to be greater than the first threshold value.

5. The data processing device according to claim 4, wherein said significant candidate extraction unit, when extracting said significant candidates, alters a significant candidate surface bit for a combination of pose parameters that was extracted as the significant candidate to a prescribed value that identifies said significant candidates as a significant candidate surface.

6. The data processing device according to claim 1, wherein said weighting computation unit:

at each point in a significant candidate surface that was drawn by said significant candidate surface drawing unit, finds angles formed by the tangent plane at that point and surfaces formed by the axes of each base vector included in said parameter space; and

determines, of said angles that were found, the angle for which the absolute value is a minimum as the minimum angle, and, by computing the absolute value of the reciprocal of the cosine for the determined minimum angle, finds a slope weighting of a pixel at which said significant candidate surfaces cross.

7. The data processing device according to claim 1, wherein said actual surface determination unit compares the numbers computed by said number measurement unit and a second threshold value, and identifies said actual surface from among significant candidate surfaces having a number that is determined to be greater than the second threshold value.

8. The data processing device according to claim 7, wherein said actual surface determination unit, when identifying said actual surface, alters an actual surface bit for said pose parameter combination, that forms a significant candidate surface identified as the actual surface, to a prescribed value that identifies said significant candidate surface as an actual surface.

9. The data processing device according to claim 1, further comprising a smoothing unit for implementing a process to smooth said image received as input, wherein said binarization unit, based on an image that has undergone smoothing by said smoothing unit, generates a binarized image.

10. The data processing device according to claim 1, further comprising a dilation/erosion unit for subjecting a candidate region of a binarized image generated by said binarization unit to a prescribed number of dilation processes for dilating the candidate region or erosion processes for eroding the candidate region.

11. The data processing device according to claim 10, further comprising:

an image size conversion unit for changing the image size by reducing or enlarging the size of any image; and

a parameter micro-adjustment unit for causing the alternate and repeated execution of changes of image sizes of differing magnifications by said image size conversion unit and identification of a new actual surface by said actual surface determination unit until a prescribed number is reached or until a prescribed accuracy is obtained;

wherein said image size conversion unit;

before said actual surface determination unit identifies an actual surface, generates a reduced image by reducing at a prescribed reduction ratio said image received as input; and

after said actual surface determination unit identifies an actual surface, generates a new image by changing said reduced image that was generated to a size that is smaller than said image received as input, and moreover, larger than the reduced image.

12. The data processing device according to claim 1, further comprising:

a parameter space coarse-graining unit for increasing the discretization width of said parameter space to a predetermined width to coarsen the parameter space and find an actual surface, and after once finding an actual surface, changing discretization to make the parameter space finer; and

a parameter micro-adjustment unit for repeatedly executing derivation of new actual surfaces in the parameter space of different discretization widths in neighboring regions of an actual surface that has been found in parameter space that has undergone said discretization until reaching a prescribed number of processes or until the discretization width reaches a prescribed value.

13. The data processing device according to claim 1, further comprising:

a positional relation estimation unit for, when a plurality of actual surfaces appear in said image that is received as input, estimating both the positional relation between each of said actual surfaces and identifying the actual surfaces by placing said actual surfaces in correspondence with physical entities; and

a pose significance unit for carrying out a significance process, which is a process of, based on the positional relation of said plurality of surfaces estimated by said positional relation estimation unit, placing said pose parameters in correspondence with said actual surfaces to determine the pose of an object of pose estimation with respect to the actual surfaces.

14. The data processing device according to claim 1, further comprising:

a pixel value-weighted parameter surface drawing unit for, based on a pixel value weighting found by multiplication of a combination of pose parameters that was computed by said parameter computation unit and a value computed based on pixel values of an image before binarization by said binarization unit, drawing on said parameter space parameter surfaces that accord with said surface parameter formulas;

a pixel value-weighted significant candidate extraction unit for, when at said cross point coordinates, the sum of the number of cross surfaces at which said parameter surfaces intersect each other and pixel value weightings is greater than a prescribed value, and moreover, when the number of cross surfaces are a maximum in neighboring regions that take the cross point coordinates as a reference, identifying the combination of parameters at the cross point coordinates as a significant candidate of a combination of parameters that form a reference surface that is the object of detection; and

a pixel value-weighted weighting computation unit for, based on angles formed by tangent planes at cross pixels that were found by said cross determination unit and surfaces that were formed by axes of each base vector included in said parameter space, finding a slope weighting for each cross pixel that was identified by said cross determination unit, and, by multiplying these slope weightings that were found and values that were computed based on pixel values of an image before binarization by said binarization unit, finding pixel value-weighted slope weightings.