Apparatus for measuring three-dimensional position, method thereof, and program

• US 9,141,873 B2
• Filed: 12/26/2012
• Issued: 09/22/2015
• Est. Priority Date: 12/27/2011
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Takemoto, Kazuki
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): Azima, Shaghayegh
• Application Number:
  US13/727,334
• Publication Number:
  US 20130163883A1
• Time in Patent Office: 1,000 Days
• Field of Search: 382/199, 382/181, 382/278, 382/282, 382/154, 382/128
• US Class Current: 1/1
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition

Abstract

An information processing apparatus includes an input unit configured to input a plurality of images captured from a plurality of viewpoints, an extraction unit configured to extract a region of an object from each of the plurality of images, an acquisition unit configured to obtain a contour from the region of the object, a smoothing unit configured to perform smoothing of the contour based on a point group on the obtained contour, a correlation unit configured to correlate regions of the object extracted from respective ones of the plurality of images, and a calculation unit configured to calculate a position of the object based on information of regions correlated by the correlation unit and the point group obtained by the smoothing unit.

Claims

1. An information processing apparatus comprising:

an input unit configured to input a plurality of images captured from a plurality of viewpoints;

an extraction unit configured to extract a region of an object from each of the plurality of images based on color information;

an acquisition unit configured to obtain a contour from each region extracted by the extraction unit;

a smoothing unit configured to obtain a plurality of smoothed contours by performing smoothing of each of the contours;

a correlation unit configured to correlate points on the plurality of smoothed contours between the images; and

a deriving unit configured to derive a three-dimensional coordinate of the points correlated by the correlation unit between the images,

wherein the smoothing unit includes an endpoint setting unit configured to set a first point and a second point on the contour and a setting unit configured to set a third point between the first point and the second point on the contour,

wherein the smoothing unit is configured to repeat an operation of the endpoint setting unit and an operation of the setting unit so that the third point is recursively set as the first point or the second point until a distance between the first point or the second point and the third point is smaller than a predetermined value and that the recursively set point between the first point and the second point is newly set as the third point, and

wherein the smoothing unit is configured to obtain, as the smoothed contours, a group of lines connecting adjacent points on the contours among the points set by the endpoint setting unit and the setting unit.

2. The information processing apparatus according to claim 1, wherein the setting unit is configured to set the third point at a midpoint between the first point and the second point on the contours.

3. The information processing apparatus according to claim 1, wherein the smoothing unit smoothes the contour by connecting points in the point group on the contour by line segments.

4. The information processing apparatus according to claim 1, wherein the correlation unit sets scanning lines that pass through same positions among the plurality of images, and detects corresponding points between the plurality of images based on intersections between the scanning lines and the smoothed contours.

7. The information processing apparatus according to claim 1, further comprising:

an intersection detection unit configured to detect intersections between a plurality of scanning lines corresponding to one another among the plurality of images and the smoothed contours;

an attribute determination unit configured to determine an attribute of each of the intersections detected by the intersection detection unit, the attribute indicating on which side of a scanning direction of each of the plurality of scanning lines a region having, as a contour, a smoothed contour passing through each of the intersections along each of the plurality of scanning lines is located,

wherein the correlation unit is configured to correlate points having a correspondence relation between the images among the intersections based on the attribute of each of the intersections determined by the attribute determination unit.

8. The information processing apparatus according to claim 7,

wherein the correlation unit includes:

a first degree-of-coincidence calculation unit configured to calculate a first degree of coincidence of the scanning lines by comparing the attribute of the intersection in each of the scanning lines corresponding to one another,

a first corresponding point determination unit configured to determine corresponding points having a correspondence relation between the images among the detected intersections on the scanning lines based on the first degree of coincidence, and

a second corresponding point determination unit configured to determine, as corresponding points, other intersections on the smoothed contours passing through the intersections determined as the corresponding points among the intersections that are not determined as the corresponding points by the first corresponding point determination unit.

9. The information processing apparatus according to claim 8, wherein the correlation unit further includes:

a scanning line segmentation unit configured to generate a plurality of segmented scanning lines by segmenting each of the scanning lines at positions of the corresponding points determined as corresponding points by the first corresponding point determination unit or the second corresponding point determination unit;

a second degree-of-coincidence calculation unit configured to calculate a second degree of coincidence of the segmented scanning lines between the images by comparing the attributes of the intersections between the images for each of the segmented scanning lines; and

a third corresponding point determination unit configured to determine corresponding points having a correspondence relation between the images among the intersections based on the second degree of coincidence.

10. The information processing apparatus according to claim 7, wherein the scanning lines are epipolar lines.

11. The information processing apparatus according to claim 1, further comprising a generation unit configured to generate a virtual object representing the object based on the three-dimensional coordinate of the points derived by the deriving unit.

12. The information processing apparatus according to claim 1, further comprising a combining unit configured to combine an image including the object with the virtual object based on the three-dimensional coordinate of the points derived by the deriving unit.

13. The information processing apparatus according to claim 1, wherein the image is an image captured by an imaging apparatus attached to a head-mounted display or a handheld display.

5. A method for performing information processing comprising:

inputting a plurality of images captured from a plurality of viewpoints;

extracting a region of an object from each of the plurality of images based on color information;

obtaining a contour from each extracted region;

obtaining a plurality of smoothed contours by performing smoothing of each of the contours;

correlating points on the plurality of smoothed contours between the images; and

deriving a three-dimensional coordinate of the points correlated by the correlation unit between the images,

wherein the obtaining a plurality of smoothed contours sets a first point and a second point on the contour, and sets a third point between the first point and the second point on the contour,

wherein the obtaining a plurality of smoothed contours repeats an operation of the setting the first point and the second point, and an operation of setting the third point so that the third point is recursively set as the first point or the second point until a distance between the first point or the second point and the third point is smaller than a predetermined value and that the recursively set point between the first point and the second point is newly set as the third point, and

wherein the obtaining a plurality of smoothed contours obtains, as the smoothed contours, a group of lines connecting adjacent points on the contours among the points set by the operation of the setting the first point and the second point, and the operation of setting the third point.

6. A non-transitory computer-readable storage medium storing a program for causing a computer to perform each processing in a method for performing information processing comprising:

inputting a plurality of images captured from a plurality of viewpoints;

extracting a region of an object from each of the plurality of images based on color information;

obtaining a contour from each extracted region;

obtaining a plurality of smoothed contours by performing smoothing of each of the contours;

correlating points on the plurality of smoothed contours between the images; and

deriving a three-dimensional coordinate of the points correlated by the correlation unit between the images,

wherein the obtaining a plurality of smoothed contours sets a first point and a second point on the contour, and sets a third point between the first point and the second point on the contour,

wherein the obtaining a plurality of smoothed contours repeats an operation of the setting the first point and the second point, and an operation of setting the third point so that the third point is recursively set as the first point or the second point until a distance between the first point or the second point and the third point is smaller than a predetermined value and that the recursively set point between the first point and the second point is newly set as the third point, and

wherein the obtaining a plurality of smoothed contours obtains, as the smoothed contours, a group of lines connecting adjacent points on the contours among the points set by the operation of the setting the first point and the second point, and the operation of setting the third point.