APPARATUS, METHOD AND PROGRAM FOR THREE-DIMENSIONAL-SHAPE DETECTION

• US 20070031029A1
• Filed: 09/28/2006
• Published: 02/08/2007
• Est. Priority Date: 03/31/2004
• Status: Active Grant

• Current Assignee: Brother Kogyo Kabushiki Kaisha
• Original Assignee: Brother Kogyo Kabushiki Kaisha
• Inventors: Sasaki, Hiroyuki
• Granted Patent: US 7,672,505 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G06T 1/0007 Image acquisition

Abstract

The present invention relates to three-dimensional shape detection. In the present invention, a plurality of types of pattern lights formed of a series of alternate light and dark patterns are projected onto an object in a time series, an image of the object onto which each pattern light is projected is taken, a plurality of luminance images are generated, a code image having certain codes assigned to the pixels is generated in accordance with a result of threshold processing of the plurality of luminance images with respect to a certain threshold; and the three-dimensional shape of the object is calculated. Further, in the invention, a first pixel that is adjacent to a pixel having a code of interest and that has a code different from the code of interest is detected, in a detection position in a direction crossing the pattern light in the code image, a luminance image having a light-dark boundary in a position corresponding to the first pixel is extracted, from the plurality of luminance images, a pixel area that includes a pixel in a certain area adjacent to the first pixel is determined, an approximate expression that expresses a change in luminance in the extracted luminance image in the pixel area is calculated, a position having a certain luminance threshold in the approximate expression and detecting the boundary coordinates of the code of interest in accordance with the result of calculation is calculated, and the three-dimensional shape of the object is calculated in accordance with the boundary coordinates detected by the boundary coordinate detection unit by using the code image.

Claims

1. A three-dimensional-shape detection apparatus comprising:

a projection unit that projects a plurality of types of pattern lights formed of a series of alternate light and dark patterns, onto an object in a time series;

an imaging unit that takes an image of the object onto which each pattern light is projected by the projection unit;

a luminance image generation unit that generates a plurality of luminance images by calculating the luminance of each pixel from the image taken by the imaging unit;

a code image generation unit that generates a code image having certain codes assigned to the pixels in accordance with a result of threshold processing of the plurality of luminance images generated by the luminance image generation unit with respect to a certain threshold;

a three-dimensional-shape calculation unit that calculates the three-dimensional shape of the object by utilizing the code image generated by the code image generation unit;

a first-pixel detection unit that detects a first pixel that is adjacent to a pixel having a code of interest and that has a code different from the code of interest, in a detection position in a direction crossing the pattern light in the code image;

a luminance image extraction unit that extracts a luminance image having a light-dark boundary in a position corresponding to the first pixel detected by the first-pixel detection unit, from the plurality of luminance images;

a pixel area determination unit that determines a pixel area that includes a pixel in a certain area adjacent to the first pixel;

an approximate expression calculation unit that calculates an approximate expression that expresses a change in luminance in the extracted luminance image in the pixel area determined by the pixel area determination unit; and

a boundary coordinate detection unit that calculates a position having a certain luminance threshold in the approximate expression calculated by the approximate expression calculation unit and detecting the boundary coordinates of the code of interest in accordance with the result of calculation,

wherein the three-dimensional-shape calculation unit calculates the three-dimensional shape of the object in accordance with the boundary coordinates detected by the boundary coordinate detection unit by using the code image.

2. The three-dimensional-shape detection apparatus according to claim 1, wherein the boundary coordinates of the code of interest, detected by the boundary coordinate detection unit, are calculated with a resolution higher than the resolution of the imaging unit.

3. The three-dimensional-shape detection apparatus according to claim 1, wherein the pixel area determination unit determines the pixel area from one or more successive rows of pixels starting from the row of pixels including the first pixel, besides the row of pixels including the first pixel, with reference to the code image;

the approximate expression calculation unit calculates the approximate expression for each row of pixels in the pixel area; and

the boundary coordinate detection unit calculates a position having a certain luminance threshold for each approximate expression and detects the boundary coordinates of the code of interest from the result of calculation.

4. The three-dimensional-shape detection apparatus according to claim 1, wherein the luminance image extraction unit extracts one or more luminance images that include the luminance image;

the pixel area determination unit determines the pixel area for each extracted luminance image, with reference to the code image;

the approximate expression calculation unit calculates the approximate expression for each row of pixels in the pixel area detected from each extracted luminance image; and

the boundary coordinate detection unit calculates a position having a certain luminance threshold for each approximate expression and detects the boundary coordinates of the code of interest from the result of calculation.

5. The three-dimensional-shape detection apparatus according to claim 1, wherein the luminance image extraction unit extracts one or more luminance images that include the luminance image;

the pixel area determination unit determines the pixel area for each extracted luminance image from one or more successive rows of pixels starting from the row of pixels containing the first pixel, besides the row of pixels containing the first pixel, with reference to the code image;

the approximate expression calculation unit calculates the approximate expression for each row of pixels in the pixel area for each extracted luminance image; and

the boundary coordinate detection unit calculates a position having a certain luminance threshold for each approximate expression for each extracted luminance image and detects the boundary coordinates of the code of interest from the result of calculation.

6. The three-dimensional-shape detection apparatus according to claim 3, wherein the boundary coordinate detection unit calculates a weighed average or the median value of positions calculated for the approximate expressions and detects the boundary coordinates of the code of interest from the result of calculation.

7. The three-dimensional-shape detection apparatus according to claim 3, wherein the boundary coordinate detection unit calculates an approximate expression of positions calculated for the approximate expressions and detects the boundary coordinates of the code of interest from the result of calculation.

8. The three-dimensional-shape detection apparatus according to claim 5, wherein the boundary coordinate detection unit calculates an approximate expression of positions calculated for the approximate expressions in each extracted luminance image, calculates the boundary coordinates of the code of interest from the result of calculation, calculates a weighed average or the median value of the boundary coordinates of the code of interest calculated in the extracted luminance images, and detects the boundary coordinates of the code of interest from the result.

9. A three-dimensional-shape detection method comprising:

a projection step of projecting a plurality of types of pattern lights formed of a series of alternate light and dark patterns, onto an object in a time series;

an imaging step of taking an image of the object onto which each pattern light is projected in the projection step;

a luminance image generation step of generating a plurality of luminance images by calculating the luminance of each pixel from each image taken in the imaging step;

a code image generation step of generating a code image having certain codes assigned to the pixels in accordance with a result of threshold processing of the plurality of luminance images generated in the luminance image generation step with respect to a certain threshold;

a three-dimensional-shape calculation step of calculating the three-dimensional shape of the object by utilizing the code image generated in the code image generation step;

after the code image generation step, a first-pixel detection step of detecting a first pixel that is adjacent to a pixel having a code of interest and that has a code different from the code of interest, in a detection position in a direction crossing the pattern light in the code image;

a luminance image extraction step of extracting a luminance image having a light-dark boundary in a position corresponding to the first pixel detected in the first-pixel detection step, from the plurality of luminance images;

a pixel area determination step of determining a pixel area that includes a pixel in a certain area adjacent to the first pixel;

an approximate expression calculation step of calculating an approximate expression that expresses a change in luminance in the extracted luminance image in the pixel area determined in the pixel area determination step; and

a boundary coordinate detection step of calculating a position having a certain luminance threshold in the approximate expression calculated in the approximate expression calculation step and detecting the boundary coordinates of the code of interest in accordance with the result of calculation,

wherein the three-dimensional-shape calculation step calculates the three-dimensional shape of the object in accordance with the boundary coordinates detected in the boundary coordinate detection step by using the code image.

10. A computer usable medium having computer readable instructions stored thereon, which, when executed by a computer, are configured to:

project a plurality of types of pattern lights formed of a series of alternate light and dark patterns, onto an object in a time series;

take an image of the object onto which each pattern light is projected;

generate a plurality of luminance images by calculating the luminance of each pixel from the taken image;

generate a code image having certain codes assigned to the pixels in accordance with a result of threshold processing of the plurality of luminance images with respect to a certain threshold;

calculate the three-dimensional shape of the object by utilizing the code image;

detect a first pixel that is adjacent to a pixel having a code of interest and that has a code different from the code of interest, in a detection position in a direction crossing the pattern light in the code image;

extract a luminance image having a light-dark boundary in a position corresponding to the first pixel, from the plurality of luminance images;

determine a pixel area that includes a pixel in a certain area adjacent to the first pixel;

calculate an approximate expression that expresses a change in luminance in the extracted luminance image in the pixel area; and

calculate a position having a certain luminance threshold in the approximate expression and detecting the boundary coordinates of the code of interest in accordance with the result of calculation,

wherein the three-dimensional shape of the object is calculated in accordance with the boundary coordinates detected by using the code image.