Method and system for acquiring a three-dimensional shape description

• US 6,510,244 B2
• Filed: 03/16/1999
• Issued: 01/21/2003
• Est. Priority Date: 06/13/1996
• Status: Expired due to Fees

• Current Assignee: Eyetronics NV
• Original Assignee: KU Leuven Research & Development
• Inventors: Proesmans, Marc R. A. B., Defoort, Filip P., Oosterlinck, André J. J., Van Gool, Luc S. J.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/202,184
• Publication Number:
  US 20020057832A1
• Time in Patent Office: 1,407 Days
• Field of Search: 382/106, 382/108, 382/154, 382/203, 382/255, 382/151, 382/152, 382/204, 382/103, 382/181, 382/190, 382/206, 382/276, 382/282, 382/284, 382/288, 382/291, 382/293, 382/294, 382/295, 348/42, 348/169, 348/44, 345/419-427, 356/12, 356/13, 358/504, 702/85, 702/94, 702/95
• US Class Current: 382/203
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings

Abstract

Method for acquiring a three-dimensional shape or image of a scene, wherein a predetermined pattern of lines is projected onto the scene and the shape is acquired on the basis of relative distances between the lines and/or intersections of the lines of the pattern.

Claims

1. A method of determining a three dimensional shape of a free-form object, comprising the steps of:

(a) providing a pattern of components having a known relative positioning;

(b) projecting said components of the pattern simultaneously onto the free-form object;

(c) acquiring an image of the pattern projected onto the free-form object;

(d) selecting one component out of the acquired image as a reference component;

(e) determining relative positional coordinates of the pattern components other than the reference component in the acquired image with reference to the reference component; and

(f) determining relative depths of positions on said three-dimensional shape of the free-form object by linear transformations of the relative positional coordinates based on one of a pseudo-orthographic and an orthographic geometrical model.

2. A method according to claim 1 including calculating relative, spatial positions of points on the three dimensional object directly from relative, observed positions of the components in the image.

3. A method according to claim 1 comprising projecting one single pattern and reconstructing the three dimensional shape of the free-form object on the basis of a single acquired image.

4. A method according to claim 1, wherein a sequence of three dimensional shapes is reconstructed from images acquired sequentially in time.

5. A method according to claim 1, wherein at least one pattern of lines is projected onto the object and the three dimensional shape is acquired on the basis of observed, relative distances between the lines and/or intersections of the lines of the pattern.

6. A method according to claim 5, wherein the pattern of lines is a grid of straight lines.

7. A method according to claim 6, wherein the grid consists of two mutually perpendicular series of parallel lines.

8. A method according to claim 1 including a step of minimizing inaccuracies in the extracted three dimensional shape that are due to perspective effects in the pattern projection or in the acquisition of the image(s).

9. A method according to claim 5, including extracting pattern lines and/or intersections thereof from the image or images.

10. The method according to claim 1 further comprising the steps of:

determining a reliability of relative topological positioning of said components in said acquired image,

modifying the relative positions in response to the reliability determining step to generate further relative positions, and

determining relative depths of positions on said three-dimensional shape of the free-form object from the further relative position of said components in said acquired image.

11. A method according to claim 9, wherein the intersections of the lines are determined and the relative positions thereof in the pattern(s) are associated with relative sequential numbers.

12. A method according to claim 9, wherein positions of the lines are determined with a a precision better than of an imaging device used to acquire the image.

13. A method according to claim 12, wherein the positions of the lines are refined by treating them as a ‘

snake’

or through a regularization technique.

14. A method according to claim 9, wherein intersections are determined with a higher precision than of an imaging device used to acquire the image.

15. A method according to claim 1, wherein the correctness of the extracted pattern(s) is checked and/or corrections are applied to the extracted pattern(s) through the comparison of the pattern that is actually projected with the shape the extracted pattern would yield if its three dimensional interpretation were viewed from a position from which the pattern is projected.

16. A method according to claim 1, including filtering out the pattern(s) to obtain texture.

17. A method according to claim 1, wherein a code is added to the projected pattern(s).

18. A method according to claim 17, wherein the code is used to correctly process and/or identify depth discontinuities.

19. A method according to claim 1, wherein calibration is achieved by presenting an object and extracting its shape in order to derive system parameters.

20. A method according to claim 19, wherein an object is presented that contains two planes that subtend a known angle and that are used for calibration.

21. A method according to claim 19, including taking at least two views of the same object to calibrate system parameters.

22. A method according to claim 21, wherein system parameters are calibrated by exploiting the fact that upon reconstruction from every image the shape of the part visible in more than one of the views should be identical.

23. A method according to claim 21 or 22, including projecting a pattern on a larger part of the object than that for which the three dimensional shape is reconstructed from a single image, while using a connectivity of the pattern to facilitate the registration and/or combination of multiple, partial reconstructions obtained from multiple images.

24. A method according to claim 19, wherein the shape of the object is known and the system parameters are obtained by a comparison of the reconstructed shape with the known shape.

25. A method according to claim 1, wherein the calibration of the system only requires the determination of the height-width proportion (i.e. the ‘

aspect ratio’

) of the pixels of a device use for acquiring the image and the relative direction of the pattern projection and image acquisition.

26. A system for obtaining a three-dimensional shape of an image of a free-form object, comprising:

at least one pattern generator for simultaneously projecting a pattern of components having a known relative positioning;

at least one imaging device for acquiring an image of the pattern projected onto the free-form object; and

means for selecting one component out of the acquired image as a reference component,

means for determining relative positional coordinates of the pattern components other than the reference component in the acquired image with reference to the reference component; and

computing means for determining relative depths of positions on said three-dimensional shape of the free-form object by linear transformations of the relative positional coordinates based on one of a pseudo-orthographic and an orthographic geometrical model.

27. A system according to claim 26, wherein only one pattern generator and one imaging device are used.

28. A system according to claim 26, wherein the pattern generator and the imaging device have a relative arrangement such that at least a part of the pattern is observed by the imaging device, and the projection and imaging direction(s) do not coincide.

29. A system according to claim 26, further comprising:

means for determining a reliability of relative topological positioning of said components in said acquired image,

means for modifying the relative positions in response to the reliability determining step to generate further relative positions, and

means for determining relative depths of positions on said three-dimensional shape of the free-form object from the further relative position of said components in said acquired image.

30. A system according to claim 26, wherein the pattern generator comprises means for detecting radiation that is invisible to the human eye.

31. A system according to claim 26, wherein the pattern generator comprises means for detecting a combination of different wavelengths.

32. A system according to claim 26, wherein the pattern generator comprises means for detecting the pattern during a short period of time and/or with interruptions.

33. A system according to claim 26, wherein the pattern generator, the imaging device, the computing means and the means for determining reliability are integrated into a single apparatus.

34. A method for acquiring a three-dimensional shape or image of an object or a scene including free-form surfaces, wherein one or more predetermined patterns made up of intersecting pattern components having a known relative positioning are projected simultaneously onto the object or scene, the method comprising the steps of:

projecting a pattern of intersecting components onto the object or scene along a projection path,

acquiring an image of the object or scene along an image acquisition path as viewed from a predetermined angle with respect to the projection path,

determining the relative positions of intersection points of pattern components in the acquired image with reference to a reference point in the acquired image, and

reconstructing the three dimensional shape of object or scene on the basis of computing relative three-dimensional positions of the intersection points, the relative three-dimensional positions of the intersection points being derived from one of a pseudo-orthographic and an orthographic geometrical model comprising the intersections of first and second parallel beams of rays, the first parallel beam of rays representing pattern projection rays and the second parallel beam of rays representing the image projection rays, an included angle between the first and second parallel beams of rays being a predetermined angle.

35. A method according to claim 34, wherein the step of determining the relative positions of intersection points includes constructing intersection points where the pattern components in the acquired image are interrupted.

36. A method according to claim 34, wherein the step of determining the relative positions includes the step of determining the reliability of the relative positions of the intersection points.

37. A method according to claim 34, wherein the pattern components are lines.

38. A method according to claim 37, wherein the pattern of lines is a grid of straight lines.

39. A method according to claim 38, wherein the grid consists of two mutually perpendicular series of parallel lines.

40. A method according to 37, wherein intersections of the lines are determined and the relative positions thereof in the pattern(s) are associated with a label.

41. A method according to claim 40, wherein the labels are re-determined after a reliability determination.

42. A method according to claim 34, wherein the intersection points are determined with a precision higher than that of the size of a pixel of an imaging device used to acquire the image.

43. A method according to claim 37, wherein positions of the lines are extracted from the acquired image and these positions are refined by a regularization technique.

44. A method according to claim 34, wherein one single pattern is projected and a three-dimensional shape of the object is reconstructed on the basis of a single acquired image.

45. A method according to claim 34, wherein a sequence of three-dimensional shapes is reconstructed from images acquired sequentially in time.

46. A method according to claim 34, wherein the reconstruction step is based upon observed deformations of the pattern components in the acquired image allowing for perspective effects.

47. The method according to claim 34, wherein a surface texture of the object or scene is extracted from the acquired image.

48. A method according to claim 47, wherein a texture is obtained by filtering out the pattern.

49. A method according to claim 47, wherein the surface texture is extracted during the period when no pattern is projected.

50. A method according to claim 34, wherein a code is added to the projected pattern.

51. A method according to claim 34, further comprising a calibration step of presenting an object or a scene that contains two planes which subtend a known angle.

52. A method according to claim 51, wherein the calibration step includes determination of a height-width ratio of the pixels of a device for acquiring the image and the angle between the pattern projection path and image acquisition path.

53. A system for obtaining a three-dimensional shape of a scene or object including free-form surfaces, comprising:

at least one pattern generator for projecting a pattern of intersecting components having a known relative positioning onto the scene or object;

at least one camera for observing the scene or object and for acquiring one or more images;

comprising;

computing means for determining the relative positions of intersection points of pattern components in the acquired image with reference to a reference point in the acquired image, and for reconstructing the three-dimensional shape of the three-dimensional shape of object or scene on the basis of computing relative three-dimensional positions of the intersection points, the relative three-dimensional positions of the intersection points being derived from a pseudo-orthographic or orthographic geometrical model comprising the intersections of first and second parallel beams of rays, the first parallel beam of rays representing pattern projection rays and the second parallel beam of rays representing the image projection rays, an included angle between the first and second parallel beams of rays being a predetermined angle.

54. A system according to claim 53, wherein the system is adapted for determining the reliability of the relative positioning of the components in the acquired image.

55. A system according to claim 53 wherein only one pattern generator and one camera are used.

56. A system according to claim 53, wherein the pattern generator is adapted to project radiation that is invisible to the human eye.

57. A system according to claim 53, wherein the system is adapted to extract a surface texture of the object or scene from the acquired image.