Color edge based system and method for determination of 3d surface topology

• US 20070085849A1
• Filed: 09/09/2004
• Published: 04/19/2007
• Est. Priority Date: 09/11/2003
• Status: Active Grant

• Current Assignee: C.M.D. Controlled Micro Devices Ltd., Cyber Vision (2004) Ltd.
• Original Assignee: C.M.D. Controlled Micro Devices Ltd., Cyber Vision (2004) Ltd.
• Inventors: Kosolapov, Samuel
• Granted Patent: US 7,456,842 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G01B 11/2509 Color coding

Abstract

A method and system for determining surface topology of a three-dimensional (3D) structure, based on a structured pattern that is projected onto the surface structure, and images of the pattern superposed on the structured surface are analysed to provide surface coordinates of the structure. The pattern comprises a plurality of unique color edges defined between pairs of differently-colored stripes, which substantially overcomes ambiguity problems. In one embodiment, a calibration method is provided enabling the surface coordinates to be obtained from a single image of the structure.

Claims

1-51. -51. (canceled)

52. A structured light pattern for use in the determination of 3D surface topology of a 3D structure, wherein said pattern may be projected onto such a structure and 3D surface topology thereof is determined from at least one image obtained of the structure with said pattern reflected therefrom, the pattern comprising a plurality of pairs of colored stripes, wherein each said pair comprises:

two juxtaposed discrete stripes, one said stripe being of a different color from the other stripe; and

a unique color edge defining a unique color transition boundary between the colors of said two adjacent stripes in a predetermined direction substantially orthogonal to a longitudinal direction of said stripes;

wherein each said color edge is sufficiently unique by itself within said pattern such as to enable identification of each color edge of said pattern when said pattern is reflected from said 3D structure.

53. The pattern according to claim 52, wherein said colors of said stripes in each pair are chosen to emphasize color contrast between one stripe and the other stripe of said pair.

54. The pattern according to claim 52, wherein each said stripe comprises a color that is substantially uniform within the stripe.

55. The pattern according to claim 52, wherein the pattern is based on said stripes being any one of three basic colors.

56. The pattern according to claim 52, wherein said stripes are colored exclusively in colors chosen from the group:

red, blue, green, magenta, cyan and yellow.

57. The pattern according to claim 52, further comprising at least one spacer stripe between each adjacent said pair of stripes, said at least one spacer stripe being of a color having a predetermined color contrast with a said stripe of each one of said adjacent pairs of stripes that is adjacent to said at least one spacer stripe.

58. The pattern according to claim 57, wherein said spacer stripes comprise any one of:

a white stripe between adjacent pairs of stripes;

or

a black stripe between adjacent pairs of stripes;

or

a group of stripes, comprising a black stripe sandwiched between two white stripes, between adjacent pairs of stripes.

59. The pattern according to claim 58, further comprising a trailer comprising at least one stripe adjacent to at least one or another of a first and a last said pairs taken along said direction.

60. The pattern according to claim 59, wherein said trailer stripes comprise a group, comprising any one of

a sequence of two black stripes, a white stripe, a black stripe, a white stripe and a black stripe adjacent to said first pair of stripes;

or

a sequence of a black stripe, a white stripe, a black stripe, a white stripe and a black stripe adjacent to said last pair of stripes.

61. The pattern according to claim 59, wherein said pattern comprises the following color stripe sequence:

bbwbwb wbwRGwbwBG wbwRBwbwGB wbwBRwbwGRwbw bwbwb

wherein;

b=black stripe

w=white stripe

R=red stripe

B=blue stripe

G=green stripe

62. The pattern according to claim 59, wherein said pattern comprises the following color stripe sequence:

wbwRGwbwRBwbwRMwbwRCwbwRY wbwGRwbwGBwbwGMwbwGCwbwGY wbwBRwbwBGwbwBMwbwBCwbwBY wbwMRwbwMGwbwMBwbwMCwbwMY wbwCRwbwCGwbwCBwbwCMwbwCY wbwYRwbwYGwbwYBwbwYMwbwYCwbw

wherein;

b=black stripe

w=white stripe

R=red stripe

B=blue stripe

G=green stripe

M=Magenta

C=Cyan

Y=Yellow.

63. Method for determining 3D surface topology of a 3D structure, comprising:

—

(a) illuminating said structure with a structured pattern comprising a plurality of pairs of colored stripes, wherein each said pair comprises;

two juxtaposed discrete stripes, one said stripe being of a different color from the other stripe; and

a unique color edge defining a unique color transition boundary between the colors of said two adjacent stripes in a predetermined direction substantially orthogonal to a longitudinal direction of said stripes;

wherein each said color edge is sufficiently unique by itself within said pattern such as to enable identification of each color edge of said pattern when said pattern is reflected from said 3D structure;

(b) capturing at least two images of the structure on which the said structured pattern is being shone;

(c) identifying corresponding unique color edges between said images;

(d) applying a suitable mathematical technique for determining surface coordinates of said structure from the correspondence found in step (c).

64. Method according to claim 63, wherein in step (d), said mathematical technique includes any one of a triangulation method and an epipolar method.

65. Method for determining surface topology of a 3D structure, comprising:

—

a. providing a calibration image of a structured pattern, comprising a plurality of geometrical portions including a plurality of pairs of colored stripes, wherein each said pair comprises;

two juxtaposed discrete stripes, one said stripe being of a different color from the other stripe; and

a unique color edge defining a unique color transition boundary between the colors of said two adjacent stripes in a predetermined direction substantially orthogonal to a longitudinal direction of said stripes;

wherein each said color edge is sufficiently unique by itself within said pattern such as to enable identification of each color edge of said pattern when said pattern is reflected from said 3D structure;

said calibration image being reflected from a calibration object at each one of at least two calibration stops serially spaced along an optical axis at predetermined depths from a source projecting said structured pattern;

(ii) for each said calibration image, identifying unique portions of said structured pattern corresponding to predetermined portions of said structured pattern in another said image, and determining spatial coordinates associated with said unique portions;

(iii) aligning said structure along said optical axis, projecting said structured light pattern onto a surface of said structure wherein said pattern is modified by virtue of the topology of the structure surface such that to provide a modified pattern superposed over said structure surface comprising a plurality of modified geometrical portions corresponding to said geometrical portions;

(iv) providing at least one complex image of said structure surface with said modified pattern superposed thereon, and for each said complex image identifying unique modified geometrical portions corresponding to said geometrical portions; and

(v) for each said unique modified portion, determining spatial coordinates associated therewith from the spatial coordinates obtained in (ii) for the corresponding unique portion of each said calibration image, to provide surface coordinates of said structure.

66. Method according to claim 65, comprising:

—

(a) providing a calibration image of a structured pattern reflected from a calibration object at each one of at least two calibration stops serially spaced along an optical axis at predetermined depths from a source projecting said structured pattern, said structured light pattern comprising a plurality of pairs of colored stripes, wherein each said pair comprises;

two juxtaposed discrete stripes, one said stripe being of a different color from the other stripe; and

a unique color edge defining a unique color transition boundary between the colors of said two adjacent stripes in a predetermined direction substantially orthogonal to a longitudinal direction of said stripes;

wherein each said color edge is sufficiently unique by itself

such as to enable identification of each color edge of said pattern when said pattern is reflected from said 3D structure;

(b) for each said calibration image, identifying unique color edges between stripes of each said pair of stripes and determining spatial coordinates associated with said unique color edges;

(c) aligning said structure along said optical axis, projecting said structured light pattern onto a surface of said structure wherein said pattern is modified by virtue of the topology of the structure surface such that to provide a modified pattern superposed over said structure surface comprising a plurality of modified color edges corresponding to said color edges;

(d) providing at least one complex image of said structure surface with said modified pattern superposed thereon, and for each said complex image identifying unique modified color edges between stripes of each said pair of stripes with respect to said color edges of said pattern; and

(e) for each said unique modified color edge, determining spatial coordinates associated therewith from the spatial coordinates obtained in (b) for the corresponding unique color edge of each said calibration image, to provide surface coordinates of said structure.

67. Method according to claim 66, wherein said structure surface has depth values from said source between a first depth and a second depth, and wherein one said calibration stop is at a depth from said source not greater than said first depth and another said calibration stop is at a depth from said source not less than said second depth, wherein said first depth is less than said second depth.

68. Method according to claim 66, wherein steps (a) and (b) comprise the sub steps:

(i) Providing a storage array for storing a group of {X,Y,Z} coordinates of a plurality of calibration points and a group of a plurality of {Row,Column} pairs that represent said calibration points in said image grabbing apparatus; and

also for storing the {X,Y,Z} coordinates and the {Row,Column} pairs that relate to the same calibration points;

(ii) Performing a first step of calibration, including;

(b.1) utilization of 2D calibration grid, for determining {X,Y,Z} coordinates of a plurality of calibration points residing on the 2D calibration grid when placed at a predetermined calibration stop, and finding for each calibration point a corresponding {Row,Column} pair that represents said calibration point in said image grabbing apparatus, the determined {X,Y,Z} coordinates and the corresponding {Row,Column} pairs, which represent the calibration points that reside on the same (X-Y) plane, being stored in said storage array;

(b.2) repeating step (b.1), with the 2D calibration grid placed at another one or more different calibration stops, in order to obtain calibration data that represent calibration points that reside on two or more different and parallel planes, said calibration points, which relate to every calibration stop, defining a work space and the calibration data relating to calibration points in said work space being a first calibration data set that is stored in said storage array;

(b.3) calculating, from the stored first calibration data set, a set of coefficients that represents the relationships, for each calibrating stop, between its {X,Y,Z} coordinates in the 2D calibration grid and the {Row,Column} pair that represents said calibration point in said image grabbing apparatus, said set of coefficients allowing obtaining the {X,Y,Z} coordinates of essentially any point, given a {Row,Column} pair representation of said point, by manipulating data that corresponds to calibration points that are, preferably, the closest to said point;

(iii) Placing a white planar surface at several calibration stops, as described in step (ii), while at each calibration stop performing;

c.1) Projecting a unique color pattern on said white planar surface that is located at two or more predetermined calibration stops;

c.2) Grabbing, by said image grabbing apparatus and for each calibration stop, a 2D image that is the reflection of the unique color pattern from said white planar surface;

c.3) Identifying, in the grabbed 2D image and for each calibration stop, points that represent unique calibration color edges that are contained in said unique color pattern, the identified points having corresponding {Row,Column} pairs that are stored in the storage array, each unique calibration color edge being represented by two or more sets of {Row,Column} pairs, each one of said sets of {Row,Column} pairs being obtained for each calibration stop, and all of the sets of {Row,Column} pairs being a second calibration data set that is stored in said storage array; and

c.4) Using the coefficients found in step b.3) to translate the sets of {Row,Column} pairs to corresponding {X,Y,Z} coordinates.

69. Method according to claim 68, wherein steps (c) to (e) comprise:

d.1) placing said object in said work space and projecting the unique color pattern on said object;

d.2) grabbing a 2D complex image that includes said color pattern, as deformed by said surface of the object, that is superimposed on said surface of said object;

d.3) identifying, in said complex image, {Row,Column} pairs of points that represent the unique color edges that are included in the deformed color pattern; and

d.4) Utilizing said coefficients for translating the {Row,Column} pairs identified in step d.3) into corresponding {X,Y,Z} coordinates, for obtaining 3D model of said surface of the object.

75. A computer readable medium storing instructions for programming a microprocessor means of a system adapted for determining 3D surface topology of a 3D structure to perform a method as defined in claim 65.

70. A system for the determination of 3D surface topology of a 3D structure, comprising:

(A) illumination means adapted for projecting a beam of structured light onto at least said portion, said structured light having a pattern comprising a plurality of pairs of colored stripes, wherein each said pair comprises;

two juxtaposed discrete stripes, one said stripe being of a different color from the other stripe; and

a unique color edge defining a unique color transition boundary between the colors of said two adjacent stripes in a predetermined direction substantially orthogonal to a longitudinal direction of said stripes;

wherein each said color edge is sufficiently unique by itself within said pattern such as to enable identification of each color edge of said pattern when said pattern is reflected from said 3D structure;

(B) at least one image capture means for capturing at least one image of said portion with said structured pattern projected thereon;

(C) processing means adapted for determining surface topology of at least said portion based on said at least one image provided by said at least one image capture means.

71. System according to claim 70, wherein said illumination means comprises a light source, focusing optics and a slide comprising said structured pattern.

72. System according to claim 71, wherein said illumination means is comprised in a housing, wherein said housing further optionally comprises said image capture means, and wherein said image capture means optionally comprises a digital image grabbing apparatus.

73. System according to claim 72, further comprising a calibration apparatus, comprising a calibration object movable between at least a first calibration station and a second calibration station defining a workspace therebetween in which the said portion of said structure is to be positioned, wherein said processing means is adapted for determining depth values associated with said portion based on depth values of said calibration object at said at least first and second calibration stations.

74. System according to claim 73, wherein said calibration object comprises any one of:

a two-dimensional calibration grid including a plurality of parallel vertical lines and parallel horizontal lines that form an X-Y plane, the intersection points of said horizontal lines and said vertical lines are the calibration points;

or

a white surface on which a plurality of colored calibration markers are printed, painted, etched or adhered, each colored calibration marker comprising a pair of bars, each bar having different color for providing a unique color edge.