METHOD AND APPARATUS FOR MEASURING THE THREE DIMENSIONAL STRUCTURE OF A SURFACE

• US 20150009301A1
• Filed: 01/30/2013
• Published: 01/08/2015
• Est. Priority Date: 01/31/2012
• Status: Abandoned Application

• Current Assignee: 3M Innovative Properties Company (3M Company)
• Original Assignee: 3M Innovative Properties Company (3M Company)
• Inventors: Ribnick, Evan J., Qiao, Yi, Lai, Jack W., Hofeldt, David L.
• Application Number:
  US14/375,002
• Publication Number:
  US 20150009301A1
• Time in Patent Office: Days
• US Class Current: 348/50
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/303 using photoelectric detection means
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30124 Fabrics; Textile; Paper
• CPC Class Codes: G06T 2207/30136 Metal
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

A method includes imaging a surface with at least one imaging sensor, wherein the surface and the imaging sensor are in relative translational motion. The imaging sensor includes a lens having a focal plane aligned at a non-zero angle with respect to an x-y plane of a surface coordinate system. A sequence of images of the surface is registered and stacked along a z direction of a camera coordinate system to form a volume. A sharpness of focus value is determined for each (x,y) location in the volume, wherein the (x,y) locations lie in a plane normal to the z direction of the camera coordinate system. Using the sharpness of focus values, a depth of maximum focus z_m along the z direction in the camera coordinate system is determined for each (x,y) location in the volume, and based on the depths of maximum focus z_m, a three dimensional location of each point on the surface may be determined.

Claims

1-15. -15. (canceled)

16. An apparatus, comprising:

an imaging sensor comprising a telecentric lens, wherein the lens has a focal plane aligned at a non-zero viewing angle with respect to an x-y plane in a surface coordinate system, wherein the surface and the imaging sensor are in relative translational motion, and wherein the sensor images the surface to form a sequence of images thereof;

a processor that;

aligns in each image in the sequence a reference point on the surface to form a registered sequence of images;

stacks the registered sequence of images along a z direction in a camera coordinate system to form a volume, wherein each image in the registered sequence of images comprises a layer in the volume;

computes a sharpness of focus value for each pixel within the volume, wherein the pixels lie in a plane normal to the z direction in the camera coordinate system;

computes, based on the sharpness of focus values, a depth of maximum focus value z_m for each pixel within the volume;

determines, based on the depths of maximum focus z_m, a three dimensional location of each point on the surface; and

constructs a three-dimensional model of the surface based on the three dimensional locations, optionally wherein the surface is a web of material.

18. The apparatus of claim 16, further comprising a light source to illuminate the surface.

19. The apparatus of claim 16, wherein the sensor comprises a CCD or a CMOS camera.

20. The apparatus of claim 19, wherein the processor is internal to the camera.

21. The apparatus of claim 19, wherein the processor is remote from the camera.

17. (canceled)

22. A method, comprising:

positioning a stationary imaging sensor at a non-zero viewing angle with respect to a moving web of material, wherein the imaging sensor comprises a telecentric lens to image a surface of the moving web and form a sequence of images thereof;

processing the sequence of images to;

register the images;

stack the registered images along a z direction in a camera coordinate system to form a volume;

determine a sharpness of focus value for each (x,y) location in the volume, wherein the (x,y) locations lie in a plane normal to the z direction in the camera coordinate system;

determine a depth of maximum focus z_m along the z direction in the camera coordinate system for each (x,y) location in the volume; and

determine, based on the depths of maximum focus z_m, a three dimensional location of each point on the surface of the moving web, optionally wherein the imagine sensor comprises a CCD or a CMOS camera.

25. The method of claim 22, further comprising forming, based on the three-dimensional locations, a three-dimensional model of the surface of the moving web.

26. The method of claim 22, wherein the sharpness of focus value is determined by applying a modified Laplacian sharpness metric at each (x,y) location.

27. The method of claim 22, wherein the depth of each point on the surface is determined by fitting along the z direction a Gaussian curve to estimate the depths of maximum focus z_m.

28. The method of claim 22, wherein the depth of each point on the surface is determined by fitting a quadratic function to the sharpness of focus values at each location (x,y), in the volume.

29. The method of claim 22, comprising applying a triangular meshing algorithm to the three dimensional point locations to form the model of the surface.

23-24. -24. (canceled)

30-50. -50. (canceled)

51. A method for inspecting a moving surface of a web material in real time and computing a three-dimensional model of the surface, the method comprising:

(a) capturing with a stationary sensor a sequence of images of the surface, wherein the imaging sensor comprises a camera and a telecentric lens having a focal plane aligned at a non-zero viewing angle with respect to an x-y plane of a surface coordinate system;

(b) determining a sharpness of focus value for every pixel in a last image in the sequence of images;

(c) computing a y-coordinate in a surface coordinate system at which the focal plane intersects the y-axis;

(d) based on the apparent shift of the surface in the last image, determining transitional points on the surface, wherein the transitional points have exited a field of view of the lens in the last image, but were in the field of view of the lens in an image in the sequence previous to the last image;

(e) determining the three dimensional location in a camera coordinate system of all the transitional points on the surface;

(f) repeating steps (a) to (f) for each new image acquired by the imaging sensor; and

(g) accumulating the three dimensional location in the camera coordinate system of the transitional points from the images in the sequence to form a point cloud representative of the translating surface, optionally wherein the sharpness of focus value is determined by applying a modified Laplacian sharpness metric.

53. The method of claim 51, wherein the three dimensional location of each transitional point on the surface is determined by fitting along the z direction in the camera coordinate system a Gaussian curve to estimate the depths of maximum focus z_m.

54. The method of claim 51, wherein the three dimensional location of each transitional point on the surface is determined by fitting a quadratic function to the sharpness of focus values for each pixel.

55. The method of claim 51, further comprising forming a first range map of the translating surface by re-sampling the points in the point cloud on a rectangular grid in the camera coordinate system.

56. The method of claim 55, further comprising removing noise from the first range map.

57. The method of claim 51, further comprising rotating the first range map to a surface coordinate system.

58. The method of claim 57, further comprising forming a second range map by re-sampling first range map on a grid in the surface coordinate system.

59. The method of claim 51, wherein, when the surface is moving toward a stationary imaging sensor, the viewing angle is about 38°

.

52. (canceled)

60. An online computerized inspection system for inspecting web material in real time, the system comprising:

a stationary imaging sensor comprising a camera and a telecentric lens, wherein the lens has a focal plane aligned at a non-zero viewing angle with respect to an x-y plane of a moving surface, and wherein the sensor images the surface to form a sequence of images thereof;

a processor that;

(a) determines a sharpness of focus value for every pixel in a last image in the sequence of images;

(b) computes a y-coordinate in a surface coordinate system at which the focal plane intersects the y axis;

(c) based on the apparent shift of the surface in the last image, determines transitional points on the surface, wherein the transitional points have exited a field of view of the lens in the last image, but were in the field of view of the lens in an image in the sequence previous to the last image;

(d) determines the three dimensional location in a camera coordinate system of all the transitional points on the surface;

(e) repeats steps (a) to (d) for each new image acquired by the imaging sensor; and

(f) accumulates the three dimensional location in the camera coordinate system of the transitional points from the images in the sequence to form a point cloud representative of the translating surface.

61. (canceled)