Multiframe correspondence estimation

• US 20040151365A1
• Filed: 02/03/2003
• Published: 08/05/2004
• Est. Priority Date: 02/03/2003
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Said, Amir, An Chang, Nelson Liang
• Granted Patent: US 7,146,036 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G01B 11/2536 using several gratings with variable grating pitch, projected on the object with the same angle of incidence
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19

Abstract

Systems and methods of multiframe correspondence estimation are described. Light patterns reflected from the scene are captured at one or more capture planes. In one aspect, a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes is projected onto a scene. A correspondence mapping between multipixel regions in the capture plane and corresponding regions in the projection plane is computed based at least in part on correspondence between light symbol sequence codes captured at the capture plane and light symbol sequence codes projected from the projection plane. In another aspect, the patterns consist of a single plane of light that is swept across the scene. Correspondence is determined based at least in part on exploiting the epipolar constraint associated with any pair of cameras.

Claims

1. A method of multiframe correspondence estimation, comprising:

projecting onto a scene a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes;

capturing at a capture plane light patterns reflected from the scene; and

computing a correspondence mapping between multipixel regions in the capture plane and corresponding regions in the projection plane based at least in part on correspondence between light symbol sequence codes captured at the capture plane and light symbol sequence codes projected from the projection plane.

2. The method of claim 1, wherein among the projected light symbol patterns are light patterns respectively comprising different spatial variations of light in the projection plane.

3. The method of claim 2, wherein each light symbol pattern comprises a binary pattern of light and dark rectangular stripe symbols.

4. The method of claim 3, wherein each light symbol pattern includes light and dark stripe symbols of substantially equal size in the projection plane, and light and dark stripe symbols of different light symbol patterns are of substantially different size in the projection plane.

5. The method of claim 3, wherein a first subset of light symbol patterns encodes rows of a reference grid in the projection plane, and a second subset of light symbol patterns encodes columns of the reference grid in the projection plane.

6. The method of claim 5, wherein each light symbol pattern encodes position information with a respective spatial resolution, and light symbol patterns of each of the first and second subsets exhibit a hierarchy of spatial resolutions from coarse to fine.

7. The method of claim 6, wherein symbol light patterns are projected onto scene in order from coarse resolution to fine resolution.

8. The method of claim 7, wherein each light symbol sequence code corresponds to a binary number that directly encodes a row number and a column number for each region in the reference grid.

9. The method of claim 7, further comprising identifying an error producing light pattern resolution associated with a decoding error count greater than a selected threshold.

10. The method of claim 9, wherein projecting light patterns comprises projecting light patterns in order from the coarse resolution to a resolution just coarser than the identified error producing light pattern resolution.

11. The method of claim 10, wherein projecting light patterns comprises projecting multiple unique light patterns each having a common resolution just coarser than the identified error producing light pattern resolution.

12. The method of claim 2, wherein each light pattern comprises a multicolor pattern of light.

13. The method of claim 1, wherein computing a correspondence mapping comprises mapping centroids of light symbol regions in the capture plane to centroids of corresponding light symbol regions in the projection plane.

14. The method of claim 1, wherein computing a correspondence mapping comprises mapping corner points of light symbol regions in the capture plane to corresponding corner points of light symbol regions in the projection plane.

15. The method of claim 14, wherein mapping corner points comprises interpolating corner points in the capture plane at a given light symbol pattern resolution level based at least in part on corner points identified in the capture plane at a coarser light symbol pattern resolution level.

16. The method of claim 1, wherein computing a correspondence mapping comprises determining light symbol code sequences captured at different regions of the capture plane.

17. The method of claim 16, wherein each light symbol pattern comprises light selected from a prescribed set of light colors each representing a respective code symbol.

18. The method of claim 17, wherein determining a code symbol sequence comprises assigning to a given pixel a symbol color most likely to correspond to a light color captured at the given pixel based on a measure of distance of a light color captured at the given pixel from a mean color metric computed for the most likely symbol color.

19. The method of claim 17, wherein determining a code symbol sequence comprises labeling a given pixel as indeterminate based on a measure of distance of a light color captured at the given pixel from a mean color metric computed for a symbol color most likely to correspond to the light color captured at the given pixel.

20. The method of claim 19, further comprising correcting an indeterminate pixel based on symbols assigned to neighboring determinate pixels.

21. The method of claim 1, further comprising filtering out a given captured light symbol based on an assessment of consistency of the given captured light symbol.

22. The method of claim 21, wherein captured light symbols divided into two or more regions are filtered out.

23. The method of claim 1, further comprising rotating an object in the scene about an axis.

24. A system of multiframe correspondence estimation, comprising:

a light source operable to project onto a scene a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes;

at least one camera operable to capture at a capture plane light patterns reflected from the scene; and

a processing system operable to compute a correspondence mapping between multipixel regions in the capture plane and corresponding regions in the projection plane based at least in part on correspondence between light symbol sequence codes captured at the capture plane and light symbol sequence codes projected from the projection plane.

25. A method of multiframe correspondence estimation, comprising:

projecting onto a scene a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes, wherein at least two light patterns encode position information at different spatial resolutions;

capturing at a capture plane light patterns reflected from the scene; and

computing a correspondence mapping between the capture plane and the projection plane based at least in part on correspondence between light symbol sequence codes captured at the capture plane and light symbol sequence codes projected from the projection plane.

26. The method of claim 25, wherein light symbol patterns are projected onto scene in order from coarse resolution to fine resolution.

27. The method of claim 26, wherein each light symbol sequence code corresponds to a binary number that directly encodes a row number and a column number for each region in the reference grid.

28. The method of claim 26, further comprising identifying an error producing light pattern resolution associated with a decoding error count greater than a selected threshold.

29. The method of claim 28, wherein projecting light patterns comprises projecting light patterns in order from the coarse resolution to a resolution just coarser than the identified error producing light pattern resolution.

30. The method of claim 29, wherein projecting light patterns comprises projecting multiple unique light patterns each having a common resolution just coarser than the identified error producing light pattern resolution.

31. The method of claim 25, wherein computing a correspondence mapping comprises mapping points in the capture plane to corresponding points in the projection plane at multiple spatial resolution levels to obtain a set of multiresolution correspondences.

32. The method of claim 31, further comprising validating the computed correspondence mapping based at least in part on consistency of point mappings in the set of multiresolution correspondences.

33. The method of claim 25, further comprising rotating an object in the scene about an axis.

34. A system of multiframe correspondence estimation, comprising:

a light source operable to project onto a scene a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes, wherein at least two light patterns encode position information at different spatial resolutions;

at least one camera operable to capture at a capture plane light patterns reflected from the scene; and

a processing system operable to compute a correspondence mapping between the capture plane and the projection plane based at least in part on correspondence between light symbol sequence codes captured at the capture plane and light symbol sequence codes projected from the projection plane.

35. A method of multiframe correspondence estimation, comprising:

projecting a plane of light onto a scene from a projection plane;

sweeping the plane of light across the scene;

capturing light contours reflected from the scene at a capture plane related by epipolar geometry with respect to the projection plane; and

computing a correspondence mapping between the capture plane and the projection plane based at least in part on intersection of captured light contours and epipolar lines computed for the capture plane and the projection plane.

36. A method of multiframe correspondence estimation, comprising:

projecting a plane of light onto a scene;

sweeping the plane of light across the scene;

capturing light contours reflected from the scene at multiple capture planes each pair of which is related by epipolar geometry with respect to the scene; and

computing a correspondence mapping between each pair of capture planes based at least in part on intersection of captured light contours and epipolar lines computed for each of the capture planes.

37. The method of claim 36, further comprising computing estimates of relative position coordinates of capture planes.

38. The method of claim 37, further comprising computing three-dimensional coordinates for contour points of the scene based at least in part on the computed correspondence mapping and the computed estimates of relative positions of capture planes.

39. The method of claim 36, wherein sweeping the plane of light comprises rotating an object about an axis.

40. The method of claim 39, wherein the object is rotated in discrete steps, and a correspondence mapping is computed at each rotational step.

41. A system of multiframe correspondence estimation, comprising:

a light source operable to project a plane of light onto a scene and to sweep the plane of light across the scene;

at least two cameras operable to capture light contours reflected from the scene at multiple capture planes each pair of which having an epipolar geometric orientation with respect to the scene; and

a processing system operable to compute a correspondence mapping between each pair of capture planes based at least in part on intersection of captured light contours and epipolar lines computed for each of the capture planes.

42. A method of multiframe correspondence estimation, comprising:

projecting a first light pattern onto a scene;

capturing at a capture plane light patterns reflected from the scene;

computing a coarse correspondence mapping between the capture plane and a reference plane based at least in part on the captured light patterns;

identifying one or more undersampled regions of the scene based at least in part on the computed coarse correspondence mapping;

projecting onto the undersampled regions of the scene a second light pattern different from the first light pattern; and

computing a refined correspondence mapping between the capture plane and the reference plane based at least in part on the light patterns captured during projection of the second light pattern.

43. The method of claim 42, wherein projecting a first light pattern comprises projecting onto a scene a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes.

44. The method of claim 42, wherein projecting a first light pattern comprises projecting onto a scene a sequence of patterns of light symbols that temporally encode two-dimensional position information in a projection plane with unique light symbol sequence codes, wherein at least two light patterns encode position information at different spatial resolutions.

45. The method of claim 42, wherein projecting a first light pattern comprises projecting a plane of light onto a scene.

46. A system of multiframe correspondence estimation, comprising:

a light source operable to project onto a scene a first light pattern and a second light pattern different from the first light pattern;

at least one camera operable to capture at a capture plane light patterns reflected from the scene; and

a processing system operable to compute a coarse correspondence mapping between the capture plane and a reference plane based at least in part on captured light patterns corresponding to the first light pattern, identify one or more undersampled regions of the scene based at least in part on the computed coarse correspondence mapping, and compute a refined correspondence mapping between the capture plane and the reference plane based at least in part on the light patterns captured during projection of the second light pattern.