Multiple channel locating

• US 8,531,650 B2
• Filed: 07/08/2009
• Issued: 09/10/2013
• Est. Priority Date: 07/08/2008
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Chiaro Technologies LLC (Cognex Corporation)
• Inventors: Feldkhun, Daniel, Braker, Benjamin, Moore, Eric Daniel
• Primary Examiner(s): RATCLIFFE, LUKE D
• Application Number:
  US12/499,758
• Publication Number:
  US 20100008588A1
• Time in Patent Office: 1,525 Days
• Field of Search: 356 301- 315, 356 401- 41, 356 501- 515, 356 6- 22, 356/28, 356/28.5, 35613901-1391
• US Class Current: 356/4.01
• CPC Class Codes: G01B 11/005 coordinate measuring machines
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G01B 11/2527 with phase change by in-plane movement of the patern
• CPC Class Codes: G01B 11/2536 using several gratings with variable grating pitch, projected on the object with the same angle of incidence
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/48 Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4815 using multiple transmitters
• CPC Class Codes: G01S 7/4818 using optical fibres
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/486 Receivers
• CPC Class Codes: G01S 7/499 using polarisation effects

Abstract

Methods, systems, and apparatuses are provided for estimating a location on an object in a three-dimensional scene. Multiple radiation patterns are produced by spatially modulating each of multiple first radiations with a distinct combination of one or more modulating structures, each first radiation having at least one of a distinct radiation path, a distinct source, a distinct source spectrum, or a distinct source polarization with respect to the other first radiations. The location on the object is illuminated with a portion of each of two or more of the radiation patterns, the location producing multiple object radiations, each object radiation produced in response to one of the multiple radiation patterns. Multiple measured values are produced by detecting the object radiations from the location on the object due to each pattern separately using one or more detector elements. The location on the object is estimated based on the multiple measured values.

Claims

1. A method for estimating a location on an object in a three-dimensional scene, the method comprising the steps of:

a) producing a plurality of radiation patterns by spatially modulating each of a plurality of first radiations with a distinct combination of one or more of a plurality of modulating structures, each of the first radiations having at least one of a distinct radiation path, a distinct source, a distinct source spectrum, or a distinct source polarization with respect to the other first radiations;

b) illuminating the location on the object with a portion of each of two or more of the plurality of radiation patterns, the location producing a plurality of object radiations, each object radiation produced in response to one of the plurality of radiation patterns;

c) producing a plurality of measured values by detecting the object radiations from the location on the object due to each pattern separately using one or more detector elements; and

d) estimating the location on the object based on the plurality of measured values.

2. The method of claim 1 wherein at least one of the first radiations comprises a distinct portion of a radiation field with respect to at least one of the other first radiations.

3. The method of claim 1 wherein at least one of the first radiations has at least one of a distinct spectrum or a distinct polarization with respect to at least one of the other first radiations.

4. The method of claim 1 wherein at least one of the first radiations has a spectral distribution.

5. The method of claim 1 wherein at least one of the first radiations is partially spatially coherent.

6. The method of claim 1 wherein at least two of the first radiations are generated sequentially by varying at least one of a spectrum or a polarization of a single radiation source.

7. The method of claim 1 wherein at least two of the first radiations are generated by altering a path of one or more portions of radiation from a single radiation source.

8. The method of claim 1 wherein the plurality of first radiations is produced by a plurality of distinct radiation sources.

9. The method of claim 8 wherein at least one of the plurality of radiation sources has at least one of distinct spectrum or a distinct polarization with respect to at least one of the other radiation sources.

10. The method of claim 1 further comprising directing at least one of the first radiations onto at least one of the modulating structures using at least one of a dispersive element or a polarization-selective element.

11. The method of claim 8 wherein the plurality of distinct radiation sources includes a plurality of elements of at least one of an array of laser diodes or an array of light emitting diodes.

12. The method of claim 8 wherein the plurality of distinct radiation sources includes a plurality of elements of an array of waveguide outputs.

13. The method of claim 12 wherein the waveguide outputs are elements of a flexible optical waveguide array radiation channel, and the modulating structures comprise and array of at least one of a diffractive elements or a pattern mask modulating the first radiations from the waveguide outputs at the end of the flexible waveguide array radiation channel to produce the plurality of radiation patterns.

14. The method of claim 13 wherein the flexible waveguide array radiation channel is part of an endoscope.

15. The method of claim 1 wherein each of the modulating structures spatially modulates at least one of an amplitude, a phase, or a polarization of one or more of the first radiations.

16. The method of claim 1 wherein at least one of the modulating structures includes a diffractive element.

17. The method of claim 1 wherein at least one of the modulating structures is reflective.

18. The method of claim 1 wherein the plurality of modulating structures includes a plurality of elements of an array of structures.

19. The method of claim 1 wherein the plurality of modulating structures includes one or more multiple layer structures.

20. The method of claim 1 wherein the plurality of modulating structures includes a hologram.

21. The method of claim 20 wherein at least one of the first radiations illuminates the hologram from a distinct direction with respect to at least one other first radiation illuminating the hologram.

22. The method of claim 20 wherein at least one of the first radiations illuminating the hologram has at least one of a distinct spectrum or a distinct polarization with respect to at least one other first radiation illuminating the hologram.

23. The method of claim 1 wherein at least one of the modulating structures varies in time.

24. The method of claim 1 wherein at least one of the modulating structures comprises a propagating acoustic perturbation in an acousto-optic device.

25. The method of claim 24 wherein at least one of the first radiations has a spectral distribution.

26. The method of claim 24 wherein at least one of the first radiations is partially spatially coherent.

27. The method of claim 1 wherein at least one of the modulating structures comprises a reconfigurable pattern in at least one of a liquid crystal array or a digital micromirror device array.

28. The method of claim 1 wherein at least two of the radiation patterns illuminate the object in sequence.

29. The method of claim 28 wherein the sequential illumination is effected by controlling at least one of a radiation source, one or more of the modulating structures, or a shuttering device.

30. The method of claim 1 wherein at least one of an amplitude, a position, a scaling, or an orientation of one or more of the radiation patterns varies in time.

31. The method of claim 30 wherein the at least one of the amplitude, the position, the scaling, or the orientation variation in time of the one or more of the radiation patterns is substantially correlated with at least one of a motion, a scaling, a change in orientation, or a change in the illumination response of the object.

32. The method of claim 1 wherein each of at least one of the radiation patterns has at least one of a distinct spectrum or a distinct polarization with respect to at least one of the other radiation patterns.

33. The method of claim 32 wherein the at least one of the distinct spectrum or the distinct polarization of a radiation pattern is due to at least one of a radiation source, a filter, a dispersive element, or one or more of the modulating structures.

34. The method of claim 32 wherein the at least one of the distinct spectrum or the distinct polarization varies in time.

35. The method of claim 1 wherein the object radiations from the location on the object in response to the illumination are detected by a plurality of detector elements, each detector element belonging to one of a plurality of detector arrays, each detector array recording an image of the object.

36. The method of claim 35 wherein at least two of the detector arrays record images of the object in sequence.

37. The method of claim 36 wherein the sequential recording is effected by electronically controlling the start time and duration of radiation integration for each said detector array.

38. The method of claim 36 wherein the sequential recording is effected by modulating the amplitudes of the object radiations by each said detector array using a shuttering device.

39. The method of claim 38 wherein the shuttering device comprises a liquid crystal device.

40. The method of claim 35 wherein each of at least two of the detector arrays records an image of the object, the image having at least one of a distinct spectrum or distinct polarization with respect to the other recorded images.

41. The method of claim 40 wherein the at least one of distinct spectrum or distinct polarization of the image is due to a filter filtering object radiation by the detector array.

42. The method of claim 41 wherein the filter comprises a liquid crystal device.

43. The method of claim 40 wherein the at least one of the distinct spectrum or the distinct polarization of the image is due to a dispersive element dispersing object radiation.

44. The method of claim 40 wherein the at least one of the distinct spectrum or the distinct polarization of the image is varied in time.

45. The method of claim 1 wherein the radiation patterns are directed at the object substantially along an illumination axis, vary substantially periodically along a direction that is substantially orthogonal to the illumination axis, and have distinct spatial frequencies along said direction, and estimating the location on the object based on the plurality of measured values includes the steps of:

e) estimating a periodic function from the plurality of measured values;

f) estimating with coarse angular resolution, using a frequency of the periodic function, an illumination angle between the illumination axis and a direction from a known location on the illumination axis to the location on the object geometrically projected onto an illumination plane, the illumination plane including the direction of pattern variation and the illumination axis;

g) estimating the illumination angle with fine angular resolution using a phase of the periodic function, the fine resolution estimate having an ambiguous angular offset; and

h) resolving the ambiguous angular offset using the coarse angular resolution estimate.

46. The method of claim 45 wherein the one or more detector elements detect the object radiations directed substantially along an imaging axis and estimating the location on the object based on the plurality of measured values further includes the steps of:

i) estimating a locating direction from a known location on the imaging axis to the location on the object using the known location of the one or more detector elements; and

j) estimating the location on the object from the estimated illumination angle, the estimated locating direction, and the known locations on the illumination and imaging axes by triangulation.

47. The method of claim 45 wherein at least one of the radiation patterns varies along a plurality of directions.

48. The method of claim 45 wherein at least one of the periodic radiation patterns is a Fourier component of a non-periodic radiation pattern.

49. The method of claim 1 further comprising the step of estimating a three-dimensional surface of the object from a plurality of locations estimated by repeating steps a-d.

50. A method for estimating a location on an object in a three-dimensional scene, the method comprising the steps of:

a) producing a plurality of radiation patterns, at least one of the patterns varying substantially continuously, substantially non-periodically, and substantially non-monotonically along one or more directions;

b) illuminating the location on the object with a portion of each of two or more of the plurality of radiation patterns, the illumination being substantially distinct with respect to other locations on the object lying along said one or more directions from said location, and the location producing a plurality of object radiations, each object radiation produced in response to one of the plurality of radiation patterns;

c) producing a plurality of measured values by detecting the object radiations from the location on the object due to each radiation pattern separately using one or more detector elements; and

d) estimating the location on the object based on the plurality of measured values.

51. The method of claim 50 wherein two or more of the radiation patterns are produced by shifting a first pattern along one or more directions of pattern variation.

52. The method of claim 51 wherein the first pattern has a spatially varying period along the shifting direction that is substantially orthogonal to an illumination axis, the two or more shifted patterns are directed at the object substantially along the illumination axis, and estimating the location on the object includes the steps of:

e) estimating a periodic function from the plurality of measured values;

f) estimating with coarse angular resolution, using a period of the periodic function, an illumination angle between the illumination axis and a direction from a known location on the illumination axis to the location on the object geometrically projected onto an illumination plane, the illumination plane including the direction of the period variation of the first pattern and the illumination axis;

g) estimating with fine angular resolution the illumination angle using a phase of the periodic function, the fine resolution estimate having an ambiguous angular offset; and

h) resolving the ambiguous angular offset using the coarse angular resolution estimate.

53. The method of claim 52 wherein the one or more detector elements detect the object radiations directed substantially along an imaging axis, and estimating the location on the object further includes the steps of:

i) estimating a locating direction from a known location on the imaging axis to the location on the object using a known location of the one or more detector elements; and

j) estimating the location on the object from the estimated illumination angle, the estimated locating direction, and the known locations on the illumination and imaging axes by triangulation.

54. The method of claim 52 wherein at least one of the radiation patterns varies along a plurality of directions.

55. The method of claim 50 further comprising the steps of:

e) computing a plurality of illumination symbols in a locating symbol space from the plurality of patterns, each illumination symbol corresponding to a distinct plurality of illumination values at a location in the scene with respect to other locations in the scene and being independent of an absolute scaling and an offset of the plurality of illumination values at said location in the scene;

f) computing a detection symbol in the locating symbol space from the plurality of measured values, the detection symbol being independent of the absolute scaling and the offset of the measured values; and

g) establishing a correspondence between the detection symbol and one of the illumination symbols in the locating symbol space.

56. The method of claim 55 wherein two or more of the radiation patterns are directed at the object substantially along an illumination axis and vary in a direction that is substantially orthogonal to the illumination axis, and estimating the location on the object further includes the step of estimating from the correspondence between the detection symbol and the one of illumination symbols an illumination angle between the illumination axis and a direction from a known location on the illumination axis to the location on the object geometrically projected onto an illumination plane, the illumination plane including a direction of pattern variation and the illumination axis.

57. The method of claim 56 wherein the one or more detector elements detect the object radiations directed substantially along an imaging axis, and estimating the location on the object further includes the steps of:

h) estimating a locating direction from a known location on the imaging axis to the location on the object using the known location of the one or more detector elements; and

i) estimating the location on the object from the estimated illumination angle, the estimated locating direction, and the known locations on the illumination and imaging axes by triangulation.

58. The method of claim 55 further comprising the step of measuring the plurality of illuminations at each location of a plurality of locations in the scene to produce a plurality of illumination values.

59. The method of claim 55 wherein the symbol computation includes computing a symbol vector from a source vector, the source vector comprising the measured values or the illumination values, by computing a ratio of a linear transform of the source vector and a linear combination of the source vector elements.

60. The method of claim 55 wherein the symbol computation includes computing a symbol vector from a source vector, the source vector comprising the measured values or the illumination values, the symbol vector comprising a ratio of a difference between the source vector and one or more of the source vector elements and a linear combination of the source vector elements.

61. The method of claim 55 wherein the locating symbol space maps to a linear vector space and the step of establishing a correspondence between the detection symbol and one of the illumination symbols includes:

h) computing for each illumination symbol the vector norm of the vector difference between the detection symbol and the illumination symbol; and

i) establishing the correspondence between the detection symbol and the illumination symbol used to compute the minimum vector norm of the computed vector norms.

62. The method of claim 50 further comprising illuminating a propagating acoustic perturbation in an acousto-optic device to produce at least one of the plurality of patterns.

63. The method of claim 62 wherein at least one of the acousto-optically generated patterns is made substantially stationary during said detecting by varying the amplitude of said pattern in time.

64. The method of claim 62 wherein the illumination illuminating the acoustic perturbation has a spectral distribution.

65. The method of claim 62 wherein the illumination illuminating the acoustic perturbation is partially spatially coherent.

66. The method of claim 62 wherein the plurality of patterns includes a plurality of shifted patterns formed by at least one of illuminating the acousto-optic device with a sequence of radiation pulses or modulating the amplitude of radiation diffracted from the acousto-optic device with a sequence of pulses.

67. The method of claim 50 wherein at least two of the plurality of patterns illuminate the location on the object from substantially different directions, the illumination being substantially distinct with respect to other locations on the object lying along the one or more directions of pattern variation from said location.

68. The method of claim 50 wherein estimating the location on the object includes using a lookup table.

69. A system for estimating a distance to a location on an object in a three-dimensional scene, the system comprising:

one or more radiation sources for generating a plurality of first radiations, each of the first radiations having at least one of a distinct radiation source, a distinct radiation path, distinct source spectrum, or a distinct source polarization with respect to the other first radiations;

a plurality of modulating structures for spatially modulating the plurality of first radiations, wherein each of the first radiations is modulated by a distinct combination of one or more of the modulating structures, producing a plurality of radiation patterns and the location on the object being illuminated with a portion of each of two or more of the plurality of radiation patterns, the location producing a plurality of object radiations, each object radiation produced in response to one of the plurality of radiation patterns;

one or more detector elements for detecting object radiations from the location on the object due to each pattern separately and producing a plurality of measured values; and

a processor in communication with the one or more detector elements, the one or more processors estimating the location on the object based on the plurality of measured values.

70. The system of claim 69 further comprising an integrated three-dimensional sensing system embedded in a mobile electronic device, wherein the integrated three-dimensional sensing system includes the one or more radiation sources, the plurality of modulating structures, and the one or more detector elements.

71. The system of claim 70 wherein the mobile electronic device comprises at least one of a mobile telephone, a laptop computer, a personal digital assistant, a portable gaming system, a photographic camera, or a video camera.

72. The system of claim 69 wherein the one or more processors further estimate a three-dimensional surface of the object from a plurality of locations.

73. The system of claim 72 further comprising one or more displays for displaying the three-dimensional surface.

74. The system of claim 70 further comprising two or more multiple-channel pattern projectors embedded at separate locations on the mobile electronic device.

75. The system of claim 70 further comprising two or more multiple-channel imagers embedded at separate locations on the mobile electronic device, the two or more embedded multiple-channel imagers including said one or more detectors.

76. The system in claim 69 wherein the processor estimates the location on the object using a lookup table.

77. A system for estimating a distance to a location on an object in a three-dimensional scene, the system comprising:

means for generating a plurality of first radiations, each of the first radiations having at least one of a distinct source, a distinct radiation path, distinct source spectrum, or a distinct source polarization;

means for producing a plurality of radiation patterns, wherein the means including a plurality of modulating structures and each of the first radiations is modulated by a distinct combination of one or more of the modulating structures;

means for illuminating the location on the object with two or more of the plurality of radiation patterns, the location producing a plurality of object radiations, each object radiation produced in response to one of the plurality of radiation patterns;

means for detecting radiations from the location on the object due to each pattern separately and producing a plurality of measured values; and

means for estimating the location on the object based on the plurality of measured values.