Optical ranging camera

• US 6,057,909 A
• Filed: 12/19/1997
• Issued: 05/02/2000
• Est. Priority Date: 06/22/1995
• Status: Expired due to Term

• Current Assignee: Microsoft International Holdings BV (Microsoft Corporation)
• Original Assignee: 3DV Systems Ltd. (Microsoft Corporation)
• Inventors: Iddan, Gavriel, Yahav, Giora
• Primary Examiner(s): Buczinski, Stephen C.
• Application Number:
  US08/981,357
• Time in Patent Office: 865 Days
• Field of Search: 313/105 CM, 356/376, 356/378, 356/379, 356/5.04, 356/5.03, 348/367, 348/584, 364/413.15, 702/156
• US Class Current: 356/5.04
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 17/02 Systems using the reflection of electromagnetic waves other than radio waves G01S17/66 takes precedence
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 17/933 of aircraft or spacecraft
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/4812 transmitted and received beams following a coaxial path
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/51 Display arrangements
• CPC Class Codes: G02B 7/32 using active means, e.g. light emitter including both an active and a passive focus detecting device G02B7/285; using ultrasound G02B7/40

Abstract

Apparatus for creating an image indicating distances to objects in a scene, comprising: a modulated source of radiation, having a first modulation function, which directs radiation toward a scene; a detector, which detects radiation reflected from the scene, modulated by a second modulation function, and generates, responsive to said detected modulated radiation, signals responsive to the distance to regions of the scene; a processor, which receives signals from the detector and forms an image, based on the signals, having an intensity value distribution indicative of the distance of objects from the apparatus; and a controller, which varies at least one of the first and second modulation functions, responsive to the intensity value distribution of the image formed by the processor.

Claims

1. Apparatus for creating an image indicating distances to objects in a scene, comprising:

a modulated source of radiation, having a first modulation function, which directs modulated radiation toward a scene such that a portion of the radiation from the source is reflected from points in the scene and reaches the apparatus;

a detector, which detects radiation reflected from the scene, modulated by a second modulation function, and generates, responsive to said detected modulated radiation, signals responsive to a part of the portion of the reflected radiation, the part being dependent on the distance of a point in the scene, associated with a particular signal, from the apparatus;

a processor, which receives signals from the detector and forms an image, based on the signals, having an intensity value distribution indicative of the distance of objects from the apparatus; and

a controller, which varies at least one of the first and second modulation functions, responsive to the intensity value distribution of the image formed by the processor.

2. Apparatus according to claim 1, wherein the second modulation function is delayed in time relative to the first modulation function, and the controller varies the time delay.

3. Apparatus according to claim 1, wherein the first and second modulation functions comprise pulses having respective characteristic durations, and the controller varies the duration of at least one of the pulses.

4. Apparatus according to claim 3, wherein the first and second modulation functions comprise pulses having substantially equal characteristic durations, and the controller varies the duration of the pulses in both the first and second modulation functions.

5. Apparatus according to claim 1, and comprising an image analyzer, which identifies an object of interest in the scene, wherein the controller varies the at least one modulation function in response to the distance of the object of interest from the apparatus.

6. Apparatus according to claim 1, and comprising a rangefinder, which determines the distance of an object of interest from the apparatus, wherein the controller varies the at least one modulation function in response to the distance determined by the rangefinder.

7. Apparatus according to claim 1, wherein the controller varies the at least one modulation function, so that the detector detects only radiation reflected from regions of the scene at a distance of interest from the apparatus.

8. Apparatus according to claim 7, and comprising an image processor, which receives a first image, formed by the processor, of the regions of the scene at the distance of interest, and mixes the first image with a second image to produce a mixed image.

9. Apparatus according to claim 1, wherein the detector detects radiation reflected from two objects of interest in the scene, and the controller varies the at least one modulation function so that portions of the image corresponding to the two objects have generally equal intensity values.

10. Apparatus according to claim 1, wherein the detector comprises:

a detector modulator, which receives radiation reflected from the scene and modulates it in accordance with the second modulation function; and

a detector array, comprising a plurality of detector elements, which detect radiation modulated by the detector modulator and generate, responsive to said detected modulated radiation, signals responsive to the distance to regions of the scene.

11. Apparatus according to claim 10, wherein the detector modulator comprises a shutter element array, comprising a plurality of shutter elements.

12. Apparatus according to claim 11, wherein the radiation incident on a first detector element is modulated by a first shutter element, and the radiation incident on a second detector element is modulated by a second shutter element, and wherein the second modulation function includes different first and second modulation sub-functions, which are applied to the first and second shutter elements, respectively.

13. Apparatus according to any of the preceding claims, wherein the detector includes a color filter and the image formed by the processor is a color image.

16. Apparatus according to claim 11 or 15, wherein the shutter element array comprises a liquid crystal array.

17. Apparatus according to claim 11 or 15, wherein the shutter element array comprises a microchannel array.

18. Apparatus according to claim 17, wherein the microchannel array comprises a microchannel plate and a plurality of switchable electrodes adjacent to the plate.

19. Apparatus according to claim 18, wherein the microchannel array comprises a photocathode, and wherein the electrodes comprise conductive, transparent material coated on the photocathode.

20. Apparatus according to claim 19, wherein the conductive, transparent material comprises indium tin oxide.

21. Apparatus according to claim 11 or 15, wherein the shutter element array comprises a linear shutter array.

45. A method of robot guidance, comprising:

creating an image indicating distances to objects in a scene, utilizing the apparatus of any of claims 1, 14, 42 or 43; and

acting upon an object in the scene by a robot, using the distance to the object indicated by the apparatus for guidance.

46. A method for determining dimensions of an article, comprising:

creating an image of the article, indicating distances to regions of the article, utilizing the apparatus of any of claims 1, 14, 42 or 43; and

determining, by a computer, of the dimensions of the article, using the image and the indicated distances.

47. A method according to claim 46, wherein the the dimensions are compared with a standard by the computer.

48. A method according to claim 46, and including determining, by the computer the volume of the article, using the determined dimensions.

49. A system for producing a three-dimensional model of an article, comprising:

a system for determining dimensions of the article, according to claim 46; and

rapid prototyping apparatus, which receives the dimensions determined by the system for determining dimensions, and produces the three-dimensional model according to the dimensions.

50. A system according to claim 49, wherein the rapid prototyping apparatus receives image information from the apparatus for creating the image of the article and applies the image information to a surface of the three-dimensional model.

51. A system according to claim 50, wherein the image information comprises color information.

52. A system for vehicle guidance, comprising:

apparatus for creating an image indicating distances to objects in a scene ahead of the vehicle in a direction of motion thereof, according to any of claims 1, 14, 42 or 43; and

a processor, which identifies hazards in the scene, using the image and the indicated distances.

53. A method of endoscopic imaging, comprising:

inserting a distal end of an endoscope into the body of a subject;

coupling an apparatus according to any of claims 1, 14, 42 or 43 to a proximal end of the endoscope; and

creating an image by the apparatus, by directing radiation through the endoscope into the body, and detect radiation reflected therefrom by the apparatus.

14. Apparatus for creating an image indicating distances to objects in a scene, comprising:

a modulated source of radiation, having a first modulation function, which directs radiation toward a scene;

a detector, comprising a detector modulator, having a second modulation function, and a detector array, said array comprising a plurality of detector elements, wherein the detector detects radiation reflected from a plurality of parallel spatial segments within the scene and generates, responsive to said detected radiation, signals responsive to the distance to regions of the scene; and

a processor, which forms an image including at least some of the plurality of spatial segments and having an intensity value distribution indicative of the distance of objects from the apparatus,

wherein each of the plurality of spatial segments has distance bounds, relative to the apparatus, which distance bounds are determined by the detector modulator, and

wherein at least one of the plurality of spatial segments has distance bounds that are different from the bounds of at least one other spatial segment.

15. Apparatus according to claim 14, wherein the detector modulator comprises a shutter array, comprising a plurality of shutter elements.

22. Apparatus according to claim 15, wherein each shutter element is associated with one of the plurality of spatial segments, and wherein the distance bounds of said one of the plurality of spatial segments are determined by the associated shutter element.

23. Apparatus according to claim 15, wherein each of the plurality of shutter elements is associated with a respective group of one or more detector elements.

24. Apparatus according to claim 15, wherein the detector modulator comprises a switching network, which controls the shutter elements according to the second modulation function.

25. Apparatus according to claim 24, wherein application of the second modulation function causes the shutter elements to open and shut sequentially.

26. Apparatus according to claim 24 or 25, wherein the second modulation function comprises a plurality of non-coincident pulses, each of which pulses causes a corresponding shutter element to open and shut.

27. Apparatus according to claim 23, wherein each of the respective groups of one or more detector elements comprises a row of elements of the detector array.

28. Apparatus according to claim 27, wherein each of the respective groups of one or more detector elements comprises a group of mutually-adjacent rows of elements of the detector array.

29. Apparatus according to claim 14, wherein the source of radiation produces a line of illumination on the scene, and wherein the detector receives a corresponding line of reflected radiation.

30. Apparatus according to claim 29, and comprising a scanning device, which scans the line of illumination across the scene.

31. Apparatus according to claim 29 or 30, wherein the detector modulator deflects the line of reflected radiation so as to sequentially dwell for a period of time on each of a plurality of rows of elements of the detector array.

32. Apparatus according to claim 31, wherein each of the plurality of rows of the array is associated with one of the plurality of spatial segments, and wherein the distance bounds of said one of the plurality of spatial segments are determined by the period of time that the line of reflected radiation dwells on the associated row.

33. Apparatus according to claim 31, wherein the period of time that the line of reflected radiation dwells on each of the plurality of rows is determined by the second modulation function.

34. Apparatus according to claim 14, and comprising a controller, which varies at least one of the first and second modulation functions, responsive to the intensity value distribution of the image formed by the processor.

35. Apparatus according to claim 14, wherein the detector includes a color filter.

36. A system for creating an extended image indicating distances to objects in a scene, comprising:

apparatus for creating an image according to claim 14, which forms a plurality of sequential images of the scene, at least one of said plurality of images including one or more regions of the scene not included in at least one other of said images; and

an image processor, which combines the plurality of images to form an extended image, indicating distances to objects in the scene.

37. A system according to claim 36, and comprising means for translating the apparatus laterally relative to the scene.

38. A system according to claim 37, wherein the apparatus creates images of the scene in a known relation to a rate of translation of the apparatus relative to the scene.

39. A system according to claim 37 or 38, wherein the means for translating the apparatus comprise a moving platform, to which the apparatus is mounted.

40. A system according to claim 39, wherein the moving platform comprises an aircraft.

41. A system according to claim 36, and comprising scanning optics, which scan the field of view of the apparatus over the scene.

42. Apparatus for creating an image indicating distances to objects in a scene, comprising:

a modulated source of radiation, which directs radiation toward a scene;

telecentric optics, which receive and collimate radiation reflected from the scene;

a modulator, which modulates the collimated radiation;

a detector, which detects an image formed from the modulated collimated radiation; and

a processor, which forms an image having an intensity value distribution indicative of the distance of objects from the apparatus, responsive to the intensity distribution of the detected image,

wherein the source and the detector are boresighted, and

wherein the modulator that modulates the collimated radiation reflected from the scene does not modulate the radiation directed toward the scene by the source.

43. Apparatus for creating an image indicating distances to objects in a scene, comprising:

a modulated source of radiation, having a first modulation function, which directs radiation toward a scene;

a first detector, which detects radiation reflected from the scene, modulated by a second modulation function, and generates, responsive to said detected modulated radiation, signals responsive to the distance to regions of the scene;

a second detector, boresighted with the first detector, which detects radiation reflected from the scene, unmodulated by said second modulation function, and generates, responsive to said detected radiation, signals responsive to the intensity of light reflected from the regions of the scene;

a processor, which receives signals from the first and second detectors and forms an image based on the signals, having an intensity value distribution indicative of the distance of objects from the apparatus.

44. Apparatus according to claim 43, wherein the radiation source is boresighted with the first and second detectors.

54. A method for creating an image indicating distances to objects in a scene, comprising:

modulating radiation from a radiation source according to a first modulation function, and directing said radiation toward a scene;

modulating radiation reflected from the scene according to a second modulation function;

detecting said reflected radiation and generating signals responsive to the distance to regions of the scene, in response to the detected radiation;

forming an image having an intensity value distribution indicative of the distance to objects in the scene; and

varying at least one of the first and second modulation functions, responsive to the intensity value distribution of the image.

55. A method according to claim 54, wherein varying at least one of the modulation functions comprises delaying the second modulation function relative to the first.

56. A method according to claim 54, wherein the first and second modulation functions comprise pulses, and varying at least one of the modulation functions comprises varying the duration of at least one of the pulses.

57. A method according to claim 56, wherein the duration of the first modulation function pulse is substantially greater than the duration of the second modulation function pulse.

58. A method according to claim 54, and comprising:

identifying at least one object of interest in the scene; and

determining the distance to the object, using the intensity value distribution of the image.

59. A method according to claim 58, wherein varying at least one of the modulation functions comprises varying the modulation function in response to the distance to the at least one object.

60. A method according to claim 59, wherein varying the modulation function in response to the distance to the at least one object comprises:

defining a range of distances that includes the distance to the at least one object; and

adjusting the modulation function so that detecting reflected radiation comprises detecting radiation from one or more regions of the scene that are within the range of distances.

61. A method according to claim 60, wherein identifying at least one object of interest comprises locating a plurality of points on the at least one object, and wherein defining the range of distances comprises defining a plurality of local ranges, each local range including a respective one of the plurality of points.

62. A method according to claim 60, wherein adjusting the modulation function comprises adjusting the function so that radiation from objects and regions of the scene that are outside the defined range of distances is not detected.

63. A method according to claim 60, wherein adjusting the modulation function comprises adjusting the function so that radiation from objects and regions of the scene that are outside the defined range of distances have a substantially constant intensity value distribution.

64. A method according to claim 60, and comprising forming a first two-dimensional image, said image comprising radiation detected from the one or more regions of the scene that are within the range of distances, and mixing said first two-dimensional image with a second, separately acquired or synthetically generated image, to form a mixed image.

65. A method according to claim 64, and comprising determining the location of a desired zone in the first two-dimensional image, wherein mixing the first two-dimensional image with the second, separately acquired image comprises inserting the second image in the desired zone.

66. A method according to claim 65, wherein inserting the second image in the desired zone comprises scaling the second image to fit in the desired zone.

67. A method according to claim 54, wherein detecting radiation comprises detecting radiation from two objects of interest in the scene, at substantially different respective distances, and

wherein varying at least one of the modulation functions comprises varying the modulation function so that the two objects have generally equal intensity values.

75. A method according to claim 54 or 68, wherein forming an image having an intensity value distribution indicative of the distance to objects comprises normalizing said intensity value distribution.

76. A method according to claim 75, wherein normalizing the intensity value distribution comprises correcting the distribution for varying emissivities of the object.

77. A method according to claim 75, wherein normalizing the intensity value distribution comprises correcting the distribution for different object distances.

78. A method according to claim 54 or 68, wherein detecting radiation comprises filtering the radiation according to color, and wherein forming an image comprises forming a color image.

68. A method for creating an image indicating distances to objects in a scene, comprising:

modulating radiation from a radiation source according to a first modulation function, and directing said radiation toward a scene;

defining a plurality of parallel spatial segments within the scene, each said segment having distance bounds;

detecting radiation reflected from each of the parallel spatial segments and generating signals responsive to the distance to regions of the scene, in response to the detected radiation; and

forming an image having an intensity value distribution indicative of the distance to objects in the scene,

wherein defining the plurality of spatial segments comprises defining the distance bounds of each of the plurality of segments, and

wherein the distance bounds of at least one of the plurality of spatial segments are different from the distance bounds of at least one other spatial segment.

69. A method according to claim 68, wherein the distance bounds of a first one of the plurality of spatial segments overlap the distance bounds of a second spatial segment, adjacent thereto.

70. A method according to claim 68, wherein defining a plurality of spatial segments having distance bounds comprises modulating the radiation reflected from each of the segments according to a respective modulation function.

71. A method according to claim 68, wherein detecting radiation comprises accumulating and reading out photoelectric charge produced in response to the radiation, and wherein defining the plurality of spatial segments comprises clearing the photoelectric charge produced in response to the radiation from each of the segments at a respective known, predetermined time interval before accumulating and reading out the charge.

72. A method according to claim 68, wherein directing radiation toward the scene comprises producing a line of radiation and scanning the line across the scene.

73. A method according to claim 68, wherein defining a plurality of spatial segments comprises defining a matrix of mutually adjacent spatial segments, wherein at least one segment within the matrix has different distance bounds from at least one other segment, and

wherein forming an image comprises combining the spatial segments within the matrix to produce an extended segment having distance bounds substantially equal to the union of the respective distance bounds of the segments in the matrix.

74. A method for creating an extended image-indicating distances to objects in a scene, comprising:

creating a first image of a scene, according to claim 68;

scanning the spatial segments laterally, relative to the scene;

creating a second image of the scene, in the same manner as the first image was created; and

registering and combining said first image with said second image, to create the extended image.