Method and apparatus for intelligent ranging via image subtraction

• US 20020176605A1
• Filed: 05/25/2001
• Published: 11/28/2002
• Est. Priority Date: 05/25/2001
• Status: Active Grant

• Current Assignee: Ioannis Kanellakopoulos, Nicholas Bambos, Oscar M. Stafsudd, Phyllis R. Nelson
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Kanellakopoulos, Ioannis, Nelson, Phyllis R., Bambos, Nicholas, Stafsudd, Oscar M.
• Granted Patent: US 6,711,280 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

The claimed system measures the relative position of objects with retroreflective surfaces, such as those contained in the taillights of all cars, trucks, and motorcycles. This system comprises an illuminator operating in the visible or near-infrared waveband, one or more imagers, each placed behind an e.g. fixed-focus optical lens and a bandpass filter, and a microprocessor. The microprocessor runs the software that operates the illuminator and the imagers, and processes the data collected from the imagers to produce the ranging information. Each set of ranging data is obtained by recording two images, the first one with the illuminator turned off and the second one with the illuminator turned on, and then subtracting the first image from the second one. The purpose of the subtraction process is to remove all data from the resulting image except for the returns of the retroreflective surfaces. This requires that (1) the two images are recorded in rapid succession, so that the ambient scene does not change, and (2) that both images are recorded on the same device, so that irregularities in the image intensity are automatically canceled out. This is achieved through a masking arrangement of the imagers, which allow both images to be stored on the same device before they are digitized and transferred to the microprocessor. The ranging information is then extracted from the resulting subtracted image via standard triangulation methods.

Claims

1. An apparatus for ranging an object comprising:

an illuminator to illuminate a field of view potentially including the object;

an imager to capture a first image having reflected signals from the field of view when the field of view is illuminated by the illuminator and a second image having reflected signals from the field of view when the field of view is not illuminated by the illuminator;

a circuit coupled to said imager to synchronously control said illuminator and said imager, and to generate a subtraction image of the field of view as the pixel difference between the first image and the second image; and

wherein the imager captures one of the first image and second image while the other one of the second image and first image is still captured in the imager.

2. The apparatus of claim 1, where said imager comprises an imaging pixel array in which said pixels of said array are organized into a two dimensional array comprising pixel lines forming a first and a second group of pixel lines, said first group of pixel lines being unmasked and said second group of pixel lines being masked to prevent loading of imaged data therein, and wherein a first exposure of said pixel array said circuit loads said first group of pixel lines with a first set of image data associated with the one of the first image and the second image, then shifts said first set of image data into said second group of pixel lines, and wherein a second exposure of said pixel array said circuit loads said first group of pixel lines with a second set of image data associated with the other one of the second image and the first image, and then shifts said second set of image data into said second group of pixel lines, said second set of pixel lines then storing said first and second set sets of image data.

3. The apparatus of claim 1, wherein said illuminator is placed proximate to the imager.

4. The apparatus of claim 3, wherein said illuminator is placed within approximately 5 centimeters of the imager.

5. The apparatus of claim 2, wherein said second group of pixel lines comprises a contiguous subarray of pixel lines including two thirds of said pixel array.

6. The apparatus of claim 5, wherein said pixel array comprises rows and columns of pixels and wherein said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of columns of said pixels.

7. The apparatus of claim 3, wherein said pixel array comprises rows and columns of pixels and wherein said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of rows of said pixels.

8. The apparatus of claim 2, wherein said first group of pixel lines comprises alternating pixel lines in a first half of said pixel array and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

9. The apparatus of claim 8, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixel lines forming said first group of pixel lines forms a set of columns of said pixels.

10. The apparatus of claim 8, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixel lines forming said first group of pixel lines forms a set of rows of said pixels.

11. The apparatus of claim 2, wherein said first group of pixel lines comprises alternating pixels in each line in a first half of said pixel array with each alternating pixel being offset from ones of said alternating pixels in adjacent lines of pixels to form a checkerboard pattern, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

12. The apparatus of claim 11, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixels forming said checkerboard pattern form a set of columns of said alternating pixels.

13. The apparatus of claim 11, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixels forming said checkerboard pattern form a set of rows of said alternating pixels.

14. The apparatus of claim 2, wherein said first group of pixel lines comprises contiguous pixel lines in a middle third of said pixel array, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

15. The apparatus of claim 14, wherein said pixel array comprises rows and columns of pixels and wherein middle third of said pixel array forming said first group of pixel lines forms a contiguous block of columns of said pixels.

16. The apparatus of claim 14, wherein said pixel array comprises rows and columns of pixels and wherein middle third of said pixel array forming said first group of pixel lines forms a contiguous block of rows of said pixels.

17. The apparatus of claim 2, wherein said first and second exposure are taken in time sequence without processing said image data between each exposure.

18. The apparatus of claim 1, wherein said first and second images are captured in time sequence separated by a time interval smaller than that which transpires between any substantial change in said first and second images of said field of view.

19. The apparatus of claim 18, wherein said time interval is not greater than approximately 10 milliseconds.

20. The apparatus of claim 1, wherein said circuit further determines distance to said object in said field of view from said imager.

21. The apparatus of claim 20, wherein said circuit determines absolute distance to said object in said field of view from said imager.

22. The apparatus of claim 20, wherein said circuit determines relative changes in distance to said object in said field of view from said imager.

23. The apparatus of claim 1, wherein:

said illuminator provides illumination in a narrow frequency band;

said imager is a camera; and

the apparatus further comprises a bandpass filter interposed between said camera and field of view, said bandpass filter being centered on said narrow frequency band of illumination of said illuminator.

24. The apparatus of claim 1, wherein said illuminator is modulated and said imager is locked to said modulation to receive reflected signals according to said modulation.

25. The apparatus of claim 1, wherein said circuit comprises:

a computer having a memory,

where said computer is controlled by a driver module to activate said illuminator and said imager synchronously with each other to capture said first and second images;

an image acquisition module to transfer said first and second images from said imager to said circuit;

an object detection module to detect reflective images in said subtraction image; and

a ranging module to compute said distance to said object.

26. The apparatus of claim 1, wherein a single imager is coupled to said circuit.

27. The apparatus of claim 1, further comprising a second imager coupled to said circuit, wherein the first imager is disposed proximate to the second imager by a fixed predetermined distance.

28. The apparatus of claim 27, wherein the illuminator is disposed proximate to the first imager and the second imager, and the illuminator is synchronized with the first imager and the second imager.

29. The apparatus of claim 27, further comprising a second illuminator, wherein:

the first illuminator is disposed proximate to the first imager and synchronized with the first imager; and

the second illuminator is disposed proximate to the second imager and synchronized with the second imager.

30. The apparatus of claim 1, further comprising a second imager coupled to the circuit, wherein:

the first imager has a first imager field of view and the second imager has a second imager field of view narrower than the first imager field of view.

31. The apparatus of claim 30, wherein:

the first illuminator comprises a first illuminator field of view;

the first illuminator is proximate to both the first imager and the second imager; and

the first illuminator is synchronized with the first imager and the second imager.

32. The apparatus of claim 30, further comprising a second illuminator, wherein:

the first illuminator comprises a first illuminator field of view and is disposed proximate to the first imager and synchronized with the first imager;

the second imager comprises a second illuminator field of view narrower than the first illuminator field of view and is disposed proximate to the second imager and synchronized with the second imager.

33. The apparatus of claim 1, further comprising a second, third, and fourth imager coupled to the circuit, wherein:

the first and second imagers comprise a first imager field of view;

the first and second imagers are separated from each other by a fixed, predetermined distance;

the third and fourth imagers comprise a second imager field of view narrower than the first imager field of view; and

the third and fourth imagers are physically separated from each other by a second predetermined distance.

34. The apparatus of claim 33, wherein the illuminator comprises a wide field of view and is disposed proximate to the first, second, third, and fourth imagers, and is synchronized with the first, second, third, and fourth imagers.

35. The apparatus of claim 33, further comprising a second illuminator, wherein:

the first illuminator comprises a first illuminator field of view;

the first illuminator is disposed proximate to the first and second imagers;

the first illuminator is synchronized with the first and second imagers;

the second illuminator comprises a second illuminator field of view narrower than the first illuminator field of view;

the second illuminator is disposed proximate to the third and the fourth imagers; and

the second illuminator is synchronized with the third and the fourth imagers.

36. The apparatus of claim 33, further comprising a second, third, and fourth illuminator, wherein:

the first and second illuminators comprise a first illuminator field of view;

the third and fourth illuminators comprise a second illuminator field of view narrower than the first illuminator field of view;

the first and second illuminators are disposed proximate to the first and the second imagers, respectively, and are synchronized with the first and second imagers, respectively; and

the third and fourth illuminators are disposed proximate to the third and fourth imagers, respectively, and are synchronized with the third and fourth imagers, respectively.

37. The apparatus of claim 1, wherein said reflected signals indicative of said object are reflected signals from reflectors on said object.

38. The apparatus of claim 1, wherein said reflected signals indicative of said object are reflected signals from a retroreflective surface on said object.

39. The apparatus of claim 1, wherein said imager comprises an array of pixel elements, where each pixel element receives said reflected signals and stores image data corresponding to said reflected signals at said pixel element, wherein said array of pixel elements is divided into at least a first and second set, said first set being operable only during a first exposure and said second set being operable only during a second exposure.

40. The apparatus of claim 39, wherein said first and second sets of pixels are operable by means of selective masking of said first and second sets from said reflected signals.

41. The apparatus of claim 40, wherein said first and second sets of pixels are operable by selective enablement of said first and second sets to store said image data.

42. The apparatus of claim 40, wherein said first and second sets of pixels are organized into corresponding sets of interleaved pixel elements with each pixel element of said first set being adjacent to at least one pixel element of said second set.

43. The apparatus of claim 1, wherein said circuit further comprises:

a sequence control circuit coupled to said imager for producing a stream of pixels from said imager corresponding to said first and second image;

a subtraction circuit coupled to said sequence control circuit for subtracting said second image from said first image on a pixel-by-pixel basis;

an analog-to-digital converter coupled to said subtraction circuit to generate a digitized subtraction image on a pixel-by-pixel basis; and

a processor coupled to said analog-to-digital converter for generating ranging parameters.

44. The apparatus of claim 1, wherein said circuit further comprises:

a sequence control circuit coupled to said imager for producing a stream of pixels from said imager corresponding to said first and second image;

an analog-to-digital converter coupled to said sequence control circuit to generate digitized images on a pixel-by-pixel basis;

a field programmable gate array coupled to said analog-to-digital converter to generate a digitized subtraction image on a pixel-by-pixel basis; and

a processor coupled to said field programmable gate array for generating ranging parameters.

45. The apparatus of claim 1, wherein said circuit further comprises:

a sequence control circuit coupled to said imager for producing a stream of pixels from said imager corresponding to said first and second image;

an analog-to-digital converter coupled to said sequence control circuit to generate digitized images on a pixel-by-pixel basis;

a subtractor coupled to said analog-to-digital converter to generate a digitized subtraction image on a pixel-by-pixel basis; and

a processor coupled to said application-specific integrated circuit for generating ranging parameters.

46. The apparatus of claim 1, wherein said illuminator is selected from the group comprising an LED, a headlight, and a laser.

47. The apparatus of claim 1, wherein said illuminator comprises a first and a second illuminator, said first illuminator arranged and configured to illuminate a near field of view and said second illuminator arranged and configured to illuminate a far field of view.

48. The apparatus of claim 43, wherein said imager has columns and rows of pixel detectors, wherein said sequence control circuit reads two said columns or rows from said imager and wherein said subtraction circuit subtracts said two columns or rows.

49. The apparatus of claim 44, wherein said imager has columns and rows of pixel detectors, wherein said sequence control circuit reads two said columns or rows from said imager and wherein said subtraction circuit subtracts said two columns or rows.

50. The apparatus of claim 45, wherein said imager has columns and rows of pixel detectors, wherein said sequence control circuit reads two said columns or rows from said imager and wherein said subtraction circuit subtracts said two columns or rows.

51. A method for ranging comprising:

illuminating a field of view with an illumination signal, which field of view potentially includes an object;

synchronously receiving reflected signals from the field of view with illumination and absence of illumination of the field of view;

capturing one of a first image and a second image within an array while the other one of the second image and the first image is still captured in the array, wherein the first image includes reflected signals from the field of view with illumination and the second image includes reflected signals in the absence of illumination; and

generating a subtraction image of at least a portion of said field of view comprising the pixel difference between said first and second images captured in said array.

52. The method of claim 51 wherein capturing one of a first image and a second image of an array comprises:

capturing said first and second images in an imaging pixel array in which said pixels of said array are organized into a two dimensional array comprising pixel lines forming a first and second groups of pixel lines, said first group of pixel lines being unmasked and said second group of pixel lines being masked to prevent loading of imaged data therein, and wherein a first exposure of said pixel array said circuit loads said first group of pixel lines with said one of the first image and the second image, then shifts said one of the first image and the second image into said second group of pixels, and wherein a second exposure of said pixel array said circuit loads said first group of pixel lines with the other one of the second image and the first image, and then shifts said second image into said second group of pixel lines, said second set of pixel lines then storing said first and second images.

53. The method of claim 51, wherein said illumination is performed by an illuminator disposed proximate to said array.

54. The method of claim 53, wherein the illumination is performed by an illuminator disposed less than approximately 5 centimeters from the array.

55. The method of claim 52, wherein capturing said first and second images comprises:

capturing said first and second images in said first group of pixel lines which comprises a contiguous subarray of pixel lines including one third of said pixel array, said second group of pixel lines including two thirds of said pixel array.

56. The method of claim 52, wherein said pixel array comprises rows and columns of pixels and wherein capturing said second image comprises:

capturing said second image in said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of columns of said pixels.

57. The method of claim 52, wherein said pixel array comprises rows and columns of pixels and wherein capturing said second image comprises:

capturing said second image in said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of rows of said pixels.

58. The method of claim 52, wherein capturing said first and second images comprises:

capturing each of said first and second images in alternating pixel lines in a first half of said pixel array and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

59. The method of claim 58, wherein said pixel array comprises rows and columns of pixels and where said first and second images captured in alternating pixel lines forming said first group of pixel lines forms a set of columns of said pixels.

60. The method of claim 58, wherein said pixel array comprises rows and columns of pixels and where said first and second images captured in alternating pixel lines forming said first group of pixel lines forms a set of rows of said pixels.

61. The method of claim 52, wherein capturing said first and second images comprises:

capturing said first and second images in alternating pixels in each line in a first half of said pixel array in said first group of pixel lines with each alternating pixel being offset from ones of said alternating pixels in adjacent lines of pixels to form a checkerboard pattern, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

62. The method of claim 61, wherein said pixel array comprises rows and columns of pixels and wherein capturing said first and second images comprises:

capturing said first and second images in alternating pixel lines of said first group of pixel lines which form a set of columns of said alternating pixels.

63. The method of claim 61, wherein said pixel array comprises rows and columns of pixels and wherein capturing said first and second images comprises:

capturing said first and second images in alternating pixel lines of said first group of pixel lines which form a set of rows of said alternating pixels.

64. The method of claim 52, wherein capturing said first and second images comprises:

capturing said first and second images in said first group of pixel lines which comprises contiguous pixel lines in a middle third of said pixel array, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

65. The method of claim 64, wherein said pixel array comprises rows and columns of pixels and wherein:

capturing said first and second images captures said first and second images in said middle third of said pixel array forms a contiguous block of columns of said pixels.

66. The method of claim 64, wherein said pixel array comprises rows and columns of pixels and wherein:

capturing said first and second images captures said first and second images in said middle third of said pixel array to form a contiguous block of rows of said pixels.

67. The method of claim 51, wherein capturing said first and second images is performed in time sequence without processing said image data between each capture.

68. The method of claim 51, wherein capturing said first and second images is performed in time sequence separated by a time interval smaller than that which transpires between any substantial change in said first and second images of said field of view.

69. The method of claim 68, wherein said time interval is approximately 10 milliseconds or less.

70. The method of claim 51, further comprising determining distance to said object in said field of view from a following object.

71. The method of claim 70, wherein determining distance determines absolute distance to said object in said field of view from said following object.

72. The method of claim 70, wherein determining distance determines relative changes in distance to said object in said field of view from said following object.

73. The method of claim 70, where illuminating said field of view substantially illuminates said field of view with a narrow band of frequency of light, and further comprises bandpass filtering received reflected signals.

74. The method of claim 51, further comprising modulating said illumination signal and wherein only modulated reflected signals are received.

75. The method of claim 51, further comprising detecting at least two objects in said subtraction image and computing a distance to said object based on configuration of said at least two objects.

76. The method of claim 51, wherein capturing said first and second images is performed by a single array.

77. The method of claim 51, wherein capturing said first and second images is performed by two arrays which are separated from each other at a fixed predetermined distance.

78. The method of claim 77, wherein:

the illuminator is disposed proximate to the first array and the second array and wherein the illumination is synchronized with the first array and the second array.

79. The method of claim 77, further comprising:

illuminating a second field of view with a second illumination signal, which second field of view potentially includes the object.

80. The method of claim 79, further comprising:

synchronously receiving reflected signal from said second field of view with illumination and in absence of illumination of the second field of view;

capturing a first image of the reflected signal from the second field of view within a second array when the second field of view is illuminated;

capturing a second image of the reflected signal from the second field of view when the second field of view is not illuminated within the second array; and

wherein one of the first image and the second image captured in the second array is captured while the other of the second image and the first image captured in the second array is still captured in the second array.

82. The method of claim 51, wherein:

the first and second images are further captured by a second array wherein the second array has a narrower field of view than the first array.

83. The method of claim 82, wherein:

the illuminator has a substantially wide field of view, and is placed proximate to the first and the second array, and wherein the illuminator is synchronized with the first and the second array.

84. The method of claim 82, wherein the illumination signal is produced by a first illuminator having a first field of view and wherein the reflected signal from the first field of view is received by a first array synchronized with the first illuminator, and wherein the method further comprises:

illuminating a second field of view with a second illumination signal produced by a second illuminator, which second field of view potentially includes the object, wherein the second field of view is narrower than the first field of view and wherein the second illuminator is synchronized with a second array;

synchronously receiving the reflected signal from said second field of view with illumination and in the absence of illumination of the second field of view with the second array;

capturing the first image of the reflected signal from the second field of view within the second array when the second field of view is illuminated;

capturing the second image of the reflected signal from the second field of view when the second field of view is not illuminated within the second array; and

wherein one of the first image and the second image captured in the second array is captured while the other of the second image and the first image captured in the second array is still captured in the second array.

85. The method of claim 84, further comprising:

generating a subtraction image of at least a portion of the second field of view comprising the pixel difference between the first and second images captured in the second array.

86. The method of claim 51, further comprising:

capturing the first and second images of the reflected signal within a plurality of arrays comprising the first array, a second array, a third array, and a fourth array when the field of view is illuminated, wherein two of the plurality of arrays comprise a narrower field of view than the other two arrays and are separated from each other by a fixed predetermined distance.

87. The method of claim 86, wherein said illumination is performed by a single illuminator having a wide field of view that is disposed proximate to the plurality of arrays and is synchronized with the plurality of arrays.

88. The method of claim 86, wherein:

said illumination is performed by a first illuminator having a first illuminator field of view and a second illuminator having a second illuminator field of view narrower than the first illuminator field of view,

the first illuminator disposed proximate to the first array having a first array field of view and the second array having the first array field of view and synchronized with the first array and the second array; and

the second illuminator disposed proximate to the third array having a second array field of view narrower than the first array field of view and a fourth array having the second array field of view and synchronized with the third array and the fourth array.

89. The method of claim 86, wherein:

said illumination is performed by a first illuminator having a first illuminator field of view, a second illuminator having the first illuminator field of view, a third illuminator having a second illuminator field of view narrower than the first illuminator field of view, and a fourth illuminator having the second illuminator field of view;

the first illuminator and the second illuminator disposed proximate to the first array having a first array field of view and the second array having the first array field of view and wherein the first illuminator and the second illuminator are synchronized with the first array and the second array; and

the third illuminator and the fourth illuminator disposed proximate to the third array having a second array field of view narrower than the first array field of view, and a fourth array having the second array field of view, wherein the third illuminator and the fourth illuminator are synchronized with the third array and the fourth array.

90. The method of claim 51, wherein receiving reflected signals from said field of view indicative of said object are reflected signals from reflectors on said object.

91. The method of claim 51, wherein receiving reflected signals from said field of view indicative of said object are reflected signals from retroreflective surfaces on said object.

81. The method of 80, wherein the subtraction image is generated from the pixel difference between the first and second images captured in the first array and the pixel difference between the first and second images captured in the second array.

92. A program storage device, readable by a computer, tangibly embodying at least one program of instructions executable by the computer to perform method steps of ranging, the method steps comprising the steps of:

illuminating a field of view with an illumination signal, which field of view potentially includes an object;

synchronously receiving reflected signals from the field of view with illumination and absence of illumination of the field of view;

capturing one of a first image and a second image within an array while the other one of the second image and the first image is still captured in the array, wherein the first image includes reflected signals from the field of view with illumination and the second image includes reflected signals in the absence of illumination; and

generating a subtraction image of at least a portion of said field of view comprising the pixel difference between said first and second images captured in said array.