Method and apparatus for intelligent ranging via image subtraction

• US 6,711,280 B2
• Filed: 05/25/2001
• Issued: 03/23/2004
• Est. Priority Date: 05/25/2001
• Status: Expired due to Fees

• Current Assignee: Ioannis Kanellakopoulos, Nicholas Bambos, Oscar M. Stafsudd, Phyllis R. Nelson
• Original Assignee: Ioannis Kanellakopoulos, Nicholas Bambos, Oscar M. Stafsudd, Phyllis R. Nelson
• Inventors: Kanellakopoulos, Ioannis, Nelson, Phyllis R., Bambos, Nicholas, Stafsudd, Oscar M.
• Primary Examiner(s): Patel, Jayanti K.
• Assistant Examiner(s): Choobin, Barry
• Application Number:
  US09/866,072
• Publication Number:
  US 20020176605A1
• Time in Patent Office: 1,033 Days
• Field of Search: 382/106, 382/194, 382/299, 382/260, 348/139, 348/622, 702/142, 702/157, 359/15, 359/196, 359/385, 340/903, 340/436, 356/4.04, 356/3, 356/5.11, 396/110
• US Class Current: 382/106
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

A method and system for ranging an object are disclosed. The method includes illuminating a field of view potentially including the object, synchronously receiving reflected signals from the field of view with and without illumination, capturing first and second images within an array, and generating a subtraction image using the images. One image is captured in the array while the other image is in the array. The first and second images include reflected signals from the field of view with and without illumination, respectively. The array includes first and second groups of lines that are unmasked and masked, respectively. In one exposure, the first group of lines is loaded with the first or second image. The image in the first group of lines is shifted into the second group. In another exposure, the first group of lines is loaded with the other image, which is shifted into the second group.

Claims

1. An apparatus for ranging an object comprising:

an illuminator to illuminate a field of view potentially including the object;

an imager to capture a first image having reflected signals from the field of view when the field of view is illuminated by the illuminator and a second image having reflected signals from the field of view when the field of view is not illuminated by the illuminator;

a circuit coupled to said imager to synchronously control said illuminator and said imager, and to generate a subtraction image of the field of view as the pixel difference between the first image and the second image; and

wherein the imager captures one of the first image and second image while the other one of the second image and first image is still captured in the imager; and

wherein said imager comprises an imaging pixel array in which said pixels of said array are organized into a two dimensional array comprising pixel lines forming a first and a second group of pixel lines, said first group of pixel lines being unmasked and said second group of pixel lines being masked to prevent loading of imaged data therein, and wherein a first exposure of said pixel array said circuit loads said first group of pixel lines with a first set of image data associated with the one of the first image and the second image, then shifts said first set of image data into said second group of pixel lines, and wherein a second exposure of said pixel array said circuit loads said first group of pixel lines with a second set of image data associated with the other one of the second image and the first image, and then shifts said second set of image data into said second group of pixel lines, said second set of pixel lines then storing said first and second set sets of image data.

2. The apparatus of claim 1, wherein said illuminator is placed proximate to the imager.

3. The apparatus of claim 2, wherein said illuminator is placed within approximately 5 centimeters of the imager.

4. The apparatus of claim 1, wherein said second group of pixel lines comprises a contiguous subarray of pixel lines including two thirds of said pixel array.

5. The apparatus of claim 4, wherein said pixel array comprises rows and columns of pixels and wherein said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of columns of said pixels.

6. The apparatus of claim 2, wherein said pixel array comprises rows and columns of pixels and wherein said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of rows of said pixels.

7. The apparatus of claim 1, wherein said first group of pixel lines comprises alternating pixel lines in a first half of said pixel array and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

8. The apparatus of claim 7, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixel lines forming said first group of pixel lines forms a set of columns of said pixels.

9. The apparatus of claim 7, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixel lines forming said first group of pixel lines forms a set of rows of said pixels.

10. The apparatus of claim 1, wherein said first group of pixel lines comprises alternating pixels in each line in a first half of said pixel array with each alternating pixel being offset from ones of said alternating pixels in adjacent lines of pixels to form a checkerboard pattern, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

11. The apparatus of claim 10, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixels forming said checkerboard pattern form a set of columns of said alternating pixels.

12. The apparatus of claim 10, wherein said pixel array comprises rows and columns of pixels and wherein alternating pixels forming said checkerboard pattern form a set of rows of said alternating pixels.

13. The apparatus of claim 1, wherein said first group of pixel lines comprises contiguous pixel lines in a middle third of said pixel array, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

14. The apparatus of claim 13, wherein said pixel array comprises rows and columns of pixels and wherein middle third of said pixel array forming said first group of pixel lines forms a contiguous block of columns of said pixels.

15. The apparatus of claim 13, wherein said pixel array comprises rows and columns of pixels and wherein middle third of said pixel array forming said first group of pixel lines forms a contiguous block of rows of said pixels.

16. The apparatus of claim 1, wherein said first and second exposure are taken in time sequence without processing said image data between each exposure.

17. The apparatus of claim 1, wherein said first and second images are captured in time sequence separated by a time interval smaller than that which transpires between any substantial change in said first and second images of said field of view.

18. The apparatus of claim 17, wherein said time interval is not greater than approximately 10 milliseconds.

19. The apparatus of claim 1, wherein said circuit further determines distance to said object in said field of view from said imager.

20. The apparatus of claim 19, wherein said circuit determines absolute distance to said object in said field of view from said imager.

21. The apparatus of claim 19, wherein said circuit determines relative changes in distance to said object in said field of view from said imager.

22. The apparatus of claim 1, wherein:

said illuminator provides illumination in a narrow frequency band;

said imager is a camera; and

the apparatus further comprises a bandpass filter interposed between said camera and field of view, said bandpass filter being centered on said narrow frequency band of illumination of said illuminator.

23. The apparatus of claim 1, wherein said illuminator is modulated and said imager is locked to said modulation to receive reflected signals according to said modulation.

24. The apparatus of claim 1, wherein said circuit comprises:

a computer having a memory,

where said computer is controlled by a driver module to activate said illuminator and said imager synchronously with each other to capture said first and second images;

an image acquisition module to transfer said first and second images from said imager to said circuit;

an object detection module to detect reflective images in said subtraction image; and

a ranging module to compute said distance to said object.

25. The apparatus of claim 1, further comprising a second imager coupled to said circuit, wherein the first imager is disposed proximate to the second imager by a fixed predetermined distance.

26. The apparatus of claim 25, wherein the illuminator is disposed proximate to the first imager and the second imager, and the illuminator is synchronized with the first imager and the second imager.

27. The apparatus of claim 25, further comprising a second illuminator, wherein:

the first illuminator is disposed proximate to the first imager and synchronized with the first imager; and

the second illuminator is disposed proximate to the second imager and synchronized with the second imager.

28. The apparatus of claim 1, further comprising a second imager coupled to the circuit, wherein:

the first imager has a first imager field of view and the second imager has a second imager field of view narrower than the first imager field of view.

29. The apparatus of claim 28, wherein:

the first illuminator comprises a first illuminator field of view;

the first illuminator is proximate to both the first imager and the second imager; and

the first illuminator is synchronized with the first imager and the second imager.

30. The apparatus of claim 28, further comprising a second illuminator, wherein:

the first illuminator comprises a first illuminator field of view and is disposed proximate to the first imager and synchronized with the first imager;

the second imager comprises a second illuminator field of view narrower than the first illuminator field of view and is disposed proximate to the second imager and synchronized with the second imager.

31. A method for ranging comprising:

illuminating a field of view with an illumination signal, which field of view potentially includes an object;

synchronously receiving reflected signals from the field of view with illumination and absence of illumination of the field of view;

capturing one of a first image and a second image within an array while the other one of the second image and the first image is still captured in the array, wherein the first image includes reflected signals from the field of view with illumination and the second image includes reflected signals in the absence of illumination, wherein the array is a pixel array and wherein capturing one of the first image and the second image of the array comprises

capturing said first and second images in the pixel array in which said pixels of said pixel array are organized into a two dimensional array comprising pixel lines forming a first and second groups of pixel lines, said first group of pixel lines being unmasked and said second group of pixel lines being masked to prevent loading of imaged data therein, and wherein a first exposure of said pixel array a circuit loads said first group of pixel lines with said one of the first image and the second image, then shifts said one of the first image and the second image into said second group of pixels, and wherein a second exposure of said pixel array said circuit loads said first group of pixel lines with the other one of the second image and the first image, and then shifts said second image into said second group of pixel lines, said second set of pixel lines then storing said first and second images; and

generating a subtraction image of at least a portion of said field of view comprising the pixel difference between said first and second images captured in said array.

32. The method of claim 31, wherein said illumination is performed by an illuminator disposed proximate to said array.

33. The method of claim 32, wherein the illumination is performed by an illuminator disposed less than approximately 5 centimeters from the array.

34. The method of claim 31, wherein capturing said first and second images comprises:

capturing said first and second images in said first group of pixel lines which comprises a contiguous subarray of pixel lines including one third of said pixel array, said second group of pixel lines including two thirds of said pixel array.

35. The method of claim 31, wherein said pixel array comprises rows and columns of pixels and wherein capturing said second image comprises:

capturing said second image in said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of columns of said pixels.

36. The method of claim 31, wherein said pixel array comprises rows and columns of pixels and wherein capturing said second image comprises:

capturing said second image in said contiguous subarray of pixel lines forming said second group of pixel lines forms a block of rows of said pixels.

37. The method of claim 31, wherein capturing said first and second images comprises:

capturing each of said first and second images in alternating pixel lines in a first half of said pixel array and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

38. The method of claim 37, wherein said pixel array comprises rows and columns of pixels and where said first and second images captured in alternating pixel lines forming said first group of pixel lines forms a set of columns of said pixels.

39. The method of claim 37, wherein said pixel array comprises rows and columns of pixels and where said first and second images captured in alternating pixel lines forming said first group of pixel lines forms a set of rows of said pixels.

40. The method of claim 31, wherein capturing said first and second images comprises:

capturing said first and second images in alternating pixels in each line in a first half of said pixel array in said first group of pixel lines with each alternating pixel being offset from ones of said alternating pixels in adjacent lines of pixels to form a checkerboard pattern, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

41. The method of claim 40, wherein said pixel array comprises rows and columns of pixels and wherein capturing said first and second images comprises:

capturing said first and second images in alternating pixel lines of said first group of pixel lines which form a set of columns of said alternating pixels.

42. The method of claim 40, wherein said pixel array comprises rows and columns of pixels and wherein capturing said first and second images comprises:

capturing said first and second images in alternating pixel lines of said first group of pixel lines which form a set of rows of said alternating pixels.

43. The method of claim 31, wherein capturing said first and second images comprises:

capturing said first and second images in said first group of pixel lines which comprises contiguous pixel lines in a middle third of said pixel array, and wherein said second group of pixel lines comprises all remaining pixel lines in said pixel array.

44. The method of claim 43, wherein said pixel array comprises rows and columns of pixels and wherein:

capturing said first and second images captures said first and second images in said middle third of said pixel array forms a contiguous block of columns of said pixels.

45. The method of claim 43, wherein said pixel array comprises rows and columns of pixels and wherein:

capturing said first and second images captures said first and second images in said middle third of said pixel array to form a contiguous block of rows of said pixels.

46. The method of claim 31, wherein capturing said first and second images is performed in time sequence without processing said image data between each capture.

47. The method of claim 31, wherein capturing said first and second images is performed in time sequence separated by a time interval smaller than that which transpires between any substantial change in said first and second images of said field of view.

48. The method of claim 47, wherein said time interval is approximately 10 milliseconds or less.

49. The method of claim 31, further comprising determining distance to said object in said field of view from a following object.

50. The method of claim 49, wherein determining distance determines absolute distance to said object in said field of view from said following object.

51. The method of claim 49, wherein determining distance determines relative changes in distance to said object in said field of view from said following object.

52. The method of claim 49, where illuminating said field of view substantially illuminates said field of view with a narrow band of frequency of light, and further comprises bandpass filtering received reflected signals.

53. The method of claim 31, further comprising modulating said illumination signal and wherein only modulated reflected signals are received.

54. The method of claim 31, further comprising detecting at least two objects in said subtraction image and computing a distance to said object based on configuration of said at least two objects.

55. The method of claim 31, wherein capturing said first and second images is performed by two arrays which are separated from each other at a fixed predetermined distance.

56. The method of claim 55, wherein:

the illuminator is disposed proximate to the first array and the second array and wherein the illumination is synchronized with the first array and the second array.

57. The method of claim 55, further comprising:

illuminating a second field of view with a second illumination signal, which second field of view potentially includes the object.

58. The method of claim 57, further comprising:

synchronously receiving reflected signal from said second field of view with illumination and in absence of illumination of the second field of view;

capturing a first image of the reflected signal from the second field of view within a second array when the second field of view is illuminated;

capturing a second image of the reflected signal from the second field of view when the second field of view is not illuminated within the second array; and

wherein one of the first image and the second image captured in the second array is captured while the other of the second image and the first image captured in the second array is still captured in the second array.

60. The method of claim 31, wherein:

the first and second images are further captured by a second array wherein the second array has a narrower field of view than the first array.

61. The method of claim 60, wherein:

the illuminator has a substantially wide field of view, and is placed proximate to the first and the second array, and wherein the illuminator is synchronized with the first and the second array.

62. The method of claim 60, wherein the illumination signal is produced by a first illuminator having a first field of view and wherein the reflected signal from the first field of view is received by a first array synchronized with the first illuminator, and wherein the method further comprises:

illuminating a second field of view with a second illumination signal produced by a second illuminator, which second field of view potentially includes the object, wherein the second field of view is narrower than the first field of view and wherein the second illuminator is synchronized with a second array;

synchronously receiving the reflected signal from said second field of view with illumination and in the absence of illumination of the second field of view with the second array;

capturing the first image of the reflected signal from the second field of view within the second array when the second field of view is illuminated;

capturing the second image of the reflected signal from the second field of view when the second field of view is not illuminated within the second array; and

wherein one of the first image and the second image captured in the second array is captured while the other of the second image and the first image captured in the second array is still captured in the second array.

63. The method of claim 62, further comprising:

generating a subtraction image of at least a portion of the second field of view comprising the pixel difference between the first and second images captured in the second array.

59. The method of 58, wherein the subtraction image is generated from the pixel difference between the first and second images captured in the first array and the pixel difference between the first and second images captured in the second array.

64. A program storage device, readable by a computer, tangibly at least one program of instructions executable by the computer to perform method ranging, the method steps comprising the steps of:

illuminating a field of view with an illumination signal, which field of view potentially includes an object;

synchronously receiving reflected signals from the field of view with illumination and absence of illumination of the field of view;

capturing one of a first image and a second image within an array while the other one of the second image and the first image is still captured in the array, wherein the first image includes reflected signals from the field of view with illumination and the second image includes reflected signals in the absence of illumination, wherein capturing one of the first image and the second image of the array comprises

capturing said first and second images in an imaging pixel array in which said pixels of said array are organized into a two dimensional array comprising pixel lines forming a first and second groups of pixel lines, said first group of pixel lines being unmasked and said second group of pixel lines being masked to prevent loading of imaged data therein, and wherein a first exposure of said pixel array a circuit loads said first group of pixel lines with said one of the first image and the second image, then shifts said one of the first image and the second image into said second group of pixels, and wherein a second exposure of said pixel array said circuit loads said first group of pixel lines with the other one of the second image and the first image, and then shifts said second image into said second group of pixel lines, said second set of pixel lines then storing said first and second images; and

generating a subtraction image of at least a portion of said field of view comprising the pixel difference between said first and second images captured in said array.