Method and device for tracking relative movement by correlating signals from an array of photoelements

• US 5,729,008 A
• Filed: 01/25/1996
• Issued: 03/17/1998
• Est. Priority Date: 01/25/1996
• Status: Expired due to Term

• Current Assignee: Agilent Technologies, Inc., Hewlett-Packard Company (HP Inc.), Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Baumgartner, Richard A., Blalock, Travis N., Smith, Mark T., Hornak, Thomas
• Primary Examiner(s): Westin, Edward P.
• Assistant Examiner(s): Lee, John R.
• Application Number:
  US08/591,847
• Time in Patent Office: 782 Days
• Field of Search: 250/234, 250/559.29, 250/559.3, 250/559.32, 250/559.33, 250/559.34, 250/559.36, 250/559.37, 250/559.39, 250/548, 250/208.1, 358/474, 358/482, 358/483, 358/486, 358/488, 358/496, 358/497, 358/473, 348/86, 348/88, 348/94, 348/95, 348/169
• US Class Current: 250/208.1
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 1/047 Detection, control or error compensation of scanning velocity or position H04N1/0402 and H04N1/17 take precedence
• CPC Class Codes: H04N 1/107 with manual scanning
• CPC Class Codes: H04N 2201/04712 using unbroken arrays of detectors, i.e. detectors mounted on the same substrate
• CPC Class Codes: H04N 2201/04729 in the main-scan direction
• CPC Class Codes: H04N 2201/04731 in the sub-scan direction
• CPC Class Codes: H04N 2201/04734 Detecting at frequent intervals, e.g. once per line for sub-scan control
• CPC Class Codes: H04N 2201/04743 by detecting the image directly detecting reference images H04N2201/04717

Abstract

A method and device for tracking relative movement between the device and a region of interest include correlating a reference frame of signals with a subsequently acquired sample frame of signals, with the correlating including generating a correlation output for each of a number of nearest-neighbor shifts of signals of one of the frames. Preferably, the frames are comprised of a two-dimensional array of pixels and the signals are pixel values. The pixels of a frame are operatively associated with photoelements in a two-dimensional array of photoelements, with the pixel values being indicative of light energy received at the photoelements. By correlating the reference frame and the sample frame using the nearest-neighbor approach, it is possible to determine the movement of an imaged feature during the time period between acquisition of the reference frame and acquisition of the sample frame. In the preferred embodiment, the pixel values of the reference frame may be selectively shifted to correspond to the movement of the image feature, allowing the nearest-neighbor approach to continue even after the relative movement exceeds a pixel length. Also in the preferred embodiment, the device is a scanner having a second photoelement array. The second photoelement array captures image data which is position-tagged by navigation data acquired by means of the correlation of reference and sample frames.

Claims

1. A method of tracking relative movement between a device and a region of interest, the method comprising steps of:

fixing a two-dimensional array of photoelements to said device such that each photoelement generates an output responsive to light energy received at said photoelement from said region of interest;

deriving frames of signals from the outputs from said array of photoelements at specific times such that successive frames have respective signals with regard to a correspondence between a particular one of said photoelements and particular signals within said successive frames, including deriving a first frame of said signals at a first specific time and deriving a second frame of said signals at a second specific time;

correlating said signals of said first frame to said signals of said second frame, including substeps of;

(a) selecting signals from said first frame in a shifted relationship to respective signals of said second frame such that said signals selected from said first frame are offset from said respective signals of said second frame by a shift amount, said shift amount having a correspondence with a shift of a single photoelement within said array of photoelements,

(b) generating a correlation output responsive to said signals selected from said first frame in substep (a) and to said respective signals of said second frame, and

(c) repetitively executing substeps (a) and (b) to generate a correlation output for each of a plurality of shift directions; and

based on said correlation outputs generated in said correlating step, determining a relative movement between said device and said region of interest between said first specific time and said second specific time.

2. The method of claim 1 wherein said substep (c) of repetitively executing substeps (a) and (b) includes selecting eight shift directions relative to an unshifted position, thereby generating at least eight correlation outputs.

3. The method of claim 2 wherein said step of correlating said signals of said first and second frames further includes generating a ninth correlation output that is responsive to said signals of said first frame being in an unshifted relationship to respective signals of said second frame.

4. The method of claim 1 wherein said step of deriving said frames is a step of capturing frames of pixel values indicative of light energy received at said photoelements at said specific times.

5. The method of claim 4 wherein said step of correlating said signals of said first and second frames includes repetitively translating said captured second frame of pixel values relative to said captured first frame of pixel values, thereby enabling generation of one of said correlation outputs for each translation.

6. The method of claim 1 further comprising steps of storing said first frame and correlating said signals of said stored first frame to signals of a plurality of frames derived after said first frame, wherein each of said substeps (a), (b) and (c) is executed for each correlation.

7. The method of claim 1 wherein each substep (b) of generating a correlation output includes determining a value equal to a square of a difference between a signal of said first frame with a respective signal of said second frame.

8. The method of claim 1 wherein said step of fixing said two-dimensional array to said device is a step of fixing a navigation sensor array to a scanner having an image array of photoelements.

9. The method of claim 1 wherein said step of deriving said frames of signals is a step of acquiring images of said region of interest.

10. A device for tracking relative movement between said device and a region of interest, the device comprising:

a navigation sensor having a two-dimensional array of photoelements responsive to reception of light energy to define an array field of view;

first circuit means for deriving a sequence of frames of signals in which each signal in each frame of said sequence is indicative of light energy from said region of interest received at an individual photoelement at a specific time, said each frame thereby being indicative of features within said region of interest when said region of interest is in said array field of view, said first circuit means outputting said signals, each of said signals corresponding to one photoelement of said array; and

second circuit means, connected to said first circuit means, for correlating first and second frames of said sequence of frames of signals that are output from said first circuit means to determine displacement of said features within said region of interest during a time period between deriving said first and second frames, said second circuit means including an array of correlation signal-generating circuits in one-to-one correspondence with said signals, each correlation-signal generating circuit receiving from said first frame the signal corresponding to an individual photoelement of said array and additionally being selectively connectable to receive from said second frame the signals corresponding to other photoelements immediately adjacent to said individual photoelement in said array, each correlation-signal generating circuit generating a correlation signal output responsive to said signal from said first frame and to said signals from said second frame, said correlation signal outputs being indicative of correlations between light energy received at said individual photoelement for deriving said first frame and light energy received at said other photoelements for deriving said second frame, thereby enabling said determination of displacement of said features.

11. The device of claim 10 further comprising an image sensor in fixed relation to said navigation sensor, said image sensor having a plurality of photoelements for forming image information of said region of interest.

12. The device of claim 11 further comprising means for position-tagging said image information based upon said outputs from said array of correlation signal-generating circuits.

13. The device of claim 10 wherein said correlation signal-generating circuits include difference-squared circuits, each having a first input from said individual photoelement of said array and having a second input sequentially connected to said other photoelements.

14. A method of tracking movement of a scanner device relative to a surface of interest to form an image of said surface, the method comprising steps of:

fixing an image sensor to said scanner device;

fixing a two-dimensional array of photoelements to said scanner device, each of said photoelements in said array having plural nearest-neighbor photoelements adjacent thereto;

forming first image information of said surface of interest using said image sensor;

forming second image information of said surface of interest using said array of photoelements;

assembling an image of said surface of interest from said first image information, including positioning said first image information based on said second image information, including substeps of;

(a) correlating patterns of surface reflectivity detected by said array of photoelements at a first specific time and a second specific time, including generating plural correlation outputs for each photoelement, said correlation outputs being responsive to light energy received at said each photoelement at said first specific time and light energy received at said nearest-neighbor photoelements of each said photoelement at said second specific time, and

(b) based on said patterns of surface reflectivity correlated in substep (a), determining a magnitude and direction of travel of said scanning device relative to said surface of interest.

15. The method of claim 14 wherein said substep of generating plural correlation outputs includes forming nine outputs that are equal to the squares of the differences between a pixel value indicative of light energy received at said each photoelement at said first specific time and nine pixel values indicative of light energy received by said nearest-neighbor photoelements and by said each photoelement at said second specific time later than said first specific time.

16. The method of claim 14 wherein said step of fixing a two-dimensional array of photoelements includes a step of fixing first and second two-dimensional arrays of photoelements to said scanner device at spatially separated locations, and wherein said substeps (a) and (b) of correlating patterns and determining said magnitude and direction of travel are executed based upon said second image information from each of said first and second arrays.

17. A scanner for forming an image of a surface of interest comprising:

a housing;

an imaging sensor fixed to said housing for capturing image information of said surface of interest;

a two-dimensional array of photoelements fixed to said housing for forming navigation information, each photoelement in said array having at least one nearest-neighbor photoelement adjacent to said each photoelement;

frame means for capturing successive frames of said navigation information from said photoelements in an arrangement such that each frame includes pixel values in which each pixel value is responsive to received light energy at a corresponding photoelement at a time of capturing said each frame;

means, connected to said frame means, for correlating each pixel value of a first frame with those pixel values of a second frame that correspond to nearest-neighbor photoelements of a photoelement with which said each pixel value corresponds.

18. The scanner of claim 17 wherein each of said two-dimensional array, said frame means and said means for correlating are formed on a single substrate.

19. The scanner of claim 17 further comprising means, connected to said means for correlating, for arranging said image information in response to detection of correlations of pixel values of said first and second frames.

20. The scanner of claim 19 further comprising a second array of photoelements fixed to said housing for forming navigation information, said second array being connected to a second frame means and a second means for correlating.