Model-based feature tracking system

• US 5,801,970 A
• Filed: 12/06/1995
• Issued: 09/01/1998
• Est. Priority Date: 12/06/1995
• Status: Expired due to Term

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Martin Marietta Corporation
• Inventors: Kreeger, Kevin A., Rowland, Patricia J., Sanders, Albert R.
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): ROBERTS, ANDREW S
• Application Number:
  US08/568,286
• Time in Patent Office: 1,000 Days
• Field of Search: 395/502, 395/500, 395/391, 395/958, 364/413.13, 364/424.027, 364/424.029, 364/424.031, 364/424.032, 364/424.033, 364/424.038, 364/424.039, 364/231.9, 342/63-65
• US Class Current: 703/6
• CPC Class Codes: G01S 13/66 Radar-tracking systems; Analogous systems
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 15/66 Sonar tracking systems
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/251 involving models

Abstract

Methods and apparatus providing model-based feature tracking for location assessment of a sensor platform, or tracking of a feature are disclosed wherein a series of predictive models are developed to predict a location of features in subsequently acquired sensor data. Feature location data acquired by a sensor in a previous platform position assessment is used to build a model corresponding to anticipated feature locations in data acquired for a present platform position assessment.

Claims

1. A method for tracking a feature within data gathered by a sensor, the method comprising the steps of:

developing an a priori model corresponding to a first location assessment point, which model contains data identifying the feature and an anticipated location of the feature;

extracting feature location data from sensor data gathered at the first location assessment point;

comparing the extracted feature location data with the anticipated feature location data;

making a correction involving the sensor based on results of the comparison;

building a new model with the feature location data extracted from the sensor data gathered at the first location assessment point; and

at the first location assessment point, predicting the location of the feature in sensor data to be gathered at a second location assessment point using the new model.

2. The method as claimed in claim 1 wherein the a priori model is developed based on one of a photograph, topographical data, contour image data, and anatomical data.

3. The method as claimed in claim 1 wherein the step of extracting feature location data includes pattern matching.

4. The method as claimed in claim 1 wherein the step of comparing involves utilizing a Hough transform.

5. The method as claimed in claim 1 wherein the step of making a correction involving the sensor based on results of the comparison includes one of an adjustment to the sensor, reorientation of a platform to which the sensor is attached, calculation of a correction factor, and correcting a reference point.

6. The method of claim 1 further comprising the step of:

replacing the a priori model with the new model.

7. A method for maintaining a point of reference as a sensor scans an object where one of the sensor and the object move relative to the other, the method comprising the steps of:

sensing the object at a present assessment point to produce image data;

extracting feature location data from the image data;

prior to a second assessment point, using the feature location data to prepare a first model which contains information identifying a feature and an anticipated location of the feature in a subsequent image associated with the second assessment point;

comparing the anticipated feature location data in the first model to feature location data extracted from the subsequent image;

determining when a match occurs between the anticipated feature location data in the first model and the feature location data extracted from the subsequent image;

correcting a sensor orientation based on conditions corresponding to when a match occurs between the model data and the subsequent image data; and

replacing the first model with a new model, wherein the new model is constructed using the feature location data extracted from the subsequent image.

8. The method of claim 7 wherein the step of correcting a sensor includes changing an orientation of a platform to which the sensor is attached.

9. A method for tracking a feature within sensor data gathered along a succession of location assessment points, the method comprising the steps of:

acquiring a priori data containing the feature;

extracting first feature location data from the a priori data;

building a first predictive model, based on the first feature location data, which first predictive model contains data identifying the feature and an anticipated location of the feature in sensor data to be gathered at a first location assessment point;

acquiring a plural number of sensor data sets at the first location assessment point, each of the plural sensor data sets corresponding to a sensor orientation in a sensor scan pattern;

extracting feature identification and location data from each of the plural sensor data sets and comparing the extracted feature identification and location data from each data set with the feature identification data and anticipated feature location data associated with the first predictive model until a match is found;

using a sensor orientation corresponding to the match to align the sensor; and

at the first location assessment point, preparing a second predictive model with the extracted feature identification and location data corresponding to the match, wherein the second predictive model contains data identifying the feature and an anticipated location of the feature at a second location assessment point.

10. The method as claimed in claim 9 wherein alignment of the sensor includes one of adjusting the mechanical orientation of the sensor, reorienting a platform to which the sensor is attached, calculating a sensor correction factor, and correcting a point of reference.

11. The method of claim 9 further comprising the step of:

replacing the first predictive model with the second predictive model.

12. A method for providing correction for a sensor in a sensor platform which gathers sensor data at successive position assessment points, the method comprising the steps of:

forming a succession of models each corresponding to one of the position assessment points, wherein each model contains data for predicting the location of features in sensor data to be gathered at a subsequent position assessment point, each model being formed prior to the subsequent position assessment point based on sensor data exclusively associated with a current position assessment point;

acquiring a sensor data set at each position assessment point;

extracting sensor feature location data from each sensor data set;

comparing extracted sensor feature location data and model feature location data to determine a difference between a location of the feature; and

aligning the sensor in accordance with the difference.

13. A method for tracking a feature appearing in successive sensor data sets acquired by a sensor, the method comprising the steps of:

extracting feature location data from a priori information;

forming a first predictive model based on the extracted feature location data, which first predictive model provides anticipated feature location data within a first position assessment area;

acquiring a plural number of first sensor data sets in the first position assessment area;

extracting feature location data from each sensor data set

comparing the extracted feature location data to the anticipated feature location data until a match is found;

correcting the sensor based on conditions corresponding to the match; and

at the first position assessment area, forming a second predictive model from the extracted feature location data corresponding to the match, wherein the second predictive model provides anticipated feature location data for a second position assessment area.

14. The method of claim 13 further comprising the step of:

replacing the first predictive model with the second predictive model.

15. An apparatus for maintaining a correct point of reference as a sensor scans an object where one of the sensor and the object move relative to the other, the apparatus comprising:

means for sensing the object at a first assessment point to produce image data;

means for extracting location data for a feature from the image data;

means for using the feature location data prior to a second assessment point to prepare a model which contains information identifying the feature and an anticipated location of the feature in a first subsequent image associated with the second assessment point;

means for comparing the anticipated location with feature location data extracted from the first subsequent image;

means for determining a match between the anticipated location of the feature and the feature location data extracted from the first subsequent image;

means for performing a correction involving the sensor based on conditions corresponding to the match ; and

at the second assessment point, means for using the feature location data extracted from the first subsequent image to prepare a new model which contains information regarding an anticipated location of the feature in a second subsequent image.

16. The apparatus as claimed in claim 15 wherein the correction means performs one of a sensor alignment, sensor platform reorientation, or calculation of a sensor data correction factor to maintain the correct point of reference.

17. The apparatus as claimed in claim 15 wherein the sensing means is one of an infra-red sensor, a radar set, or video camera.

18. The apparatus as claimed in claim 15 wherein a processor is used to perform sensor and model data management.

19. The apparatus as claimed in claim 18 wherein the processor is comprised of at least one GAPP™

chip.

20. The apparatus as claimed in claim 15 wherein the comparing means uses a Hough transform.

21. The apparatus of claim 15 further comprising:

means for replacing the first model with the new model.

22. An apparatus for tracking a feature appearing in data acquired by a sensor, the apparatus comprising:

at a present feature location assessment point, means for preparing a model for predicting a location of the feature in sensor data to be gathered at a future location assessment point, wherein the model is based on feature location data exclusively associated with the present feature location assessment point;

means for extracting feature location data from sensor data gathered at the future location assessment point;

means for comparing the predicted feature location from the model with the feature location data extracted from the sensor data gathered at the future location assessment point;

means for determining when the comparison results in a match;

means for deriving sensor conditions corresponding to the match; and

means for correcting a sensor orientation based on the conditions corresponding to the match.

23. An article of manufacture for use in a computer for tracking a feature through a succession of sensor data acquired by a sensor comprising a computer usable medium having computer readable program code means for causing the computer to:

effect acquisition of a first set of data containing feature information associated with a first feature location assessment point;

effect extraction of first feature data from the first set of data;

prior to a second feature location assessment point, effect building of a first predictive model based on the extracted first feature data, which first predictive model provides anticipated feature locations in a second set of data to be gathered at the second feature location assessment point;

at the second feature location assessment point, effect acquisition of the second set of data using the sensor;

effect extraction of feature location data from the second set of sensor data;

effect comparison of feature location data extracted from the second set of sensor data with the location data from the first predictive model to provide a comparison result;

effect use of the comparison result to correct a sensor angle; and

effect use of the feature location data extracted from the second set of sensor data to prepare a second predictive model, independent of any feature location data other than the feature location data from the second set of sensor data.

24. The article of manufacture described in claim 23 further comprising computer readable program code means for causing the computer to:

effect replacement of the first predictive model with the second predictive model.