Neural net based processor for synthetic vision fusion

• US 20050232512A1
• Filed: 04/20/2004
• Published: 10/20/2005
• Est. Priority Date: 04/20/2004
• Status: Abandoned Application

• Current Assignee: Max-Viz, Inc. (Astronics Corporation), Oregon Health & Science University
• Original Assignee: Max-Viz, Inc. (Astronics Corporation), Oregon Health & Science University
• Inventors: Kerr, J. Richard, Luk, Chiu Hung, Hammerstrom, Daniel W., Pavel, Misha
• Application Number:
  US10/828,742
• Publication Number:
  US 20050232512A1
• Time in Patent Office: Days
• US Class Current: 382/276
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition

Abstract

A synthetic vision fused integrated enhanced vision system includes a data base of images of an objective; a non-HVS sensor array for providing a sensor output from each sensor in the array; a feature extraction mechanism for extracting multi-resolution features of an objective and forming a single, fused feature image of the objective; a registration mechanism for comparing the fused feature image to a database of expected features of the objective and for providing registered sensor output vectors; an association engine for processing the registered sensor output vectors with the database of objective images, including an associative match mechanism for comparing the registered sensor output vectors to a data base of objective images and providing comparison vectors therefrom for selecting an objective image for display; and a HVS display for displaying a HVS perceptible image from the data base objective images.

Claims

1. A synthetic vision fused integrated enhanced vision system, comprising:

a data base of images of an objective stored in a memory;

a non-HVS sensor array for providing a sensor output from each sensor in the array;

a feature extraction mechanism for extracting multi-resolution features of an objective, and for forming a single, fused feature image of the objective the sensor outputs;

a registration mechanism for comparing the extracted fused, feature image to a database of expected features of the objective and for providing registered sensor output vectors;

an association engine for processing the registered sensor output vectors with the database of objective images;

including an associative match mechanism for comparing the registered sensor output vectors to said data base of images of the objective, and providing comparison vectors therefrom for selecting an objective image for display; and

a HVS display for displaying a HVS perceptible image from the data base objective images.

2. The system of claim 1 wherein said sensor array includes a LWIR sensor, a SWIR sensor and a MMW sensor.

3. The system of claim 1 wherein the single, fused feature image is formed by vector addition of sensor outputs.

4. The system of claim 1 wherein said feature extraction mechanism includes V1 feature detection and K-WTA processing.

5. The system of claim 1 wherein said associative match mechanism includes a best match mechanism.

6. The system of claim 1 wherein said associative match mechanism includes an exact match mechanism.

7. The system of claim 6 wherein said HVS display displays an image of an objective from said database, and wherein a comparison vector points to an image of an objective in said database after said exact match mechanism locates an exact match between a fused feature image and an image of an objective in said database.

8. The system of claim 7 wherein the input for said exact match mechanism is output from a best match mechanism.

9. The system of claim 1 wherein the registration mechanism normalizes a feature image of the objective across sensor modalities.

10. The system of claim 1 which approximates the operation of a Voronoi classifier for training the association engine with an enhanced feature image.

11. The system of claim 1 which includes a hazard detection mechanism for comparing the registered sensor output vector to a best match comparison of the output vector to the objective image database to identify possible incursion of the objective by a hazardous entity.

12. The system of claim 1 which includes a confidence monitor using entropy as a heuristic measure of system integrity.

13. A method of forming a synthetically fused image comprising:

detecting an objective with a sensor array;

providing a sensor output from each sensor in the sensor array and providing a data base of objective images;

extracting features of the objective from each sensor output;

forming a single, fused feature image from the extracted features of each sensor output;

registering the extracted features with known features of the objective to provide registered sensor output vectors;

processing the registered sensor output vectors in an association engine to locate an objective image of the objective in the data base of objective images; and

displaying a HVS perceptible image from the objective image data base.

14. The method of claim 13 wherein said detecting includes providing a sensor array having a LWIR sensor, a SWIR sensor and a MMW sensor.

15. The method of claim 13 wherein said registering includes normalizing a feature image of the objective across sensor modalities.

16. The method of claim 13 wherein said association engine performs a Voronoi classification for training the association engine with an enhanced feature image.

17. The method of claim 13 wherein said extracting features includes V1-like feature extraction using a K-WTA protocol.

18. The method of claim 13 wherein said registering the extracted features with known features of the objective to provide registered sensor output vectors includes comparing extracted features with known features of a generic representation of a class of similar objectives.

19. The method of claim 13 wherein said processing the registered sensor output vectors in an association engine includes processing by a neural network.

20. The method of claim 13 which includes processing using edge extraction.

21. The method of claim 13 which includes processing by a Palm association engine process.

22. The method of claim 13 wherein said forming a single, fused feature image includes forming a fused feature image by adding vectors of extracted vectors.

23. The method of claim 13 wherein said processing includes a best match comparison between the registered sensor output vector and the data base of objective images.

24. The method of claim 23 which further includes detecting hazards by comparing the registered sensor output vectors to the best match comparison to identify possible incursion of the objective by a hazardous entity.

25. The method of claim 13 wherein said processing includes an exact match comparison between the registered sensor output vector and the data base of objective images, and generating a pointer from the exact match comparison.

26. The method of claim 25 wherein said displaying includes displaying an image selected from the database of objective images as indicated by the pointer.

27. The method of claim 25 wherein said exact match comparison includes using a registered sensor output vector as an input to a best match comparison, and using the best match output vector as the exact match input.