Method of three dimensional positioning using feature matching

• US 7,376,262 B2
• Filed: 08/04/2004
• Issued: 05/20/2008
• Est. Priority Date: 08/04/2003
• Status: Active Grant

• Current Assignee: American GNC Corporation
• Original Assignee: American GNC Corporation
• Inventors: Hu, Guohui, Coleman, Norman, Lin, Ching-Fang
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): ALLISON, ANDRAE S
• Application Number:
  US10/912,401
• Publication Number:
  US 20050031167A1
• Time in Patent Office: 1,385 Days
• Field of Search: 382/154, 382/156, 382/262, 382/168, 382/171, 382/103, 382/173, 382/106, 382/181, 382/190, 356/12
• US Class Current: 382/154
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

An object positioning solves said problems encountered in machine vision, which employs electro-optic (EO) image sensors enhanced with integrated laser ranger, global positioning system/inertial measurement unit, and integrates these data to get reliable and real time object position. An object positioning and data integrating system comprises EO sensors, a MEMS IMU, a GPS receiver, a laser ranger, a preprocessing module, a segmentation module, a detection module, a recognition module, a 3D positioning module, and a tracking module, in which autonomous, reliable and real time object positioning and tracking can be achieved.

Claims

1. A method of three dimensional positioning of objects, comprising the steps of:

(a) providing images of objects by two or more EO (electro-optic) sensors provided on a subject;

(b) preprocessing said images to form preprocessed images;

(c) segmenting said preprocessed images into segmented images;

(d) performing line detection, circle detection and eigenvector projection with said segmented images and performing corner detection and fiducial feature detection with said preprocessed images to obtain detected lines, detected circles, detected corners, and detected fiducial features;

(e) getting detected corners and fiducial 3D positions;

(f) performing inertial navigation system (INS) processing with GPS measurements, including position, velocity and time received from a global positioning system and inertial measurements, including body angular rates and specific forces, from an inertial measurement unit to obtain attitude and azimuth information of said subject; and

matching said preprocessed images and with said attitude and azimuth information;

(g) identifying a certain object by grouping said detected corners and fiducial 3D positions to obtain a recognized certain object including calculated 3D corners and fiducial features; and

(h) obtaining a 3D target position from said 3D corners and fiducial features.

2. The method, as recited in claim 1, wherein the step (f) further comprises a step of providing a laser ranger measurement between said objects and said subject and matching said laser ranger measurement with said preprocessed images.

3. The method, as recited in claim 1, wherein the step (b) further comprises the steps of:

(b-1) sending said images from said EO sensors to a preprocessing module;

(b-2) performing Median Filtering to suppress noise in a Median Filter module;

(b-3) performing Histogram Equalization to enhance said images in a Histogram Equalization module; and

(b-4) inverting said images in an Inverse Image module when an object image library is black.

4. The method, as recited in claim 2, wherein the step (b) further comprises the steps of:

(b-1) sending said images from said EO sensors to a preprocessing module;

(b-2) performing Median Filtering to suppress noise in a Median Filter module;

(b-3) performing Histogram Equalization to enhance said images in a Histogram Equalization module; and

(b-4) inverting said images in an Inverse Image module when an object image library is black.

5. The method, as recited in claim 3, wherein the step (c) further comprises the steps of:

(c-1) receiving said preprocessed images from said preprocessing module and performing Threshold Black/White in a Threshold/White module;

(c-2) performing Suppress Black in a Suppress Black module;

(c-3) performing Suppress White in a Suppress White module, and

(c-4) performing edge detection in a Sobel Filter module.

6. The method, as recited in claim 4, wherein the step (c) further comprises the steps of:

(c-1) receiving said preprocessed images from said preprocessing module and performing Threshold Black/White in a Threshold/White module;

(c-2) performing Suppress Black in a Suppress Black module;

(c-3) performing Suppress White in a Suppress White module, and

(c-4) performing edge detection in a Sobel Filter module.

7. The method, as recited in claim 5, wherein in the step (d), said segmented images is sent form said segmentation module to a detection module, wherein said line detection, said circle detection and said eigenvector projection are performed in a Line Detection Module, a Circle Detection module and an Eigenvector Projection module respectively.

8. The method, as recited in claim 6, wherein in the step (d), said segmented images is sent form said segmentation module to a detection module, wherein said line detection, said circle detection and said eigenvector projection are performed in a Line Detection Module, a Circle Detection module and an Eigenvector Projection module respectively.

9. The method, as recited in claim 7, wherein in the step (d), said preprocessed images is also sent from said preprocessing module to said detection module, wherein said corner detection and fiducial feature detection are performed a Corner Detection module and a Gabor Filter module.

10. The method, as recited in claim 8, wherein in the step (d), said preprocessed images is also sent from said preprocessing module to said detection module, wherein said corner detection and fiducial feature detection are performed a Corner Detection module and a Gabor Filter module.

11. The method, as recited in claim 9, wherein in the step (e), said detected corners and fiducial features are sent to a 3D Positioning module and a Recognition module.

12. The method, as recited in claim 10, wherein in the step (e), said detected corners and fiducial features are sent to a 3D Positioning module and a Recognition module.

13. The method, as recited in claim 11, wherein in the step (g), further comprises the steps of:

(g-1) receiving said preprocessed images from said preprocessing module, matching with a processed target template and outputting to a Tracking module;

(g-2) receiving said detected corners from said Corner Detection module;

(g-3) matching corners in said images to get disparities and calculating said 3D positions for each pair of said corners in said 3D Positioning module;

(g-4) receiving said detected fiducial features from said Gabor Filter module;

(g-5) matching corners in said images to get said disparities and calculating said 3D positions for each pair of said corners in said 3D Positioning module;

(g-6) receiving said detected lines, said detected circles, said detected corners, and said detected fiducial features from said Detection module and getting detected corners and fiducial 3D positions from said 3D Positioning module; and

(g-7) grouping said detected corners and fiducial 3D positions in a Graph Matching module, a Corner Classifier module and a Neural network module, to identify said certain object.

14. The method, as recited in claim 12, wherein in the step (g), further comprises the steps of:

(g-1) receiving said preprocessed images from said preprocessing module, matching with a processed target template and outputting to a Tracking module;

(g-2) receiving said detected corners from said Corner Detection module;

(g-3) matching corners in said images to get disparities and calculating said 3D positions for each pair of said corners in said 3D Positioning module;

(g-4) receiving said detected fiducial features from said Gabor Filter module;

(g-5) matching corners in said images to get said disparities and calculating said 3D positions for each pair of said corners in said 3D Positioning module;

(g-6) receiving said detected lines, said detected circles, said detected corners, and said detected fiducial features from said Detection module and getting detected corners and fiducial 3D positions from said 3D Positioning module; and

(g-7) grouping said detected corners and fiducial 3D positions in a Graph Matching module, a Corner Classifier module and a Neural network module, to identify said certain object.

15. The method, as recited in claim 13, wherein the step (h) further comprises a step of receiving said recognized certain object in a Relative Position Tracking module, wherein said recognized certain object includes calculated 3D corners and fiducial features, to get said 3D target position.

16. The method, as recited in claim 14, wherein the step (h) further comprises a step of receiving said recognized certain object in a Relative Position Tracking module, wherein said recognized certain object includes calculated 3D corners and fiducial features, to get said 3D target position.

17. The method, as recited in one of claim 14, wherein the step (g-1) further comprises the steps of:

(g-1-1) retrieving a target knowledge database to get said processed target template, receiving said attitude and azimuth from said AHRS/INS/GPS Integration module, and rotating said processed target template in a Matched Filter module;

(g-1-2) receiving said laser range from said Laser Ranger module, and selectively shrinking and enlarging said processed images from the step (a) in said Matched Filter module; and

(g-1-3) doing said Match Filter in said Matched Filtering module.