Robust ground-plane homography estimation using adaptive feature selection

• US 9,165,208 B1
• Filed: 03/10/2014
• Issued: 10/20/2015
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Jeong, Changsoo S., Kim, Kyungnam, Chen, Yang, Khosla, Deepak, Cheng, Shinko Y., Zhang, Lei, Honda, Alexander L.
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US14/203,470
• Time in Patent Office: 589 Days
• Field of Search: 382/103, 382/173, 382/190
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

Described is system and method for robust ground-plane homography estimation using adaptive feature selection. The system determines feature correspondences of an image that correspond with at least one moving object in each image in a set of images. Additionally, feature correspondences of the image that correspond with at least one above-ground object are determined in each image. Feature correspondences that correspond with each moving object in each image are excluded, and feature correspondences that correspond with each above-ground object in each image are excluded. Each image is divided into a plurality of sub-regions comprising features correspondences. The number of feature correspondences in each sub-region is limited to a predetermined threshold to ensure that feature correspondences are evenly distributed over each image. Finally, a ground-plane homography estimation between the set of images is generated.

Claims

1. A system for homography estimation, the system comprising one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when the instructions are executed, the one or more processors perform operations of:

determining feature correspondences that correspond with at least one moving object in each image in a set of images;

determining feature correspondences that correspond with at least one above-ground object in each image in the set of images;

excluding feature correspondences that correspond with each moving object in each image in the set of images;

excluding feature correspondences that correspond with each above-ground object in each image in the set of images;

dividing each image in the set of images into a plurality of sub-regions comprising feature correspondences;

limiting the number of feature correspondences in each sub-region to a predetermined threshold to ensure that feature correspondences are evenly distributed over each image in the set of images; and

generating a ground-plane homography estimation between the set of images by excluding the feature correspondences that correspond with moving and above-ground objects in the set of images and limiting the number of feature correspondences in each sub-region to ensure that feature correspondences are evenly distributed over each image in the set of images.

2. The system as set forth in claim 1, wherein the one or more processors further performs an operation of limiting the number of feature correspondences in each sub-region to an average, number of feature correspondences of all sub-regions of the image.

3. The system as set forth in claim 2, wherein the one or more processors further performs an operation of determining the average number of feature correspondences of all sub-regions of the image, k, according to the following:

4. The system as set forth in claim 3, wherein if the ratio of n_i to N is greater than the predetermined threshold, then k feature correspondences are randomly selected from the number of feature correspondences on the i^th sub-region for generating the ground-plane homography estimation between the images.

5. A computer-implemented method for homography estimation, comprising an act of causing a data processor to execute instructions stored on a non-transitory memory such that upon execution, the data processor performs operations of:

determining feature correspondences that correspond with at least one moving object in each image in a set of images;

determining feature correspondences that correspond with at least one above- ground object in each image in the set of images;

excluding feature correspondences that correspond with each moving object in each image in the set of images;

excluding feature correspondences that correspond with each above-ground object in each image in the set of images;

dividing each image in the set of images into a plurality of sub-regions comprising feature correspondences;

limiting. the number of feature correspondences in each sub-region to a predetermined threshold to ensure that feature correspondences are evenly distributed over each image in the set of images; and

generating a ground-plane homography estimation between the set of images by excluding the feature correspondences that correspond with moving and above-ground objects in the set of images and limiting the number of feature correspondences in each sub-region to ensure that feature correspondences are evenly distributed over each image in the set of images.

6. The method as set forth in claim 5, wherein the data processor further performs an operation of limiting the number of feature correspondences in each sub-region to an average number of feature correspondences of all sub-regions of the image.

7. The method as set forth in claim 6, wherein the data processor further performs an operation of determining the average number of feature correspondences of all sub-regions of the image, k, according to the following:

8. The method as set forth in claim 7, wherein if the ratio of n_i to N is greater than the predetermined threshold, then k feature correspondences are randomly selected from the number of feature correspondences on the i^th sub-region for generating the ground-plane homography estimation between the images.

9. A computer program product for homography estimation, the computer program product comprising computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

determining feature correspondences that correspond with at least one moving object in each image in a set of images;

determining feature correspondences that correspond with at least one above-ground object in each image in the set of images;

excluding feature correspondences that correspond with each moving object in each image in the set of images;

excluding feature correspondences that correspond with each above-ground object in each image in the set of images;

dividing, each image in the set of images into a plurality of sub-regions comprising feature correspondences;

limiting the number of feature correspondences in each sub-region to a predetermined threshold to ensure that feature correspondences are evenly distributed over each image in the set of images; and

generating a ground-plane homography estimation between the set of images by excluding the feature correspondences that correspond with moving and above-ground objects in the set of images and limiting the number of feature correspondences in each sub-region to ensure that feature correspondences are evenly distributed over each image in the set of images.

10. The computer program product as set forth in claim 9, further comprising instructions for causing the processor to perform an operation of limiting the number of feature correspondences in each sub-region to an average number of feature correspondences of all sub-regions of the image.

11. The computer program product as set forth in claim 10, further comprising instructions for causing the processor to perform an operation of determining the average number of feature correspondences of all sub-regions of the image, k, according to the following:

12. The computer program product as set forth in claim 11, wherein if the ratio of n_i to N is greater than the predetermined threshold, then k feature correspondences are randomly selected from the number of feature correspondences on the i^th sub-region for generating the ground-plane homography estimation between the images.