Image processing method and apparatus for correlating a test image with a template

• US 5,982,930 A
• Filed: 01/13/1998
• Issued: 11/09/1999
• Est. Priority Date: 01/05/1996
• Status: Expired due to Term

• Current Assignee: Mcdonnell Douglas Corporation (The Boeing Co.)
• Original Assignee: Mcdonnell Douglas Corporation (The Boeing Co.)
• Inventors: Wright, Steven A., Neff, Michael G., Johnson, Ted L., Ratkowski, Victor E. Jr.
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US09/006,402
• Time in Patent Office: 665 Days
• Field of Search: 382/209, 382/216, 382/217, 382/218, 382/219, 382/221, 382/278, 382/294, 382/304, 382/103, 364/819, 364/820, 348/169
• US Class Current: 382/209
• CPC Class Codes: F41G 7/007 Preparatory measures taken before the launching of the guided missiles
• CPC Class Codes: F41G 7/2226 comparing the observed data with stored target data, e.g. target configuration data
• CPC Class Codes: F41G 7/2253 Passive homing systems, i.e. comprising a receiver and do not requiring an active illumination of the target
• CPC Class Codes: F41G 7/2293 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding

Abstract

The image correlation method and apparatus correlates or matches a test image with a template. The image correlation apparatus includes an image processor for partitioning the template into a number of labels, for determining the total number of pixels N_T which form the template and for determining the number of pixels N_i which form each of the labels i. The image correlation apparatus also includes comparison means for comparing the test image to the template. The comparison means determines, for each predetermined gray level j, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template. The comparison means also determines, for each predetermined gray level j, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template. The image correlation apparatus further includes correlation means for determining the correlation X between the test image and the template according to a predetermined equation which is based, at least in part, upon N_j,i, N_j, N_i and N_T. The image correlation apparatus can also include an address generator for creating a number of relative offsets between the template and the test image. Thus, the test image can be compared to the template at each relative offset and the relative offset which provides the greatest correlation therebetween can be determined. Consequently, the test image and the template can be effectively matched such that a preselected object designated within the template can be located and identified within the test image.

Claims

1. An apparatus for recognizing a preselected object within a test image, comprising:

template generation means for creating a template comprised of a plurality of pixels representative of predetermined gray levels, wherein said template generation means comprises template processing means for partitioning the template into a plurality of labels, and wherein each label is comprised of at least one pixel;

object designation means, responsive to said template generation means, for designating the selected object within the template;

image capturing means for capturing a test image comprised of a plurality of pixels representative of predetermined gray levels;

offset means, responsive to said template generation means and said image capturing means, for creating at least one relative offset between the template and the test image;

comparison means, responsive to said template generation means and said image capturing means, for comparing the test image to the template at each of the relative offsets such that at least some of the pixels of the test image correspond to respective pixels of the template, wherein said comparison means comprises;

means for determining, at each relative offset and for each of a plurality of predetermined gray levels j, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template;

means for determining, at each relative offset and for each of a plurality of predetermined gray levels j, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template; and

correlation means for determining, at each relative offset, the correlation X between the test image and the template according to;

##EQU4## wherein N_T is the total number of pixels which comprise the template, N_i is the number of pixels which form each of the plurality of labels i of the template, i_min and i_max are the minimum and maximum label designations, respectively, and wherein j_min and j_max are the minimum and maximum gray levels, respectively;

means, responsive to said comparison means, for determining the relative offset between the test image and the template which provides the greatest correlation X therebetween; and

object recognition means, responsive to said object designation means and said relative offset determining means, for selecting an object within the test image at the relative offset which provides the greatest correlation X which corresponds to the designated object in the template.

2. An apparatus according to claim 1 further comprising rotation means for creating at least one relative rotational offset between the template and the test image, wherein said comparison means is responsive to said rotation means for comparing the test image to the template at each relative rotational offset, wherein said N_j,i determining means comprises means for determining, at each relative rotational offset, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining means comprises means for determining, at each relative rotational offset, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation means comprises means for determining the correlation X between the test image and the template at each relative rotational offset.

3. An apparatus according to claim 2 further comprising means, responsive to said correlation means, for determining the relative rotational offset between the test image and template which provides the greatest correlation therebetween.

4. An apparatus according to claim 1 further comprising scaling means for scaling the template relative to the test image according to at least one predetermined scale, wherein said comparison means is responsive to said scaling means for comparing the test image to the template at each predetermined scale, wherein said N_j,i determining means comprises means for determining, at each predetermined scale, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining means comprises means for determining, at each predetermined scale, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation means comprises means for determining the correlation X between the test image and the template at each predetermined scale.

5. An apparatus according to claim 4 further comprising the means, responsive to said correlation means, for determining the predetermined scale which provides the greatest correlation between the template and the test image.

6. An apparatus according to claim 5 wherein said template processing means comprises:

means for determining the total number of pixels N_T which comprise the template; and

means for determining the number of pixels N_i which form each of the plurality of labels i of the template.

7. An apparatus according to claim 1 wherein the template and the test image are obtained from first and second viewpoints, respectively, and wherein the apparatus further comprises:

an image processor for geometrically warping the template to compensate for differences between the first and second viewpoints; and

template memory means for storing indicia representative of the plurality of pixels which comprise the geometrically warped template.

8. An apparatus according to claim 1 wherein the template and the test image are obtained from first and second viewpoints, respectively, and wherein the apparatus further comprises:

an image processor for texture mapping the template to compensate for differences between the first and second viewpoints; and

template memory means for storing indicia representative of the plurality of pixels which comprise the texture mapped template.

9. An apparatus according to claim 1 further comprising:

an image processor for removing at least some of the background shading from the template; and

template memory means for storing indicia representative of the plurality of pixels which comprise the template having at least some of the background shading removed therefrom.

10. An apparatus according to claim 1 wherein said template processing means comprises means for dividing the template into a plurality of pieces, and wherein said comparison means further comprises means for comparing each piece of the template to a predetermined range of the test image such that at least some deformation between the pieces of the template is permitted.

11. An apparatus according to claim 10 wherein said N_j,i determining means comprises means for separately determining, for each piece of the template, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the respective piece of the template.

12. An apparatus according to claim 10 wherein said N_j determining means comprises means for separately determining, for each piece of the template, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the respective piece of the template.

13. An apparatus according to claim 1 wherein said image capturing means comprises means for capturing a plurality of temporally distinct test images, and wherein said comparison means comprises means for separately comparing each of the temporally distinct test images to the template, wherein said N_j,i determining means comprises means for separately determining the number of pixels of each of the temporally distinct test images N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining means comprises means for separately determining the number of pixels of the temporally distinct test images N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation means comprises means for separately determining the correlation X between each of the temporally distinct test images and the template.

14. An apparatus according to claim 13 wherein said relative offset determining means comprises:

means for determining, for each relative offset, a sum of the correlations X determined for each of temporally distinct test images at the respective offset which have been weighted according to a predetermined time-based formula; and

means for determining the relative offset between the test image and the template which provides the greatest sum of weighted correlations X over time.

15. An apparatus according to claim 1 wherein said template processing means comprises:

means for dividing the plurality of predetermined gray levels into a plurality of bins, wherein each bin includes a range of gray levels;

means, responsive to said dividing means, for determining, for each of the plurality of bins, an actual count of the number of pixels which are representative of gray levels within the range of gray levels included within the respective bin;

means, responsive to said actual count determining means, for assigning a predetermined minimum count to each bin which has an actual count determined by said count determining step which is less than the predetermined minimum count; and

means, responsive to said actual count determining means and said assigning means, for reassigning the plurality of predetermined gray levels to the plurality of bins such that the greater of the predetermined minimum count or the actual count for each respective bin is within a predetermined range, wherein the pixels which are representative of gray levels within a respective bin form a label.

16. An apparatus according to claim 15 wherein said template processing means further comprises:

means for determining the maximum number of pixels of the template which are representative of a single predetermined gray level; and

means, responsive to said means for determining the maximum number of pixels, for computing the predetermined minimum count to be equal to a predetermined percentage of the maximum number of pixels which are representative of a single predetermined gray level.