Image processing method and apparatus for correlating a test image with a template

• US 5,946,422 A
• Filed: 01/13/1998
• Issued: 08/31/1999
• Est. Priority Date: 01/05/1996
• Status: Expired due to Fees

• Current Assignee: Mcdonnell Douglas Corporation (The Boeing Co.)
• Original Assignee: Mcdonnell Douglas Corporation (The Boeing Co.)
• Inventors: Wright, Steven A., Neff, Michael G., Johnson, Ted L., Ratkowski, Victor E. Jr.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US09/006,124
• Time in Patent Office: 595 Days
• Field of Search: 382/103, 382/170, 382/209, 382/219, 382/263, 382/278, 382/217, 382/218
• US Class Current: 382/278
• CPC Class Codes: F41G 7/007 Preparatory measures taken before the launching of the guided missiles
• CPC Class Codes: F41G 7/2226 comparing the observed data with stored target data, e.g. target configuration data
• CPC Class Codes: F41G 7/2253 Passive homing systems, i.e. comprising a receiver and do not requiring an active illumination of the target
• CPC Class Codes: F41G 7/2293 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding

Abstract

The image correlation method and apparatus correlates or matches a test image with a template. The image correlation apparatus includes an image processor for partitioning the template into a number of labels, for determining the total number of pixels N_T which form the template and for determining the number of pixels N_i which form each of the labels i. The image correlation apparatus also includes comparison means for comparing the test image to the template. The comparison means determines, for each predetermined gray level j, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template. The comparison means also determines, for each predetermined gray level j, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template. The image correlation apparatus further includes correlation means for determining the correlation X between the test image and the template according to a predetermined equation which is based, at least in part, upon N_j,i, N_j, N_i and N_T. The image correlation apparatus can also include an address generator for creating a number of relative offsets between the template and the test image.

Claims

1. An apparatus for comparing a test image to a template wherein both the test image and the template are comprised of a plurality of pixels representative of respective predetermined gray levels, and wherein the template is partitioned into a plurality of labels comprised of at least one pixel, the apparatus comprising:

template memory means for storing indicia representative of the plurality of pixels which comprise the template;

image memory means for storing indicia representative of the plurality of pixels which comprise the test image;

offset means, responsive to said template memory means, for creating a relative offset between the template and the test image;

a plurality of processing elements, responsive to said template memory means and said image memory means, for separately comparing the plurality of pixels which comprise the template to the plurality of pixels which comprise the test image such that at least some of the pixels of the test image correspond to respective pixels of the template, wherein at least one of said processing elements is further responsive to said offset means for comparing the plurality of pixels which comprise the template to the plurality of pixels which comprise the test image and which have a relative offset therebetween, and wherein said processing elements comprise;

means for determining, for each of a plurality of predetermined gray levels j, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to respective pixels of a predetermined label i of the template; and

means for determining, for each of a plurality of predetermined gray levels j, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template;

correlation means, responsive to said plurality of processing elements, for determining the correlation X between the test image and the template according to;

##EQU4## wherein N_T is the total number of pixels which comprise the template, N_i is the number of pixels which form each of the plurality of labels i of the template, i_min and i_max are the minimum and maximum label designations, respectively, and j_min and j_max are the minimum and maximum gray levels, respectively.

2. An apparatus according to claim 1 wherein said plurality of processing elements are disposed in parallel such that each processing element concurrently compares the test image to the template.

3. An apparatus according to claim 1 wherein said offset means generates a plurality of relative offsets between the template and the test image, and wherein each processing element separately compares the template to the test image at a respective one of the relative offsets.

4. An apparatus according to claim 3 wherein said N_j,i determining means comprises means for determining, at each relative offset, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining means comprises means for determining, at each relative offset, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation means comprises means for determining the correlation X between the test image and the template at each relative offset.

5. An apparatus according to claim 4 further comprising means for determining the relative offset between the test image and template which provides the greatest correlation therebetween.

6. An apparatus according to claim 1 further comprising rotation means, responsive to said template memory means, for creating at least one relative rotational offset between the template and the test image, wherein said plurality of processing elements are adapted to compare the test image to the template at each relative rotational offset.

7. An apparatus according to claim 6 wherein said N_j,i determining means comprises means for determining, at each relative rotational offset, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining means comprises means for determining, at each relative rotational offset, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation means comprises means for determining the correlation X between the test image and the template at each relative rotational offset.

8. An apparatus according to claim 7 further comprising means, responsive to said correlation means, for determining the relative rotational offset between the test image and template which provides the greatest correlation therebetween.

9. An apparatus according to claim 1 further comprising scaling means, responsive to said template memory means, for scaling the template relative to the test image according to at least one predetermined scale, wherein said plurality of processing elements are adapted to compare the test image to the template at each predetermined scale.

10. An apparatus according to claim 9 wherein said N_j,i determining means comprises means for determining, at each predetermined scale, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining means comprises means for determining, at each predetermined scale, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation means comprises means for determining the correlation X between the test image and the template at each predetermined scale.

11. An apparatus according to claim 10 further comprising the means, responsive to said correlation means, for determining the predetermined scale which provides the greatest correlation between the template and the test image.

12. An apparatus according to claim 1 wherein the plurality of pixels which comprise the test image and the template are assigned respective addresses, and wherein the apparatus further comprises an address generator which includes said offset means for creating a relative offset between the template and the test image based upon the respective addresses of the pixels which comprise the template and the test image.

13. An apparatus according to claim 1 wherein each processing element comprises a field programmable gate array and an associated memory device.

14. An apparatus according to claim 1 wherein said image memory means comprises a portion of each processing element.

15. An apparatus according to claim 1 wherein said correlation means comprises a portion of each processing element.

16. An apparatus according to claim 1 further comprising template processing means for processing the plurality of pixels which comprise the template, wherein said template processing means comprises:

means for partitioning the template into a plurality of labels, wherein each label is comprised of at least one pixel;

means for determining the total number of pixels N_T which comprise the template; and

means for determining the number of pixels N_i which form each of the plurality of labels i of the template.

17. An apparatus according to claim 1 wherein the template and the test image are obtained from first and second viewpoints, respectively, and wherein the apparatus further comprises an image processor for geometrically warping the template to compensate for differences between the first and second viewpoints prior to storing indicia representative of the plurality of pixels which comprise the template in said template memory means.

18. An apparatus according to claim 1 wherein the template and the test image are obtained from first and second viewpoints, respectively, and wherein the apparatus further comprises an image processor for texture mapping the template to compensate for differences between the first and second viewpoints prior to storing indicia representative of the plurality of pixels which comprise the template in said template memory means.

19. An apparatus according to claim 1 further comprising an image processor for removing at least some of the background shading from the template prior to storing indicia representative of the plurality of pixels which comprise the template in said template memory means.

20. An apparatus according to claim 1 further comprising an image processor for dividing the template into a plurality of pieces, and wherein said plurality of processing elements are responsive to said image processor for comparing each piece of the template to a predetermined range of the test image such that at least some deformation between the pieces of the template is permitted.

21. An apparatus according to claim 20 wherein said N_j,i determining means comprises means for separately determining, for each piece of the template, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the respective piece of the template.

22. An apparatus according to claim 20 wherein said N_j determining means comprises means for separately determining, for each piece of the template, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the respective piece of the template.

23. A method for recognizing a preselected object within a test image, the method comprising the steps of:

creating a template comprised of a plurality of pixels representative of predetermined gray levels, wherein said creating step comprises partitioning the template into a plurality of labels, and wherein each label is comprised of at least one pixel;

designating the selected object within the template;

capturing a test image comprised of a plurality of pixels representative of predetermined gray levels;

creating at least one relative offset between the template and the test image;

comparing the test image to the template at each of the relative offsets such that at least some of the pixels of the test image correspond to respective pixels of the template, wherein said comparing step comprises, for each relative offset, the steps of;

determining, for each of a plurality of predetermined gray levels j, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template;

determining, for each of a plurality of predetermined gray levels j, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template; and

determining the correlation X between the test image and the template according to;

##EQU5## wherein N_T is the total number of pixels which comprise the template, N_i is the number of pixels which form each of the plurality of labels i of the template, i_min and i_max are the minimum and maximum label designations, respectively, and wherein j_min and j_max are the minimum and maximum gray levels, respectively;

determining the relative offset between the test image and the template which provides the greatest correlation X therebetween; and

selecting an object within the test image at the relative offset which provides the greatest correlation X which corresponds to the designated object in the template.

24. A method according to claim 23 wherein said comparing step comprises the step of concurrently comparing the test image to the template at a plurality of the relative offsets.

25. A method according to claim 23 further comprising the step of creating at least one relative rotational offset between the template and the test image, wherein said comparing step comprises the step of comparing the test image to the template at each relative rotational offset, wherein said N_j,i determining step comprises the step of determining, at each relative rotational offset, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining step comprises the step of determining, at each relative rotational offset, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation determining step comprises the step of determining the correlation X between the test image and the template at each relative rotational offset.

26. A method according to claim 25 further comprising the step of determining the relative rotational offset between the test image and template which provides the greatest correlation therebetween.

27. A method according to claim 23 further comprising the step of scaling the template relative to the test image according to at least one predetermined scale, wherein said comparing step comprises the step of comparing the test image to the template at each predetermined scale, wherein said N_j,i determining step comprises the step of determining, at each predetermined scale, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining step comprises the step of determining, at each predetermined scale, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation determining step comprises the step of determining the correlation X between the test image and the template at each predetermined scale.

28. A method according to claim 27 further comprising the step of determining the predetermined scale which provides the greatest correlation between the template and the test image.

29. A method according to claim 23 further comprising the steps of:

determining the total number of pixels N_T which comprise the template; and

determining the number of pixels N_i which form each of the plurality of labels i of the template.

30. A method according to claim 23 wherein said template creating step comprises creating the template from a first viewpoint, wherein said test image capturing step comprises capturing the test image from a second viewpoint, and wherein the method further comprises the steps of:

geometrically warping the template, prior to said comparing step, to compensate for differences between the first and second viewpoints; and

storing indicia representative of the plurality of pixels which comprise the geometrically warped template.

31. A method according to claim 23 wherein said template creating step comprises creating the template from a first viewpoint, wherein said test image capturing step comprises capturing the test image from a second viewpoint, and wherein the method further comprises the steps of:

texture mapping the template, prior to said comparing step, to compensate for differences between the first and second viewpoints; and

storing indicia representative of the plurality of pixels which comprise the texture mapped template.

32. A method according to claim 23 further comprising the steps of:

removing at least some of the background shading from the template prior to said comparing step; and

storing indicia representative of the plurality of pixels which comprise the template following the removal of at least some of the background shading.

33. A method according to claim 23 further comprising the step of dividing the template into a plurality of pieces, and wherein said comparing step comprises the step of comparing each piece of the template to a predetermined region of the test image such that at least some deformation between the pieces of the template is permitted.

34. A method according to claim 33 wherein said N_j,i determining step comprises the step of separately determining, for each piece of the template, the number of pixels of the test image N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the respective piece of the template.

35. A method according to claim 33 wherein said N_j determining step comprises the step of separately determining, for each piece of the template, the number of pixels of the test image N_j representative of a predetermined gray level j which correspond to the respective piece of the template.

36. A method according to claim 23 wherein said test image capturing step comprises the step of capturing a plurality of temporally distinct test images, and wherein said comparing step comprises the step of separately comparing each of the temporally distinct test images to the template, wherein said N_j,i determining step comprises the step of separately determining the number of pixels of each of the temporally distinct test images N_j,i representative of a predetermined gray level j which correspond to a predetermined label i of the template, wherein said N_j determining step comprises the step of separately determining the number of pixels of the temporally distinct test images N_j representative of a predetermined gray level j which correspond to the template, and wherein said correlation determining step comprises the step of separately determining the correlation X between each of the temporally distinct test images and the template.

37. A method according to claim 36 wherein said relative offset determining step comprises the steps of:

determining, for each relative offset, a sum of the correlations X determined for each of temporally distinct test images at the respective offset which have been weighted according to a predetermined time- based formula; and

determining the relative offset between the test image and the template which provides the greatest sum of weighted correlations X over time.