Method and apparatus for aligning and comparing images of the face and body from different imagers

• US 6,496,594 B1
• Filed: 10/21/1999
• Issued: 12/17/2002
• Est. Priority Date: 10/22/1998
• Status: Expired due to Term

• Current Assignee: Francine J. Prokoski
• Original Assignee: Francine J. Prokoski
• Inventors: Prokoski, Francine J.
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): DESIRE, GREGORY M
• Application Number:
  US09/422,273
• Time in Patent Office: 1,153 Days
• Field of Search: 382/115-118, 382/124-126, 382/299-300, 382/276, 382/278, 463/29, 235/380
• US Class Current: 382/118
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/415 the glands, e.g. tonsils, adenoids or thymus
• CPC Class Codes: A61B 5/418 lymph vessels, ducts or nodes
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A method and apparatus for comparing an infrared image of a person to a database of visual images of persons and calculating the probability that each is a match to the infrared image is characterized by extracting minutiae from the infrared image and extracting visible minutiae from the visible images. Coincident minutiae which occur in both spectra are used to scale and register the infrared and the visible images. Other minutiae are spectrum-dependent, but must obey rules relative to minutiae of the other spectrum, due to the anatomical structure of the human face and body. The primary application is for identification of persons seen in infrared surveillance imagery, using a reference database of visual images. Other applications include compression of talking head video and animation of synthetic faces. The method and apparatus can also be applied to areas of the body other than the face, to compare images from different spectra including images from medical sensors.

Claims

1. A method for aligning anatomical images taken from first and second different spectral bands, comprising the steps of

(a) identifying minutiae in a first anatomical image which are dependent upon the first spectral band thereof, said first spectral band comprising a visual band;

(b) identifying minutiae in a second anatomical image which are dependent upon the second spectral band thereof, said second spectral band comprising an infrared band;

(c) identifying coincident minutiae which occur in said first and second anatomical images and which correspond to characteristic features of the anatomical images; and

(d) morphing one anatomical image to the other anatomical image to overlay the coincident minutiae, whereby the anatomical images are in alignment.

2. A method as defined in claim 1, and further comprising the step of calculating the degree of morphing required to achieve optimal alignment, said morphing step including the steps of stretching, warping, and shrinking said one image with respect to said other image.

3. A method for identifying an unknown person from a first anatomical image taken in a first spectral band by comparing it with a plurality of second anatomical images of known persons taken in a second different spectral band and stored in a database, said first and second spectral bands comprising visual and infrared bands, comprising the steps of

(a) identifying minutiae in each anatomical image which are dependent upon the spectral band thereof;

(b) identifying coincident minutiae which occur in the anatomical images and which correspond to a characteristic feature of the images;

(c) overlaying the coincident minutiae by morphing said first anatomical image onto each of said second anatomical images,

(d) selecting one of said second anatomical images requiring the least morphing as the most likely matching anatomical image from said database and eliminating the remaining second anatomical images; and

(e) comparing the degree of morphing with a threshold to determine whether the match is sufficient to identify the unknown person as one from the database.

4. A method as defined in claim 3, and further comprising the stop of assigning a level of confidence to the match based on the difference between the threshold and the degree of morphing.

5. A method as defined in claim 3 and further comprising the steps of

(f) selecting those second images which have a degree of morphing within the threshold as possible matches to the first image;

(g) establishing exclusion zones for the second images;

(h) determining whether any features of the first image extend into the exclusion zones of the second images when they are aligned and overlaid;

(i) discarding from further consideration any second images whose exclusion zones are violated by the first image;

(j) considering any remaining second images as possible candidates for matching the first image;

(k) rank ordering the possible candidates by the degree of morphing required; and

(l) adjusting the rank to accommodate multiple second images of the same person.

6. Apparatus for aligning anatomical images taken from first and second different spectral bands, comprising

(a) means for generating a first anatomical image from a visual spectral band;

(b) means for generating a second anatomical image from an infrared spectral band;

(c) means for identifying coincident minutiae which occur in both anatomical images and which correspond to a characteristic feature of the anatomical images; and

(d) means for overlaying the coincident minutiae by morphing one anatomical image to the other anatomical image to compare the coincident minutiae and determine whether there is a match of the two anatomical images.

7. Apparatus as defined in claim 6, and further comprising means for calculating the degree of morphing required to achieve optimal alignment, said overlaying means stretching, warping, and shrinking said one image with respect to said other image.

8. Apparatus as defined in claim 7, wherein said image generating means comprise a camera for generating a visual image and an infrared camera for generating an infrared image.

9. Apparatus as defined in claim 8, and further comprising a database for storing a plurality of visual images of known individuals whereby an image from an unknown individual can be compared with said stored images to identify the unknown individual.

10. A method for identifying an unknown person from a first image taken in one spectral band by comparing it with a plurality of second images of known persons taken in a different spectral band and stored in a database, comprising the steps of

(a) identifying minutiae in each image which are dependent upon the spectral band thereof;

(b) identifying coincident minutiae which occur in the images and which correspond to a characteristic feature of the images;

(c) overlaying the coincident minutiae by morphing said first image onto each of said second images;

(d) selecting one of said second images requiring the least morphing as the most likely matching image from said database;

(e) comparing the degree of morphing with a threshold to determine whether the match is sufficient to identify the unknown person as one from the database;

(f) selecting those second images which have a degree of morphing within the threshold as possible matches to the first image;

(g) establishing exclusion zones for the second images;

(h) determining whether any features of the first image extend into the exclusion zones of the second images when they are aligned and overlaid;

(i) discarding from further consideration any second images whose exclusion zones are violated by the first image;

(j) considering any remaining second images as possible candidates for matching the first image;

(k) rank ordering the possible candidates by the degree of morphing required;

(l) adjusting the rank to accommodate multiple second images of the same person;

(m) establishing a set of anatomical rules which govern the relative location of minutiae from the two spectral bands;

(n) testing each of the possible candidates against each of the rules applied to the first image;

(o) eliminating from further consideration any candidate which violates any rule;

(p) assessing the number of remaining candidates;

(q) adding additional anatomical rules if a further reduction is required;

(r) testing each remaining candidate against the added rules;

(s) rank ordering the possible candidates by the degree of morphing required; and

(t) adjusting the rank to accommodate multiple second images of the same person.

11. Apparatus for identifying an unknown person from a first anatomical image taken in one of a visual and infrared spectral band by comparing it with a plurality of second anatomical images of known persons taken in the other of a visual and infrared spectral band and stored in a database, comprising:

(a) a digital storage device for the capture of first anatomical images;

(b) a first processor which standardizes the first anatomical images;

(c) a database of standardized second anatomical images considered as potential candidate matches to the first anatomical image;

(d) a second processor which extracts and analyzes spectrum-dependent minutiae of each anatomical image;

(e) a selector which aligns the first and second anatomical images;

(f) a comparator which matches coincident minutiae from the first and second anatomical images and excludes some second anatomical images as possible matches based upon the morphing required to overlay the coincident minutiae;

(g) anatomical rules encoded into image comparison steps;

(h) an evaluator which tests the first anatomical image for exclusion zones and anatomical rules relative to the second anatomical image and excludes as possible candidate matches those second anatomical images which fail the tests;

(i) a rank ordering processor which calculates the probability each candidate is a match, based upon the degree of processing and morphing required; and

(j) output means for outputting said anatomical images, a rank ordering list, and confidence measures.