Method and apparatus for aligning and comparing images of the face and body from different imagers

• US 6,751,340 B2
• Filed: 07/15/2002
• Issued: 06/15/2004
• Est. Priority Date: 10/22/1998
• Status: Expired due to Fees

• Current Assignee: Francine J. Prokoski
• Original Assignee: Francine J. Prokoski
• Inventors: Prokoski, Francine J.
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): DESIRE, GREGORY M
• Application Number:
  US10/195,629
• Publication Number:
  US 20030108223A1
• Time in Patent Office: 701 Days
• Field of Search: 382/115-118, 382/124-126, 382/215-229, 382/294-300, 382/276-278, 463/29, 235/380, 348/14.09, 348/14-3-, 348/211.12, 348/211.1
• US Class Current: 382/118
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/415 the glands, e.g. tonsils, adenoids or thymus
• CPC Class Codes: A61B 5/418 lymph vessels, ducts or nodes
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A method and apparatus for comparing an infrared image of a person to a database of visual images of persons and calculating the probability that each is a match to the infrared image is characterized by extracting minutiae from the infrared image and extracting visible minutiae from the visible images. Coincident minutiae which occur in both spectra are used to scale and register the infrared and the visible images. Other minutiae are spectrum-dependent, but must obey rules relative to minutiae of the other spectrum, due to the anatomical structure of the human face and body. The primary application is for identification of persons seen in infrared surveillance imagery, using a reference database of visual images. Other applications include compression of talking head video and animation of synthetic faces. The method and apparatus can also be applied to areas of the body other than the face, to compare images from different spectra including images from medical sensors.

Claims

1. A method for encoding facial expression and movement, comprising the steps of

(a) generating simultaneous and corresponding visual and infrared images of a face over an interval of time;

(b) identifying and extracting spectrum-dependent minutiae from each image at selected intervals within said time period;

(c) tracking the movement of said minutiae during said time period;

(d) aligning said visual and infrared images via said coincident minutiae for each image; and

(e) calculating the locations of coincident minutiae in the visual image corresponding to the infrared minutiae.

2. A method as defined in claim 1, and further comprising the step of storing said images in a database in accordance with the locations of coincident minutiae.

3. A method for compression of talking head video, comprising the steps of

(a) taking simultaneous and corresponding visual and infrared video image of a face;

(b) extracting infrared minutiae from each infrared frame;

(c) determining the face axes of the infrared face;

(d) transmitting a baseline video frame;

(e) tracking the movement direction and extent of each infrared minutia frame-to-frame;

(f) tracking the movement direction and extent of the face axes frame-to-frame;

(g) establishing a range of face movement to be permitted without sending a new baseline video image;

(h) transmitting the movement vectors for all facial infrared minutiae when the face movement is within the permitted range;

(i) transmitting a new visual baseline image when the face movement is outside the permitted range;

(j) displaying the baseline video image at a receiving end; and

(k) distorting the displayed video image by superimposing morphing in accordance with the transmitted infrared minutiae vectors frame-to-frame.

4. A method as defined in claim 3, further comprising the steps of

(a) morphing between a new baseline image and the last presented image in order to smooth the transition to a new baseline;

(b) determining the amount of morphing needed to accomplish a smooth transition; and

(c) sending a signal to the transmission end to change the permitted range of face movement.

5. Apparatus for talking head video compression, comprising

(a) a dualband infrared/visual camera;

(b) an infrared minutiae extraction and face axes subsystem connected with said camera;

(c) a face movement tracker connected with said subsystem;

(d) an infrared spectral minutiae tracker connected with said face movement tracker;

(e) a transmitter of baseline visual image;

(f) a transmitter of infrared spectral minutiae movements within baseline connected with said image tracker;

(g) a receiver of minutiae movement changes connected with said minutiae movement tracker;

(h) a display of baseline and animated visual image connected with said receiver;

(i) a visual face animator using infrared minutiae vectors;

(j) a receiver of new head baseline position when head movement exceeds permitted range;

(k) a morpher of new baseline head position and last constructed head position; and

(l) a tuner which calculates the amount of morphing needed in response to a new baseline.