Method and apparatus for using thermal infrared for face recognition

• US 7,406,184 B2
• Filed: 07/03/2003
• Issued: 07/29/2008
• Est. Priority Date: 07/03/2002
• Status: Active Grant

• Current Assignee: Equinox Corp. (Oracle Corporation)
• Original Assignee: Equinox Corp. (Oracle Corporation)
• Inventors: Socolinsky, Diego A., Eveland, Christopher K., Wolff, Lawrence B., Selinger, Andrea, Neuheisel, Joshua D.
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): TUCKER, WESLEY J
• Application Number:
  US10/611,859
• Publication Number:
  US 20040005086A1
• Time in Patent Office: 1,853 Days
• Field of Search: 382115-118, 382/209, 382/218, 382/181, 382/284, 382128-134
• US Class Current: 382/118
• CPC Class Codes: G01J 5/026 Control of working procedures of a pyrometer, other than calibration; Bandwidth calculation; Gain control
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A methodology and apparatus is described that incorporates the use of thermal infrared imaging into a complete end-to-end face recognition system. Since thermal infrared primarily images emissive phenomenology there are advantages over video imaging in the reflective domain (e.g., visible, near-infrared) for face recognition, particularly with respect to image invariance in the presence of varying illumination incident on the subject. On the other hand video cameras in the reflective domain provide important image detail not apparent in thermal infrared imagery. Fusion of thermal infrared imagery with reflective domain (e.g., visible, near-infrared) imagery provides even further advantages. Embodiments of such systems that incorporate thermal infrared imaging for automatic face recognition are important for access control, verification, identification, and surveillance and monitoring.

Claims

1. A method for performing face recognition, comprising:

producing a first video image input produced from a scene sensed in the reflective domain from wavelengths in the range of 0.3 microns to 2.5 microns;

producing a second video image input from said scene sensed in the thermal infrared domain;

applying non-uniformity correction (NUC) to the second video image, thereby forming a corrected thermal infrared video image;

creating a representation template for a face from a fused combination of the first video image obtained from the reflective domain and the corrected thermal infrared video image formed by applying NUC to the second video image obtained from the thermal infrared domain; and

utilizing said face representation template for comparison and matching for face recognition.

2. The method in claim 1, wherein said method is used for face recognition system applications including access control, rank ordered identification and verification.

3. The method in claim 2, wherein the face representation template is a single or combination of templates of fused reflective domain and thermal infrared domain imagery.

4. The method of claim 2, further including automatically detecting faces in a scene to extract image region(s) in the reflective domain and thermal infrared domain from which to initiate creation of a face representation template.

5. The method of claim 4, further including geometrically normalizing face image regions in the reflective domain and thermal infrared domain.

6. The method of claim 5, further including assigning a set of sub-windows for geometrically normalized face image regions in the reflective domain and in the thermal infrared domain.

7. The method of claim 6, further including forming face representation templates from each sub-window.

8. The method of claim 7, further including combining face representation templates for each sub-window.

9. An apparatus for implementing face recognition comprising:

at least one sensor configuration for simultaneously acquiring a reflective spectrum image in the wavelength range of 0.3 microns to 2.5 microns and a thermal infrared spectrum image and producing corresponding reflective spectrum and thermal infrared spectrum image signals;

a computer system comprising a computational processor, a main memory and a storage memory;

an interface card connected to said at least one sensor configuration to receive said reflective spectrum and thermal infrared spectrum signals and to send said signals to said main memory within said computer system, and

wherein said computer system includes software for processing said input reflective spectrum and thermal infrared signals to create and store a face representation template in said storage memory and process said face representation template for comparison and face recognition.

10. The apparatus of claim 9, wherein said computer includes software using input reflective spectrum/thermal infrared spectrum imagery to detect faces in a scene.

11. The apparatus of claim 10, wherein said computer includes software which is able to compare and match face representation templates of unknown individuals, with those of known individuals.

12. The apparatus of claim 11, wherein said reflective spectrum image and thermal infrared spectrum image are spatially co-registered.

13. The apparatus of claim 12, wherein said reflective spectrum domain is the visible spectrum and said the thermal infrared spectrum is the longwave infrared (LWIR) spectrum.