True color infrared photography and video

• US 7,079,682 B2
• Filed: 05/18/2004
• Issued: 07/18/2006
• Est. Priority Date: 11/07/2000
• Status: Expired due to Fees

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Joseph W. Niesen
• Inventors: Niesen, Joseph W.
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Sherali, Ishrat
• Application Number:
  US10/848,621
• Publication Number:
  US 20050013482A1
• Time in Patent Office: 791 Days
• Field of Search: 382/162, 382164-167, 382170-171, 382/173, 382/181, 382/100, 382/103, 382/115, 382/118, 382/305, 382282-284, 382190-191, 382/156, 250/330, 250/316.1, 348 29- 34, 348/43, 348/217, 348152-153, 348/161, 348/164, 342/179, 359/618
• US Class Current: 382/162
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

A method beginning at (10) is provided for creating high-fidelity visible coloring from infrared images of a scene under surveillance. The infrared images captured at (12) are analyzed at (14) to determine if an object, such as a face, is identifiable within the image. If an object is identifiable at (16) the object features are compared to a plurality of stored object features at (20). If there is a match at (22), the color characteristics of the object are obtained at (24) and the object is colored at (26) based on the stored database feature information. If there is no match at (22) or identifiable object at (16) and object color cannot be identified at (23), the image is analyzed at (28) to determine if a pattern, such as clothing, is identifiable within the image. If a pattern is identifiable at (30), the color characteristics of the pattern are obtained at (34) and the pattern is colored at (36) according to infrared reflectance characterization in conjunction with the stored pattern information. If no pattern is identifiable at (30), the non-pattern and non-feature containing portions of the image are colored at (38) according to infrared reflectance characterization.

Claims

1. A method of producing a true color image comprising the steps of:

obtaining an infrared image of a scene over different spectral bands;

determining if an object within said image is identifiable;

segmenting said image into an object containing portion and a non-object containing portion if said object is identifiable;

comparing said object containing portion to a plurality of stored object feature characteristics;

coloring said object containing portion of said image according to information from said stored object feature characteristics if said object matches a set of said stored object feature characteristics;

determining if a pattern within said image is identifiable;

segmenting said image into a pattern containing portion and a non-pattern containing portion if said pattern is identifiable;

coloring said pattern containing portion of said image using multispectral imaging in conjunction with stored pattern information; and

coloring said non-object and non-pattern containing portions of said image by applying multispectral imaging to said image.

2. The method of claim 1 further comprising determining whether true color is identifiable in said object containing portion of said image if said object does not match any set of said stored object feature characteristics.

3. The method of claim 2 further comprising assigning true colors to said object if said true color is identifiable in said object containing portion of said image.

4. The method of claim 1 wherein said step of coloring said object containing portion of said image further comprises obtaining said information regarding said object features characteristics from a database.

5. The method of claim 1 wherein said step of coloring said pattern containing portion of said image further comprises obtaining said pattern information from a database.

6. A method of producing a true color image comprising the steps of:

obtaining an infrared image of a scene over different spectral bands;

determining if an object within said image is identifiable;

segmenting said image into an object containing portion and a non-object containing portion when said object is identifiable;

comparing said object containing portion to a plurality of stored object characteristics;

coloring said object containing portion of said image according to color information for a given stored object characteristic when said object matches the given stored object characteristic;

determining if a pattern within said non-object containing portion is identifiable;

segmenting said non-object containing portion of the image into a pattern containing portion and a non-pattern containing portion when said pattern is identifiable;

coloring said pattern containing portion of said image using multispectral imaging in conjunction with stored pattern information; and

coloring said non-pattern containing portions of said image using multispectral imaging.

7. The method of claim 6 further comprising determining whether true color is identifiable in said object containing portion of said image when said object does not match any of said stored object characteristics.

8. The method of claim 7 further comprising assigning true colors to said object when said true color is identifiable in said object containing portion of said image.

9. The method of claim 6 wherein coloring said pattern containing portion of said image further comprises:

coloring said pattern containing portion of said image using multispectral imaging;

comparing said pattern to a plurality of stored patterns, each pattern being associated with pattern color information; and

coloring said pattern containing portion of said image according to pattern color information for a given stored pattern when said pattern matches the given stored pattern.

10. A method of producing a true color image comprising the steps of:

obtaining an infrared image of a scene over different spectral bands;

determining if a pattern within said image is identifiable;

segmenting said image into a pattern containing portion and a non-pattern containing portion when said pattern is identifiable;

coloring said pattern containing portion of said image using multispectral imaging;

comparing said pattern to a plurality of stored patterns, each pattern being associated with pattern color information;

coloring said pattern containing portion of said image according to pattern color information for a given stored pattern when said pattern matches the given stored pattern; and

coloring said non-pattern containing portions of said image using multispectral imaging.