Method and system for converting at least one first-spectrum image into a second-spectrum image

• US 8,478,028 B2
• Filed: 08/03/2007
• Issued: 07/02/2013
• Est. Priority Date: 08/04/2006
• Status: Expired due to Fees

• Current Assignee: Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Original Assignee: Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Inventors: Hogervorst, Maarten Andreas, Toet, Alexander, Kooi, Frank Leonard
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US12/376,253
• Publication Number:
  US 20100002956A1
• Time in Patent Office: 2,160 Days
• Field of Search: 348/29, 348/32, 348/33, 348/43, 348/152, 348/153, 348/161, 348/164, 348/169, 348/E5.09, 358/1.1, 358/3.28, 358/540, 358/450, 382/100, 382/103, 382/115, 382/118, 382/162, 382/164, 382/165, 382/173, 382/181, 382/284, 600/476, 600/473
• US Class Current: 382/162
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77

Abstract

Method for converting at least one image of a first spectrum into an image of a second spectrum, comprising: —recording at least one first-spectrum reference image (NRI) of at least one reference scene (RS) with a first-spectrum recording apparatus (1), the first-spectrum reference image (NRI) comprising first image portions with corresponding first-spectrum sensor reference data (b1, b2); —providing corresponding second-spectrum reference information (RGB); —providing at least one set of reference data (T1, T2) from at least part of the first-spectrum sensor reference data (b1, b2) relating to the reference scene (RS), and at least a corresponding part of the second-spectrum information (RGB) relating to that reference scene (RS); —providing at least one target first-spectrum image (NTI) or target second-spectrum image; and —converting each target first-spectrum image (NTI) or target second-spectrum image into a respective second-spectrum image (CI) or first-spectrum image, respectively, using the set of reference data (T1, T2).

Claims

1. A method for converting at least one image of a first spectrum into an image of a second spectrum, comprising:

recording a first-spectrum reference image of a reference scene with a first-spectrum recording apparatus, the first-spectrum recording apparatus being a night-vision apparatus, the first-spectrum reference image comprising first image portions with corresponding first-spectrum sensor reference data;

providing corresponding second-spectrum reference information;

providing a set of reference data from at least part of the first-spectrum sensor reference data relating to the reference scene, and at least a corresponding part of the second-spectrum information relating to the reference scene;

providing a target first-spectrum image; and

converting the target first-spectrum image into a second-spectrum image using the set of reference data,

wherein the first-spectrum sensor reference data comprises multi-band data, relating to at least two different parts of a light spectrum, and the second-spectrum reference information is true-color color information,

wherein the providing the set of reference data comprises;

selecting, for a same reference scene, a plurality of first, first-spectrum, image portions and corresponding second, second-spectrum, image portions, such that reference scene image coordinates of a first image portion substantially correspond to reference scene image coordinates of a corresponding second image portion of the same reference scene, and

creating an index containing the first-spectrum sensor reference data and the second-spectrum information for the same reference scene such that each entry in the index refers to the first-spectrum sensor reference data and the corresponding second-spectrum information associated with a selected first image portion and a corresponding second image portion of the same reference scene; and

wherein the converting the target first-spectrum image into the second-spectrum image comprises;

finding, for individual portions of the target first-spectrum image, a closest matching entry in the index by comparing first-spectrum sensor data associated with the individual portions of the target first-spectrum image and the first-spectrum sensor reference data contained within entries of the index containing the first-spectrum reference data, and

replacing, for the individual portions of the target first-spectrum image, each target first-spectrum image portion with the second, second-spectrum, image portion, the second image portion being associated with the closest matching entry in the index for the individual portion of the target first-spectrum image.

2. The method according to claim 1, comprising:

generating an indexed data set involving the indexing of a plurality of achievable first-spectrum sensor reference data parts which can be generated by the first-spectrum recording apparatus;

assigning the second-spectrum reference information to the indexed data set, utilizing the at least one first-spectrum reference image, such that the indexed first-spectrum sensor reference data parts and corresponding second-spectrum reference information parts are provided with the same index pointer.

3. The method according to claim 1, wherein the second-spectrum reference information is already known or is being provided by:

providing at least one second-spectrum reference image of each reference scene, the at least one second-spectrum reference image comprising second, second-spectrum, image portions with corresponding second-spectrum information.

4. The method according to claim 1 further comprising

converting a target second-spectrum image into a first-spectrum image:

finding, for individual portions of the target second-spectrum image, a closest matching entry in the index by comparing second-spectrum sensor data associated with the individual portions of the target second-spectrum image and the second-spectrum sensor reference data contained within entries of the index containing, the second-spectrum reference data, and

replacing, for the individual portions of the target second-spectrum image, each target second-spectrum image portion with the first, first-spectrum, image portion, the first image portion being associated with the closest matching entry in the index for the individual portion of the target second-spectrum image.

5. The method according to claim 2, comprising during converting target first-spectrum images into respective second-spectrum images:

finding a closest match between first-spectrum sensor data associated with each target image portion of a target image and the first-spectrum sensor reference data of the first image portions;

replacing each target image portion with the respective index entry pointer to provide an index image; and

converting the index image into the second-spectrum image by replacing each index entry pointer with the respective second-spectrum information of that index entry or, vice-versa, in case of conversion of target second-spectrum images into respective first-spectrum images;

finding a closest match between second-spectrum information associated with each target image portion of a target second-spectrum image and the second-spectrum reference data of the second image portions;

replacing each target image portion with the respective index entry pointer to provide an index image; and

converting the index image into the first-spectrum image by replacing each index entry pointer with the respective first-spectrum information of that index entry.

6. The method according to claim 1, wherein the first-spectrum sensor reference data and the target image are generated by substantially the same type of first-spectrum recording apparatus.

14. A reference data set, generated in a method according to claim 1, the reference data set being provided from at least part of the first-spectrum sensor reference data relating to the reference scene, and at least a corresponding part of the second-spectrum reference information relating to that reference scene.

15. A non-transitory computer-readable medium comprising a reference data set according to claim 14.

16. The method according to claim 2, comprising generating the indexed data set involving the indexing of a plurality of achievable first-spectrum sensor reference data parts which can be generated by the first-spectrum recording apparatus, by providing a list of all achievable first-spectrum sensor values or an array of a plurality of achievable first-spectrum sensor values.

17. The method according to claim 3, wherein the second-spectrum reference information is already known via visual inspection of a reference scene.

18. The method according to claim 2, including indexing the first-spectrum sensor reference data and the second-spectrum information in at least one look-up table.

7. A system for converting a first-spectrum image into a second-spectrum image, the system comprising:

a first-spectrum recording apparatus configured to generate first-spectrum sensor reference data relating to a reference scene, the first-spectrum recording apparatus being a night-vision apparatus, the first-spectrum sensor reference data comprising multi-band data relating to at least two different parts of the spectrum;

a second-spectrum information provider configured to provide second-spectrum reference information relating to the reference scene, the second-spectrum reference information being true-color color information;

a target image generator providing a first-spectrum target image of a target scene, wherein the target image generator is a night-vision apparatus;

a converter configured to convert the first-spectrum target image into a second-spectrum image using the first-spectrum sensor reference data in combination with a corresponding part of the second-spectrum reference information; and

a data processor, configured to process the first-spectrum sensor reference data and the second-spectrum reference information to provide at least one set of reference data by;

selecting, for a same reference scene a plurality of first, first-spectrum, image portions from a first-spectrum reference image and corresponding second, second-spectrum, image portions, such that reference scene image coordinates of a first image portion substantially correspond to reference scene image coordinates of a corresponding second image portion of the same reference scene; and

creating an index containing the first-spectrum sensor reference data and the second-spectrum reference information for the same reference scene such that each entry in the index refers to the first-spectrum sensor reference data and the corresponding second-spectrum reference information associated with a selected first image portion and a corresponding second image portion of the same reference scene; and

wherein the converter is configured to convert the target first-spectrum image into the second-spectrum image by;

finding, for individual portions of the target first-spectrum image, a closest matching entry in the index by comparing first-spectrum sensor data associated with the individual portions of the target first-spectrum image and the first-spectrum sensor reference data contained within entries of the index containing the first-spectrum reference data, and

replacing, for the individual portions of the target first-spectrum image, each target first-spectrum image portion with the second, second-spectrum, image portion, the second image portion being associated with the closest matching entry in the index for the individual portion of the target first-spectrum image.

8. The system according to claim 7, comprising a memory wherein at least one set of reference data, comprising at least part of the mentioned first-spectrum sensor reference data, or potentially achievable first-spectrum sensor reference data, in combination with a corresponding part of the mentioned second-spectrum reference information, is stored.

9. The system according to claim 7, the system being configured to utilize various sets of reference data, relating to different reference scenes, wherein the converter is controllable to select one of the sets of reference data for converting target images into second-spectrum images.

10. The system according to claim 7, wherein the converter is configured to replace each target second-spectrum image portion with a first, first-spectrum, image portion, which first image portion is associated with the second image portion that most closely matched that target second-spectrum image portion, the matching being based on the second-spectrum reference data, to generate the first-spectrum image.

11. The system according claim 7, wherein the converter is configured to replace each target image portion with the respective index entry pointer to provide an index image, and to replace each index entry pointer with the respective second-spectrum or first-spectrum information of that index entry.

12. The system according to claim 7, wherein the indexed data set comprises at least one table relating to first image portions and respective first-spectrum sensor data and relating to second image portions of a corresponding second-spectrum image and respective second-spectrum reference information.

13. The system according to claim 7, comprising a second-spectrum image recorder to record the second-spectrum reference information.

19. The system according to claim 13, comprising the second-spectrum image recorder to record at least one second-spectrum reference image.