METHOD AND SYSTEM FOR CONVERTING AT LEAST ONE FIRST-SPECTRUM IMAGE INTO A SECOND-SPECTRUM IMAGE

• US 20100002956A1
• Filed: 08/03/2007
• Published: 01/07/2010
• Est. Priority Date: 08/04/2006
• Status: Active Grant

• Current Assignee: Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Original Assignee: Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Inventors: Kooi, Frank Leonard, Hogervorst, Maarten Andreas, Toet, Alexander
• Granted Patent: US 8,478,028 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77

Abstract

Method for converting at least one image of a first spectrum into an image of a second spectrum, comprising: —recording at least one first-spectrum reference image (NRI) of at least one reference scene (RS) with a first-spectrum recording apparatus (1), the first-spectrum reference image (NRI) comprising first image portions with corresponding first-spectrum sensor reference data (b1, b2); —providing corresponding second-spectrum reference information (RGB); —providing at least one set of reference data (T1, T2) from at least part of the first-spectrum sensor reference data (b1, b2) relating to the reference scene (RS), and at least a corresponding part of the second-spectrum information (RGB) relating to that reference scene (RS); —providing at least one target first-spectrum image (NTI) or target second-spectrum image; and —converting each target first-spectrum image (NTI) or target second-spectrum image into a respective second-spectrum image (CI) or first-spectrum image, respectively, using the set of reference data (T1, T2).

Claims

1. Method for converting at least one image of a first spectrum into an image of a second spectrum, comprising:

recording at least one first-spectrum reference image of at least one reference scene with a first-spectrum recording apparatus, the first-spectrum reference image comprising first image portions with corresponding first-spectrum sensor reference data;

providing corresponding second-spectrum reference information;

providing at least one set of reference data from at least part of the first-spectrum sensor reference data relating to the reference scene, and at least a corresponding part of the second-spectrum information relating to that reference scene;

providing at least one target first-spectrum image or target second-spectrum image; and

converting each target first-spectrum image or target second-spectrum image into a respective second-spectrum image or first-spectrum image, respectively, using the set of reference data.

2. Method according to claim 1, wherein the recording apparatus is a night-vision apparatus.

3. Method according to claim 1, comprising:

generating an indexed data set involving the indexing of a plurality of achievable first-spectrum sensor reference data parts which can be generated by the first-spectrum recording apparatus, for example by providing a list of all achievable first-spectrum sensor values or an array of a plurality of achievable first-spectrum sensor values;

assigning the second-spectrum reference information to the indexed data set, utilizing the at least one first-spectrum reference image, such that the indexed first-spectrum sensor reference data parts and corresponding second-spectrum reference information parts are provided with the same index pointer.

4. Method according to claim 1, wherein the second-spectrum reference information is already known, for example via visual inspection of a reference scene, or is being provided by:

providing at least one second-spectrum reference image of each reference scene, the at least one second-spectrum reference image comprising second, second-spectrum, image portions with corresponding second-spectrum information.

5. Method according to claim 4, wherein the set of reference data is being generated by:

selecting a plurality of first, first-spectrum, image portions and corresponding second, second-spectrum, image portions, such that reference scene image coordinates of a first image portion substantially correspond to reference scene image coordinates of the corresponding second image portion;

indexing the first-spectrum sensor reference data and the second-spectrum information, for example in at least one lookup table, such that each entry in a resulting index refers to the first-spectrum sensor reference data and the corresponding second-spectrum information associated with a selected first image portion and a corresponding second image portion.

6. Method according to claim 5, comprising, in case of conversion of target first-spectrum images into respective second-spectrum images:

finding a closest match between first-spectrum sensor data associated with each target first-spectrum image portion of a target first-spectrum image and the first-spectrum sensor reference data of the first image portions;

replacing each first-spectrum target image portion with the second, second-spectrum, image portion, which second image portion is associated with the first image portion that most closely matched that target image portion, the matching being based on the first-spectrum sensor data, to generate the second-spectrum image;

or, vice-versa, in case of conversion of target second-spectrum images into respective first-spectrum images;

finding a closest match between second spectrum information associated with each target second-spectrum image portion of a target second-spectrum image and the second-spectrum information of the second image portions;

replacing each second-spectrum target image portion with the first, first-spectrum, image portion, which first image portion is associated with the first image portion that most closely matched that target image portion, the matching being based on the second-spectrum information, to generate the first-spectrum image.

7. Method according to claim 5, comprising in case of conversion of target first-spectrum images into respective second-spectrum images:

finding a closest match between first-spectrum sensor data associated with each target image portion of a target image and the first-spectrum sensor reference data of the first image portions;

replacing each target image portion with the respective index entry pointer to provide an index image; and

converting the index image into the second-spectrum image by replacing each index entry pointer with the respective second-spectrum information of that index entry or, vice-versa, in case of conversion of target second-spectrum images into respective first-spectrum images;

finding a closest match between second-spectrum information associated with each target image portion of a target second-spectrum image and the second-spectrum reference data of the second image portions;

replacing each target image portion with the respective index entry pointer to provide an index image; and

converting the index image into the first-spectrum image by replacing each index entry pointer with the respective first-spectrum information of that index entry.

8. Method according to claim 1, wherein the first-spectrum sensor reference data comprises multi-band data, relating to at least two different parts of the spectrum.

9. Method according to claim 1, wherein the second-spectrum reference information is true-colour colour information.

10. Method according to claim 1, wherein the first-spectrum sensor reference data and the target image are generated by substantially the same type of first-spectrum recording apparatus.

11. A system for converting at least one first-spectrum image into a second-spectrum image, the system being configured to carry out a method according to any of the preceding claims, the system comprising:

at least a first-spectrum recording apparatus configured to generate first-spectrum sensor reference data relating to at least one reference scene;

at least a second-spectrum information provider configured to provide second-spectrum information relating to the same reference scene; and

at least a target image generator to provide at least a first-spectrum target image or at least a second spectrum target image of a target scene;

wherein the system is provided with a converter which is configured to convert each target first-spectrum image into a respective second-spectrum image, or, vice-versa, each target second-spectrum image into a respective first-spectrum image, using at least part of the first-spectrum sensor reference data in combination with a corresponding part of the second-spectrum information.

12. A system according to claim 11, comprising a memory wherein at least one set of reference data, comprising at least part of the mentioned first-spectrum sensor reference data, or potentially achievable first-spectrum sensor reference data, in combination with a corresponding part of the mentioned second-spectrum information, is stored.

13. A system according to claim 11, the system being configured to utilize various sets of reference data, relating to different reference scenes, wherein the converter is controllable to select one of the sets of reference data for converting target images into second-spectrum images.

14. A system according to claim 11, wherein the converter is provided with at least one indexed data set comprising a plurality of index entries, wherein each entry at least comprises:

the first-spectrum sensor reference data associated with a first image portion selected from a first-spectrum reference image, or potentially achievable first-spectrum sensor reference data;

the corresponding second-spectrum information.

15. A system according to claim 14, wherein the converter is configured to find a closest match between first-spectrum or second spectrum data associated with each target image portion of a target image and the first-spectrum data or second-spectrum data, respectively, relating to the reference scene.

16. A system according to claim 15, wherein the converter is configured to replace each target first-spectrum image portion with a second, second-spectrum, image portion, which second image portion is associated with the first image portion that most closely matched that target image portion, the matching being based on the first-spectrum sensor data, to generate the second-spectrum image.

17. A system according to claim 15, wherein the converter is configured to replace each target second-spectrum image portion with a first, first-spectrum, image portion, which first image portion is associated with the second image portion that most closely matched that target second-spectrum image portion, the matching being based on the second-spectrum reference data, to generate the first-spectrum image.

18. A system according to claim 15, wherein the converter is configured to replace each target image portion with the respective index entry pointer to provide an index image, and to replace each index entry pointer with the respective second-spectrum or first-spectrum information of that index entry.

19. A system according to claim 14, wherein the indexed data set comprises at least one table relating to first image portions and respective first-spectrum sensor data and relating to second image portions of a corresponding second-spectrum image and respective second-spectrum information.

20. A system according to claim 11, comprising a data processor, configured to process the first-spectrum sensor reference data and the second-spectrum information to provide at least one set of reference data, wherein the data processor is configured to:

select a plurality of first, first-spectrum, image portions from a first-spectrum reference image and corresponding second, second-spectrum, image portions from a corresponding second-spectrum image, such that reference scene image coordinates of a first image portion substantially correspond to reference scene image coordinates of the corresponding second image portion; and

index the first-spectrum sensor reference data and the second-spectrum information, for example in at least one lookup table, such that each entry in a resulting index refers to the first-spectrum sensor reference data and the corresponding second-spectrum information associated with a selected first image portion and a corresponding second image portion.

21. A system according to claim 11, comprising a second-spectrum image recorder to record the second-spectrum information, for example to record at least one second-spectrum reference image.

22. A system according to claim 11, wherein the target image generator is a night-vision apparatus.

23. A computer program product, comprising a computer-readable medium including computer code, configured to carry out conversion of a first-spectrum target image into a second-spectrum image, or vice-versa, according to a method according to claim 1 when executed by a computer device, and/or configured to provide a converter of a system according to claim 11 when executed by a computer device.

24. A reference data set, generated in a method according to claim 1, the reference data set being provided from at least part of the first-spectrum sensor reference data relating to the reference scene, and at least a corresponding part of the second-spectrum information relating to that reference scene.

25. A computer-readable medium comprising a reference data set according to claim 24.