IMAGE PROCESSING METHOD, IMAGE PROCESSING DEVICE, AND STORAGE MEDIUM

• US 20190320126A1
• Filed: 12/15/2017
• Published: 10/17/2019
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: AKASHI, Ryuichi
• Granted Patent: US 10,931,895 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 5/208 for use with infrared or ultraviolet radiation, e.g. for separating visible light from infrared and/or ultraviolet radiation
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules

Abstract

An image processing device (10) is provided with: an estimation unit (11) for estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and a generation unit (12) for generating, using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data configured only from information representing the visible light and near-infrared light image data configured only from information representing the near-infrared light.

Claims

1. An image processing method comprising:

estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and

generating, by using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data composed only of information representing visible light and near-infrared light image data composed only of information representing near-infrared light, wherein

each integral value of the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light relating to each of the one or more color channels of the color image sensor satisfies a predetermined condition.

2. The image processing method according to claim 1, further comprising:

generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light; and

generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light.

3. The image processing method according to claim 1, further comprising:

generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light; and

generating the near-infrared light image data composed only of information representing near-infrared light, by subtracting the generated visible light image data from the image data in which visible light and near-infrared light are captured.

4. The image processing method according to claim 1, further comprising:

generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light; and

generating the visible light image data composed only of information representing visible light, by subtracting the generated near-infrared light image data from the image data in which visible light and near-infrared light are captured.

5. The image processing method according to claim 1, wherein

spectral distribution characteristic of incident light is modeled by a weighted sum of basis vectors r_i(λ

) (i=1 to M, where M is an integer of 1 or greater), is modeled by a weighted sum of basis vectors r_i(λ

) (i=1 to M+1), is modeled by the weighted sum of the basis vectors r_i(λ

) (i=1 to M) and a mean vector r₀(λ

) of the spectral distribution characteristic, or is modeled by the weighted sum of the basis vectors r_i(λ

) (i=1 to M+1) and the mean vector r₀(λ

).

6. The image processing method according to claim 1, wherein

the one or more color channels included in the image data are a red channel, a green channel, and a blue channel.

7. An image processing device comprising:

at least one memory storing a program; and

at least one processor configured to execute the program to perform;

estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and

generating, by using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data composed only of information representing visible light and near-infrared light image data composed only of information representing near-infrared light, wherein

each integral value of the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light relating to each of the one or more color channels of the color image sensor satisfies a predetermined condition.

8. The image processing device according to claim 7, wherein

the at least one processor is configured to perform:

generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light, and

generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light.

9. A non-transitory storage medium recording an image processing program for causing a computer to execute:

estimation processing of estimating, by using image data and a spectral sensitivity characteristic of a color image sensor in a wavelength range from visible light to near-infrared light, a spectral distribution characteristic of an incident light incident on the color image sensor, based on a model expressing the spectral distribution characteristic of the incident light, the image data including one or more color channels and acquired by capturing, using the color image sensor, the incident light including visible light and near-infrared light; and

generation processing of generating, by using the estimated spectral distribution characteristic of the incident light and the spectral sensitivity characteristic of the color image sensor, visible light image data composed only of information representing visible light and near-infrared light image data composed only of information representing near-infrared light, wherein

each integral value of the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light relating to each of the one or more color channels of the color image sensor satisfies a predetermined condition.

10. The storage medium according to claim 9, wherein

the image processing program causes a computer to execute:

first generation processing of generating the visible light image data composed only of information representing visible light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of visible light; and

second generation processing of generating the near-infrared light image data composed only of information representing near-infrared light, by using a product of the estimated spectral distribution characteristic of the incident light by the spectral sensitivity characteristic of the color image sensor in a wavelength range of near-infrared light.