Image processing apparatus, image processing method, image pickup apparatus, computer program and recording medium

• US 7,589,771 B2
• Filed: 12/16/2005
• Issued: 09/15/2009
• Est. Priority Date: 12/22/2004
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kasai, Masanori, Hosaka, Hajime
• Primary Examiner(s): Ho; Tuan V
• Assistant Examiner(s): Cowan; Euel K
• Application Number:
  US11/304,651
• Publication Number:
  US 20060132642A1
• Time in Patent Office: 1,369 Days
• Field of Search: 348/208.1, 348/215.1, 348/252, 348/222.1, 382/266, 382/264, 382/269
• US Class Current: 348/252
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

Noises are reduced in an image picked up in natural light without using lighting equipment that affect the picked up image. The present invention provides an image processing apparatus that includes an image acquisition unit that acquires a visible light image and an invisible light image corresponding to the visible light image, and a noise reduction unit that reduces noises of the visible light image by using the invisible light image.

Claims

1. An image processing apparatus comprising:

image acquisition means for acquiring a visible light image and an invisible light image corresponding to the visible light image;

first filtering means for applying a low pass filter to the visible light image, a filter level of the low pass filter being selected independently in the X-direction and the Y-direction of the visible light image according to an evaluation value of edges detected from the invisible light image;

second filtering means for applying a high pass filter to the invisible light image;

synthesis means for synthetically combining the output of the low pass filter and the output of the high pass filter to create a noise-reduced visible light image; and

displaying means for displaying the noise-reduced visible light image on a display.

2. The apparatus according to claim 1, wherein the invisible light image is an infrared image.

3. The apparatus according to claim 1, wherein the invisible light image is an ultraviolet image.

4. The apparatus according to claim 1, wherein the low pass filter is an edge preserving low pass filter.

5. The apparatus according to claim 4, wherein the edge preserving low pass filter removes noises from the visible light image, while preserving the edges detected from the invisible light image.

6. The apparatus according to claim 1, wherein the visible light image is transformed into a color image and a luminance image and filters are applied respectively to the color image and the luminance image obtained as a result of transformation.

7. The apparatus according to claim 6, wherein a bilateral filter is applied to the color image and a cross bilateral filter is applied to the luminance image.

8. The apparatus according to claim 1, further comprising:

aberration correction means for correcting the aberration of the visible light image and that of the invisible light image.

9. The apparatus according to claim 8, wherein the aberration correction means correct the aberrations, using the error between the luminance image obtained by transforming the visible light image and the invisible light image.

10. An image pickup apparatus comprising:

visible light image pickup means for picking up a visible light image according to a first spectral characteristic of having sensitivity mainly relative to visible light;

invisible light image pickup means for picking up an invisible light image according to a second spectral characteristic of having sensitivity mainly to invisible light;

luminance image generating means for generating a luminance image from the visible light image;

error computing means for computing error values by comparing transformation images of the luminance image with the invisible light image;

aberration correction means for correcting the aberration of the visible light image and that of the invisible light image based on the computed error values;

noise reduction means for reducing noises of the visible light image by using the invisible light image; and

displaying means for displaying the noise-reduced visible light image on a display.

11. An image processing method comprising:

acquiring a visible light image and an invisible light image corresponding to the visible light image;

applying a low pass filter to the visible light image, a filter level of the low pass filter being selected independently in the X-direction and the Y-direction of the visible light image according to an evaluation value of edges detected from the invisible light image;

applying a high pass filter to the invisible light image;

synthetically combining the output of the low pass filter and the output of the high pass filter to create a noise-reduced visible light image; and

displaying the noise-reduced visible light image on a display.

12. A computer-readable recording medium storing a computer program for causing a computer to execute a method comprising:

acquiring a visible light image and an invisible light image corresponding to the visible light image applying a low pass filter to the visible light image,

applying a low pass filter to the visible light image, a filter level of the low pass filter being selected independently in the X-direction and the Y-direction of the visible light image according to an evaluation value of edges detected from the invisible light image;

applying a high pass filter to the invisible light image;

synthetically combining the output of the low pass filter and the output of the high pass filter to create a noise-reduced visible light image; and

displaying the noise-reduced visible light image on a display.

13. An image pickup apparatus comprising:

a visible light image pickup unit that picks up a visible light image according to the first spectral characteristic of having sensitivity mainly relative to visible light;

an invisible light image pickup unit that picks up an invisible light image according to the second spectral characteristic of having sensitivity mainly to invisible light;

a luminance image generating unit that generates a luminance image from the visible light image;

an error computing unit that computes error values by comparing transformation images of the luminance image with the invisible light image;

an aberration correction unit that corrects the aberration of the visible light image and that of the invisible light image based on the computed error values;

a noise reduction unit that reduces noises of the visible light image by using the invisible light image; and

a display unit that displays the noise-reduced visible light image.