Image processing apparatus, image device, image processing method, and program for reducing noise or false colors in an image
First Claim
1. An image processing apparatus comprising:
- an image sensor that has an RGBW array including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light; and
an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor,wherein the image sensor has a periodic array of a unit composition formed of each RGBW pixel,wherein the image processor converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array,wherein the image processor has an array converter which generates the RGB array image formed of the Bayer array, andwherein the array converter has a W pixel interpolator which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image;
an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
a correlation processor which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which performs array conversion for a composition pixel of the sensor image and generates an RGB image of the Bayer array by calculating a pixel value based on an estimation in which the W pixel and the R, G and B pixels have a positive correlation.
1 Assignment
0 Petitions
Accused Products
Abstract
There are provided an apparatus and a method which generate an RGB image having less color noise and fewer false colors by inputting an RGBW image. The apparatus has an image sensor having an RGBW array, and an image processing unit which performs image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor. The image sensor has a periodic array of a unit composition formed of each RGBW pixel, and has an array in which composition ratios of each RGB pixel within the unit composition are adapted to be the same as each other. The image processing unit converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs at least either array conversion processing for generating an RGB array image or signal processing for generating each RGB image signal in which all RGB pixel values are set for each pixel position of the sensor image.
41 Citations
11 Claims
-
1. An image processing apparatus comprising:
-
an image sensor that has an RGBW array including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light; and an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor, wherein the image sensor has a periodic array of a unit composition formed of each RGBW pixel, wherein the image processor converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array, wherein the image processor has an array converter which generates the RGB array image formed of the Bayer array, and wherein the array converter has a W pixel interpolator which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image;
an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
a correlation processor which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which performs array conversion for a composition pixel of the sensor image and generates an RGB image of the Bayer array by calculating a pixel value based on an estimation in which the W pixel and the R, G and B pixels have a positive correlation. - View Dependent Claims (2)
-
-
3. An image processing apparatus comprising:
-
an image sensor that has an RGBW array including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light; and an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor, wherein the image sensor has a periodic array of a unit composition formed of each RGBW pixel, wherein the image processor converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array, wherein the image sensor is configured to output an image having an added pixel value of the same color pixel values which are adjacent to each other in an oblique direction, as the sensor image, wherein the image processor has an array converter which converts a pixel array of the sensor image configured to have the added pixel value and formed of the RGBW pixel signal, and which generates an RGB array image formed of the Bayer array, and wherein the array converter has an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
a correlation processor which inputs the whole W pixel image signal output as the sensor image and the RGBW corresponding low frequency signal, and which performs array conversion on a composition pixel of the sensor image and generates an RGB image of the Bayer array, by calculating a pixel value based on an estimation in which the W pixel and the RGB pixel have a positive correlation.
-
-
4. An image processing apparatus comprising:
-
an image sensor that has an RGBW array including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light; and an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor, wherein the image sensor has a periodic array of a unit composition formed of each RGBW pixel, wherein the image processor converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array, wherein the image sensor is configured to output an image including a long-period exposure pixel and a short-period exposure pixel which are set according to a control of a controller, as the sensor image, wherein the image processor has an array converter which generates a high dynamic range (HDR) image by applying the sensor image including the long-period exposure pixel and the short-period exposure pixel, and which generates an RGB array HDR image formed of the Bayer array by converting the pixel array, and wherein the array converter has a W pixel interpolator which generates a whole W pixel long-period exposure image in which a long-period exposure W pixel is set for all pixel positions, by applying a long-period exposure pixel signal of the sensor image, and which generates a whole W pixel short-period exposure image in which a short-period exposure W pixel is set for all pixel positions, by applying a short-period exposure pixel signal of the sensor image;
an RGBW pixel generator which generates a long-period exposure RGBW corresponding low frequency signal which is a low frequency signal of each long-period exposure RGBW pixel signal at each pixel position, by applying the long-period exposure pixel signal of the sensor image, and which generates a short-period exposure RGBW corresponding low frequency signal which is a low frequency signal of each short-period exposure RGBW pixel signal at each pixel position, by applying the short-period exposure pixel signal of the sensor image;
an HDR composition processor which generates a whole W pixel image signal as the high dynamic range (HDR) image and the RGBW corresponding low frequency signal by inputting the whole W pixel long-period exposure image, the whole W pixel short-period exposure image, the long-period exposure RGBW corresponding low frequency signal, and the short-period exposure RGBW corresponding low frequency signal; and
a correlation processor which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal as the high dynamic range (HDR) image generated by the HDR composition processor, and which performs array conversion on a composition pixel of the sensor image and generates an RGB image of the Bayer array, by calculating a pixel value based on an estimation in which the W pixel and the RGB pixel have a positive correlation. - View Dependent Claims (5, 6)
-
-
7. An image processing apparatus comprising:
-
an image sensor that has an RGBW array including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light; and an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing by inputting a sensor image formed of an RGBW pixel signal output from the image sensor, wherein the image sensor has a periodic array of a unit composition formed of each RGBW pixel, wherein the image processor converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array, wherein in the image sensor, two rows of a long-period exposure pixel row and a short-period exposure pixel row are alternately set according to a control of a controller, wherein the image sensor is configured to output an image which is formed of a long-period exposure pixel signal having an added pixel value of the same color pixel values which are adjacent to each other in an oblique direction of the long-period exposure pixel row in a unit of two rows, and a short-period exposure pixel signal having an added pixel value of the same color pixel values which are adjacent to each other in an oblique direction of the short-period exposure pixel row in a unit of two rows, as the sensor image, wherein the image processor has an array converter which generates a high dynamic range (HDR) image by applying the sensor image including the long-period exposure pixel and the short-period exposure pixel, and which generates an RGB array HDR image formed of the Bayer array by converting the pixel array, and wherein the array converter has a W pixel interpolator which generates a whole W pixel long-period exposure image in which a long-period exposure W pixel is set for all pixel positions, by applying a long-period exposure pixel signal of the sensor image, and which generates a whole W pixel short-period exposure image in which a short-period exposure W pixel is set for all pixel positions, by applying a short-period exposure pixel signal of the sensor image;
an RGBW pixel generator which generates a long-period exposure RGBW corresponding low frequency signal which is a low frequency signal of each long-period exposure RGBW pixel signal at each pixel position, by applying the long-period exposure pixel signal of the sensor image, and which generates a short-period exposure RGBW corresponding low frequency signal which is a low frequency signal of each short-period exposure RGBW pixel signal at each pixel position, by applying the short-period exposure pixel signal of the sensor image;
an HDR composition processor which generates a whole W pixel image signal as the high dynamic range (HDR) image and the RGBW corresponding low frequency signal by inputting the whole W pixel long-period exposure image, the whole W pixel short-period exposure image, the long-period exposure RGBW corresponding low frequency signal, and the short-period exposure RGBW corresponding low frequency signal; and
a correlation processor which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal as the high dynamic range (HDR) image generated by the HDR composition processor, and which performs array conversion on a composition pixel of the sensor image and generates an RGB image of the Bayer array, by calculating a pixel value based on an estimation in which the W pixel and the RGB pixel have a positive correlation. - View Dependent Claims (8, 9)
-
-
10. An image processing method executed in an image processing apparatus,
wherein the image processing apparatus includes an image sensor that has a periodic array of a unit composition formed of each RGBW pixel including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light, and an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing on a sensor image which is an output of the image sensor, wherein the image processor converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array, wherein the image processor has an array converter which generates the RGB array image formed of the Bayer array, and wherein the array converter has a W pixel interpolator which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image; - an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
a correlation processor which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which performs array conversion for a composition pixel of the sensor image and generates an RGB image of the Bayer array by calculating a pixel value based on an estimation in which the W pixel and the R, G and B pixels have a positive correlation.
- an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
-
11. A non-transitory computer-readable storage device encoded with computer-executable instructions that, when executed by an image processing apparatus, perform image processing,
wherein the image processing apparatus includes an image sensor that has a periodic array of a unit composition formed of each RGBW pixel including R, G and B pixels which are light receiving elements corresponding to wavelength lights of each RGB color and a W pixel which is a light receiving element receiving substantially all RGB wavelength light, and an image processor comprising a processing device and a memory storing instructions that, when executed by the processing device, perform image processing on a sensor image which is an output of the image sensor, wherein the image processing unit converts a pixel array of the sensor image formed of the RGBW pixel signal, and performs array conversion processing for generating an RGB array image formed of a Bayer array, wherein the image processor has an array converter which generates the RGB array image formed of the Bayer array, and wherein the array converter has a W pixel interpolator which generates a whole W pixel image signal by setting the W pixel at an RGB pixel position of the sensor image; - an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
a correlation processor which inputs the whole W pixel image signal and the RGBW corresponding low frequency signal, and which performs array conversion for a composition pixel of the sensor image and generates an RGB image of the Bayer array by calculating a pixel value based on an estimation in which the W pixel and the R, G and B pixels have a positive correlation.
- an RGBW pixel generator which generates an RGBW corresponding low frequency signal which is a low frequency signal of each RGBW pixel signal at each pixel position of the sensor image; and
Specification