Image processing device

• US 9,319,647 B2
• Filed: 10/04/2012
• Issued: 04/19/2016
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Kanou, Tadahiko, Adachi, Masahiko
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): Wu, Zhenzhen
• Application Number:
  US14/364,902
• Publication Number:
  US 20140347530A1
• Time in Patent Office: 1,293 Days
• Field of Search: 348/254, 348/255, 348/276, 348/222.1
• US Class Current: 1/1
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/841 to modify gamut
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

In an original image color pixels and transparent pixels are arranged, the color pixels having gradation values assigned according to received light levels of respective color light-receiving pixels and the transparent pixels having gradation values assigned according to received light levels of respective transparent light-receiving pixels. An image processing device includes an image generation unit which generates a high-sensitivity image for the original image by calculating for each color pixel, a corrected gradation value correcting a first reference gradation value of gray calculated based on the gradation value of itself or the gradation values of other color pixels in the periphery, according to the gradation values of transparent pixels in the periphery; and the corrected gradation value of each color pixel and the gradation value of each transparent pixel are assigned as gradation values of the pixels in the corresponding arranged positions of the high-sensitivity image.

Claims

1. An image processing device comprising:

a camera which takes an image using an image pickup device including an arrangement of a plurality of color light-receiving pixels which receive light through a color filter and a plurality of transparent light-receiving pixels which receive light without using a color filter;

a computer, including at least a central processing unit, communicatively coupled to the camera; and

a non-transitory computer readable medium storing instructions for controlling the central processing unit to function as an image generation unit which generates at least one of a high-sensitivity image and a wide dynamic range image based on a corrected gradation value of each of a plurality of color pixels, for an original image taken by the camera, the original image including an arrangement of the plurality of color pixels having gradation values individually assigned according to the received light levels of the respective color light-receiving pixels and a plurality of transparent pixels having gradation values individually assigned according to the received light levels of the respective transparent light-receiving pixels, by calculating, for each of the plurality of color pixels, the corrected gradation value in which a first reference gradation value of gray calculated based on the gradation value of itself or the gradation values of other color pixels arranged in the periphery is corrected according to the gradation values of transparent pixels arranged in the periphery, wherein;

the image generation unit generates the high-sensitivity image by assigning the corrected gradation value of each of the plurality of color pixels and the gradation value of each of the plurality of transparent pixels as gradation values of pixels in a corresponding arranged positions of the high-sensitivity image,

the color filter is a three-primary-color filter and the color light-receiving pixel receives light through a filter of any one of the three primary colors.

the image generation unit calculates, for each transparent pixel of the original image, the gradation values of the three primary colors on the basis of the gradation values of the color pixels arranged in the periphery and calculates a gradation difference degree between a second reference gradation value of gray calculated from the gradation values of the three primary colors and the gradation value of itself,

the image generation unit calculates, for each color pixel of the original image, the gradation values of the three primary colors on the basis of the gradation value of itself or the gradation values of other color pixels arranged in the periphery and calculates the first reference gradation value from the gradation values of the three primary colors, and

the image generation unit calculates the corrected gradation value by correcting the first reference gradation value on the basis of the gradation difference degree of the transparent pixels arranged in the periphery.

2. The image processing device according to claim 1, wherein the image generation unit generates a wide dynamic range image by assigning, for each color pixel of the original image, a first composite gradation value obtained by combining the first reference gradation value of itself with the gradation value of a pixel. in the corresponding arranged position of the high-sensitivity image as the gradation value of a pixel in the corresponding arranged position of the wide dynamic range image, and by assigning, for each transparent pixel of the original image, a second composite gradation value obtained by combining the second reference gradation value of itself with the gradation value of a pixel in the corresponding arranged position of the high-sensitivity image as the gradation value of a pixel in the corresponding arranged position of the wide dynamic range image.

3. The image processing device according to claim 2, wherein the image generation unit calculates the first composite gradation value and the second composite gradation value by performing an addition with weighting by conversion to values in the same range between the first reference gradation value or the second reference gradation value and the gradation value of the high-sensitivity image.

4. The image processing device according to claim 2, wherein:

the image generation unit generates a color image by assigning, for each transparent pixel of the original image, the gradation value of a color calculated based on the gradation values of color pixels existing in the periphery as the gradation value of a pixel in the corresponding arranged position of the color image, and by assigning, for each color pixel of the original image, the gradation value of a color calculated based on the gradation value of itself or the gradation values of other color pixels arranged in the periphery as the gradation value of a pixel in the corresponding arranged position of the color image; and

the image processing device includes an object detection unit which performs a first object detection processing for detecting an object from the high-sensitivity image, a second object detection processing for detecting an object from the color image, and third object detection processing for detecting an object from the wide dynamic range image while switching among the first object detection processing, the second object detection processing, and the third object detection processing according to the type of object to be detected or according to imaging conditions of the camera.

5. The image processing device according to claim 1, wherein:

the image generation unit generates, for each transparent pixel of the original image, a color image by assigning the gradation value of a color calculated based on the gradation values of color pixels existing in the periphery as the gradation value of a pixel in the corresponding arranged position of the color image, and by assigning, for each color pixel of the original image, the gradation value of a color calculated based on the gradation value of itself or the gradation values of other color pixels arranged in the periphery as the gradation value of a pixel in the corresponding arranged position of the color image; and

the image processing device includes an object detection unit which performs a first object detection processing for detecting an object from the high-sensitivity image and a second object detection processing for detecting an object from the color image while switching between the first object detection processing and the second object detection processing according to the type of object to be detected.

6. The image processing device according to claim 1,

wherein the image generation unit generates the wide dynamic range image by assigning a first composite gradation value obtained by combining the first reference gradation value with the corrected gradation value as the gradation value of the pixel in the corresponding arranged position of the wide dynamic range image, and

by calculating, for each transparent pixel, a second reference gradation value of gray based on the gradation values of color pixels arranged in the periphery and assigning a second composite gradation value obtained by combining the second reference gradation value with the gradation value of itself as the gradation value of the pixel in the corresponding arranged position of the wide dynamic range image.

7. The image processing device according to claim 6, wherein:

the color filter is a three-primary-color filter and the color light-receiving pixel receives light through a filter of one of the three primary colors;

the image generation unit calculates, for each transparent pixel of the original image, the gradation values of the three primary colors on the basis of the gradation values of the color pixels arranged in the periphery, calculates the second reference gradation value from the respective gradation values of the three primary colors, and calculates a gradation difference degree between the gradation value of itself and the second reference gradation value; and

the image generation unit calculates, for each color pixel of the original image, the gradation values of the three primary colors on the basis of the gradation value of itself or the gradation values of other color pixels arranged in the periphery, calculates the first reference gradation value from the respective gradation values of the three primary colors, and calculates the corrected gradation value by correcting the first reference gradation value according to the gradation difference degree of the transparent pixels arranged in the periphery.

8. The image processing device according to claim 7, wherein the image generation unit calculates the first composite gradation value by performing an addition with weighting by conversion to values in the same range between the first reference gradation value and the corrected gradation value and calculates the second composite gradation value by performing an addition with weighting by conversion to values in the same range between the second reference gradation value and the gradation value of the transparent pixel.

9. The image processing device according to claim 6, wherein:

the image generation unit generates a color image by assigning, for each transparent pixel of the original image, the gradation value of a color calculated based on the gradation values of color pixels existing in the periphery as the gradation value of a pixel in the corresponding arranged position of the color image, and by assigning, for each color pixel of the original image, the gradation value of a color calculated based on the gradation value of itself or the gradation values of other color pixels arranged in the periphery as the gradation value of a pixel in the corresponding arranged position of the color image; and

the image processing device includes an object detection unit which performs second object detection processing for detecting an object from the color image and third object detection processing for detecting an object from the wide dynamic range image while switching between the second object detection processing and the third object detection processing according to the type of object to be detected or according to the imaging conditions of the camera.