Image processing device, image processing method, and solid-state imaging device

• US 8,823,825 B2
• Filed: 02/05/2013
• Issued: 09/02/2014
• Est. Priority Date: 02/24/2012
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Tatsuzawa, Yukiyasu, Hiwada, Kazuhiro
• Primary Examiner(s): Misleh, Justin P
• Application Number:
  US13/759,269
• Publication Number:
  US 20130222625A1
• Time in Patent Office: 574 Days
• Field of Search: None
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

According to one embodiment, a high dynamic range synthesizing circuit includes an interpolation processing unit. The interpolation processing unit generates an interpolated signal for a first pixel, which is set as a target pixel, through an interpolation process using a second image signal from a second pixel which is a peripheral pixel. The interpolation processing unit generates an interpolated signal for the second pixel, which is set as a target pixel, through an interpolation process using the first image signal from the first pixel which is a peripheral pixel.

Claims

1. An image processing device comprising:

a high dynamic range synthesizing circuit that synthesizes a first image signal in accordance with an amount of incident light on each first pixel during a first charge accumulation period and a second image signal in accordance with an amount of incident light on each second pixel during a second charge accumulation period shorter than the first charge accumulation period,

wherein the high dynamic range synthesizing circuit includes

a signal level adjusting unit that adjusts a signal level of the second image signal in accordance with a ratio between the first and second charge accumulation periods,

an interpolation processing unit to which the first image signal and the second image signal of which the signal level is adjusted by the signal level adjusting unit are input and that generates interpolated signals, and

an expansion processing unit that performs an expansion process between the interpolated signals by a signal level exceeding an output saturation level at which saturation of an output charge for the amount of incident light during the first charge accumulation period occurs,

wherein the interpolation processing unit

generates the interpolated signal through an interpolation process on the first pixel, which is set as a target pixel, using the second image signal from each second pixel which is a peripheral pixel located in periphery of the first pixel which is the target pixel, and

generates the interpolated signal through an interpolation process on the second pixel, which is set as a target pixel, using the first image signal from each first pixel which is a peripheral pixel located in periphery of the second pixel which is the target pixel.

2. The image processing device according to claim 1, wherein the interpolation processing unit performs the interpolation process such that a signal component originated from the first pixel and a signal component originated from the second pixel are equally included.

3. The image processing device according to claim 1,

wherein the high dynamic range synthesizing circuit further includes a mixture signal adding unit that adds a first mixture signal in which the first and second image signals are mixed at a first ratio and a second mixture signal in which the first and second image signals are mixed at a second ratio,

wherein the second ratio is a ratio in which the first image signal is weighted for the first ratio,

wherein the interpolation processing unit concurrently generates the first and second mixture signals as the interpolated signals, and

wherein the mixture signal adding unit adds the first and second mixture signals subjected to the expansion process at a ratio appropriate to luminance of the target pixel.

4. The image processing device according to claim 3, wherein the mixture signal adding unit performs adjustment so that the second mixture signal is weighted, when the luminance of the target pixel is lower than luminance corresponding to the output saturation level.

5. The image processing device according to claim 3, wherein the mixture signal adding unit outputs the first mixture signal, when the luminance of the target pixel is higher than luminance corresponding to the output saturation level.

6. The image processing device according to claim 3, wherein the first ratio is a ratio of 1:

1.

7. The image processing device according to claim 3, wherein the interpolation processing unit changes the second ratio in accordance with a switchable set code.

8. The image processing device according to claim 3, wherein the expansion processing unit changes a ratio of the expansion process for the second mixture signal in accordance with a switchable set code.

9. The image processing device according to claim 1, wherein the peripheral pixels are eight pixels included in a pixel block centering on the target pixel and are pixels with the same color as that of the target pixel.

10. An image processing method comprising:

high dynamic range synthesis of synthesizing a first image signal in accordance with an amount of incident light on each first pixel during a first charge accumulation period and a second image signal in accordance with an amount of incident light on each second pixel during a second charge accumulation period shorter than the first charge accumulation period,

wherein the high dynamic range synthesis includes

adjusting a signal level of the second image signal in accordance with a ratio between the first and second charge accumulation periods,

generating interpolated signals using the first image signal and the second image signal of which the signal level is adjusted, and

performing an expansion process between the interpolated signals by a signal level exceeding an output saturation level at which saturation of an output charge for the amount of incident light during the first charge accumulation period occurs,

wherein, in the generating of the interpolated signals,

an interpolation process is performed on the first pixel, which is set as a target pixel, using the second image signal from each second pixel which is a peripheral pixel located in periphery of the first pixel which is the target pixel, and

an interpolation process is performed on the second pixel, which is set as a target pixel, using the first image signal from each first pixel which is a peripheral pixel located in periphery of the second pixel which is the target pixel.

11. The image processing method according to claim 10, wherein the interpolation process is performed such that a signal component originated from the first pixel and a signal component originated from the second pixel are equally included.

12. The image processing method according to claim 10,

wherein the high dynamic range synthesis further includes adding a first mixture signal in which the first and second image signals are mixed at a first ratio and a second mixture signal in which the first and second image signals are mixed at a second ratio,

wherein the second ratio is a ratio in which the first image signal is weighted for the first ratio,

wherein the first and second mixture signals are concurrently generated as the interpolated signals, and

wherein the first and second mixture signals subjected to the expansion process are added at a ratio appropriate to luminance of the target pixel.

13. The image processing method according to claim 12, wherein in the adding of the first and second mixture signals, adjustment is performed so that the second mixture signal is weighted, when the luminance of the target pixel is lower than luminance corresponding to the output saturation level.

14. The image processing method according to claim 12, wherein the first mixture signal is output as a high dynamic range synthesis signal, when the luminance of the target pixel is higher than luminance corresponding to the output saturation level.

15. The image processing method according to claim 12, wherein the first ratio is a ratio of 1:

1.

16. The image processing method according to claim 12, wherein the second ratio is changed in accordance with a switchable set code.

17. The image processing method according to claim 12, wherein a ratio of the expansion process for the second mixture signal is changed in accordance with a switchable set code.

18. The image processing method according to claim 10, wherein the peripheral pixels are eight pixels included in a pixel block centering on the target pixel and are pixels with the same color as that of the target pixel.

19. A solid-state imaging device comprising:

a pixel array that first pixels detecting an amount of incident light during a first charge accumulation period and second pixels detecting an amount of incident light during a second charge accumulation period shorter than the first charge accumulation period; and

a high dynamic range synthesizing circuit that synthesizes a first image signal output in accordance with the amount of incident light by the first pixel and a second image signal output in accordance with the amount of incident light by the second pixel,

wherein the high dynamic range synthesizing circuit includes

a signal level adjusting unit that adjusts a signal level of the second image signal in accordance with a ratio between the first and second charge accumulation periods,

an interpolation processing unit to which the first image signal and the second image signal of which the signal level is adjusted by the signal level adjusting unit are input and that generates interpolated signals, and

an expansion processing unit that performs an expansion process between the interpolated signals by a signal level exceeding an output saturation level at which saturation of an output charge for the amount of incident light during the first charge accumulation period occurs,

wherein the interpolation processing unit

generates the interpolated signal through an interpolation process on the first pixel, which is set as a target pixel, using the second image signal from each second pixel which is a peripheral pixel located in periphery of the first pixel which is the target pixel, and

generates the interpolated signal through an interpolation process on the second pixel, which is set as a target pixel, using the first image signal from each first pixel which is a peripheral pixel located in periphery of the second pixel which is the target pixel.

20. The solid-state imaging device according to claim 19,

wherein in the pixel array, a first line area in which the first pixels are parallel in a horizontal direction and a second line area in which the second pixels are parallel in the horizontal direction are alternately arranged in a vertical direction, and

wherein the interpolation processing unit

performs an interpolation process on the first pixel, which is set as the target pixel, using the second image signals from the second pixels which are the peripheral pixels included in the second line area adjacent to the first line area including the first pixel which is the target pixel, and

performs an interpolation process on the second pixel, which is set as the target pixel, using the first image signals from the first pixels which are the peripheral pixels included in the first line area adjacent to the second line area including the second pixel which is the target pixel.