Imaging apparatus and imaging processing method for detecting and correcting flash band

• US 9,215,379 B2
• Filed: 01/17/2012
• Issued: 12/15/2015
• Est. Priority Date: 01/24/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Gomita, Jun
• Primary Examiner(s): KHAN, USMAN A
• Application Number:
  US13/351,412
• Publication Number:
  US 20120188403A1
• Time in Patent Office: 1,428 Days
• Field of Search: 348/226.1
• US Class Current: 1/1
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A flash band processing circuit includes: a flash band detecting circuit that detects a start line and an end line of a flash band, which is a level difference for each line generated within a frame in accordance with flash light, based on a difference in exposure periods of a video signal output for each frame by a pixel included in an imaging device employing a rolling shutter system.

Claims

1. A flash band processing circuit comprising:

a flash band detecting circuit that detects a start line and an end line of a flash band, which is a difference in luminance level for each line generated within a frame of video frames input to the flash band processing circuit, in accordance with flash light in an imaging device employing a rolling shutter system, based on a comparison of luminance level of a pixel located at a position in a current frame and luminance level of a pixel located at the same position in a past frame for one or more frames output by the imaging device, wherein the start line is included in the past frame of the video frames and the end line is included in the current frame of the video frames.

2. The flash band processing circuit according to claim 1, wherein the flash band detecting circuit detects generation of the flash band within the current frame output by the imaging device or generation of the flash band from the past frame, output by the imaging device from a predetermined number of frames before the current frame, to the current frame.

3. The flash band processing circuit according to claim 2, wherein the flash band detecting circuit detects the generation of the flash band by:

acquiring a comparison result by comparing luminance levels between frames in which the current frame and the past frame are consecutive for each pixel,

specifying a horizontal line in which a variation amount is large based on the comparison result, and

detecting a line that is a boundary between consecutive horizontal lines in which the variation amount is large and the consecutive horizontal lines in which the variation amount is small.

4. A flash band processing circuit comprising:

a flash band detecting circuit that detects a start line and an end line of a flash band, which is a level difference for each line generated within a frame in accordance with flash light, based on a difference in exposure periods of a video signal output for each frame by a pixel included in an imaging device employing a rolling shutter system;

a level comparing unit that acquires a ratio of luminance levels of the pixel present at a specific position of a current frame and a past frame as a level variation ratio and acquires a level variation amount that indicates whether or not the level variation ratio is equal to or higher than a first threshold value as a variation amount determining value, wherein the past frame is output by the imaging device a predetermined number of frames before the current frame;

a horizontal integration unit that acquires a first integral value determining a value that indicates whether a horizontal integral value acquired by integrating the variation amount determining value, of which the level variation ratio is equal to or higher than the first threshold value, for each pixel included in a horizontal line of the current frame is equal to or larger than a second threshold value and acquires a first integral value sign that represents a positive sign or a negative sign of the first integral value;

an area determining unit that, based on the first integral value and the first integral value sign, outputs an increased area determining value determining an increased area in which luminance level of the pixel included in the current frame is higher than that in the past frame in a case where the first integral value sign is positive and outputs a decreased area determining value determining a decreased area in which luminance level of the pixel included in the current frame is lower than that in the past frame in a case where the first integral value sign is negative;

a vertical integration unit that outputs a flash band start determining value representing a start of the flash band within the frame in a case where an increased area determining value integral value acquired by integrating the increased area determining value for each horizontal line in the vertical direction is equal to or larger than a third threshold value and outputs a flash band end determining value representing an end of the flash band within the frame in a case where a decreased area determining value integral value acquired by integrating the decreased area determining value for each horizontal line in the vertical direction is equal to or larger than a fourth threshold value; and

a determination maintaining unit that maintains the flash band start determining value and the flash band end determining value for a predetermined period in accordance with a line period and a frame period of the frame, outputs a flash band start line determining value determining the start line of the flash band based on the flash band start determining value, and outputs a flash band end line determining value determining the end line of the flash band based on the flash band end determining value to the area determining unit.

5. The flash band processing circuit according to claim 4, wherein the horizontal integration unit acquires a second integral value determining value representing whether or not an integral value acquired by integrating a result of subtracting one variation amount determining value from the other variation amount determining value is equal to or larger than a fifth threshold value for each pixel included in the horizontal lines adjacent to each other in the vertical direction located at a same position in the horizontal direction and acquires a second integral value sign representing whether the second integral value has a positive sign or a negative sign, and

wherein the area determining unit detects whether or not the flash band due to the flash light is present by determining whether or not a high-frequency component in the vertical direction is present within the frame based on the second integral value determining value and the second integral value sign.

6. The flash band processing circuit according to claim 5, wherein a sign that represents the detection of the flash band is attached to an auxiliary data area of the frame in which the flash band has been detected.

7. The flash band processing circuit according to claim 6, further comprising:

a flash band correcting unit that outputs an output frame of which the flash band generated within the frame is corrected based on the flash band start determining value and the flash band end determining value.

8. The flash band processing circuit according to claim 7, wherein the flash band correcting unit compares luminance values of the pixel included in the current frame and the past frame and outputs the output frame in which the pixel having a larger luminance value is replaced.

9. The flash band processing circuit according to claim 7, wherein the flash band correcting unit compares luminance values of the pixel included in the current frame and the past frame and outputs the output frame in which the pixel having a smaller luminance value is replaced.

10. The flash band processing circuit according to claim 7, wherein the flash band correcting unit outputs the output frame that is acquired by replicating the past frame.

11. The flash band processing circuit according to claim 7, wherein the flash band correcting unit outputs the output frame acquired by adding and averaging luminance values of the pixel included in the current frame and the past frame.

12. A method of processing a flash band, the method comprising:

in a flash band processing circuit;

detecting a start line and an end line of a flash band, which is a difference in luminance level for each line generated within a frame of video frames input to the flash band processing circuit, in accordance with flash light in an imaging device employing a rolling shutter system, based on a comparison of luminance level of a pixel located at a position in a current frame and luminance level of a pixel located at the same position in a past frame for one or more frames output by the imaging device, wherein the start line is included in the past frame of the video frames and the end line is included in the current frame of the video frames.

13. An imaging apparatus comprising:

a flash band detecting circuit that detects a start line and an end line of a flash band, which is a difference in luminance level for each line generated within a frame of video frames input to the flash band detecting circuit, in accordance with flash light in an imaging device employing a rolling shutter system, based on a comparison of luminance level of a pixel located at a position in a current frame and luminance level of a pixel located at the same position in a past frame for one or more frames output by the imaging device; and

a flash band correcting circuit that outputs one or more output frames that are corrected to a video in which there is no flash band generated within the one or more output frames,

wherein the start line is included in the past frame of the video frames and the end line is included in the current frame of the video frames.

14. An imaging processing method comprising:

in a flash band processing circuit;

detecting a start line and an end line of a flash band, which is a difference in luminance level for each line generated within a frame of video frames input to the flash band processing circuit, in accordance with flash light in an imaging device employing a rolling shutter system, based on a comparison of luminance level of a pixel located at a position in a current frame and luminance level of a pixel located at the same position in a past frame for one or more frames output by the imaging device; and

outputting one or more output frames that are corrected to a video in which there is no flash band generated within the one or more output frames,

wherein the start line is included in the past frame of the video frames and the end line is included in the current frame of the video frames.