IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD

• US 20150161773A1
• Filed: 04/26/2013
• Published: 06/11/2015
• Est. Priority Date: 05/09/2012
• Status: Active Grant

• Current Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Original Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Inventors: Takahashi, Masashi, Chihara, Nobuhiro, Yamaguchi, Muneaki
• Granted Patent: US 10,521,885 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Provided is an image processing apparatus that can reduce degradation in image quality due to heat haze and the like. An input image memory 110 stores an input image group 200 and an input image 210. A time-series smoothed image memory 112 stores a time-series smoothed image 220 obtained by time series smoothing. A moving body removed image memory 113 stores a moving body removed image 230 that is a background image. A histogram analyzing unit 130 computes a differential of histograms of partial images for the input image 210 and the time-series smoothed image 220 or the moving body removed image 230, and calculates similarity information between the images. On the basis of the similarity information, a region dividing unit 140 divides the input image 210 into a moving body region, and a region other than the moving body region. An image correcting unit 150 performs image synthesis of the input image 210 or the time series-smoothed image 220, for each of the regions.

Claims

1-5. -5. (canceled)

6. An image processing device for reducing degradation in visual images, comprising:

a smoothed image updater generating a temporally-smoothed image from images close to an input image in time;

an histogram analyzer generating histograms of the input image and the temporally-smoothed image in respective windows centered at positions to be compared each other and generating similarity information of images based on difference of the histograms;

a region divider dividing the input image, depending on the similarity information, into a region of moving object and a region other than the region of moving object; and

an image corrector generating a synthesized image by combining the input image and the temporally-smoothed image selectively according to the regions and providing a corrected image by applying adaptive filter in space domain to the synthesized image.

7. The image processing device of claim 6, wherein:

the region divider divides the image with two thresholds of higher and lower values in a selective manner, wherein the lower threshold is applied around a region of pixels with values higher than the higher threshold and the higher threshold is applied to others or wherein the higher threshold is applied around a region of pixels with values lower than the lower threshold and the lower threshold is applied to others.

8. The image processing device of claim 6, wherein:

the adaptive filter of the image corrector calculates edge strength of the input image or the synthesized image, classifies the image into three categories of frontier region, texture region and flat region depending on comparison of the edge strength with a first threshold and a second threshold and applies different filter processing for each categorized region to the synthesized image.

9. The image processing device of claim 8, wherein:

the adaptive filter of the image corrector applies to the frontier region a edge-smoothing filter smoothening contours, to the texture region a bilateral filter suit for smoothing while saving contours and to the flat region a Gaussian filter defocusing softly.

10. The image processing device of claim 7, further comprising:

a distortion calculator calculating strength of local turbulence in the input image; and

a parameter optimizer controlling, depending on the strength of local turbulence, at least one of an interval of the positions to be compared, a size of the window and the threshold.

11. The image processing device of claim 6, further comprising:

a super-resolution processor locally aligning to and overlaying with the corrected image the images close to the input image in time,

wherein the locally aligning is only processed within a same region and is not processed between different regions in the region of moving object, the region of background and the region of quasi-moving object.

12. The image processing device of claim 8, further comprising:

a super-resolution processor locally aligning to and overlaying with the corrected image the images close to the input image in time,

wherein the locally aligning is only processed within a same region in the region of moving object, the region of background and the region of quasi-moving object and is not processed in the flat region where the edge strength becomes weaker.

13. The image processing device of claim 6, wherein the histogram analyzer calculates the difference of the histograms in a manner of summing plural version of difference of the histograms with different number of tones.

14. An image processing device for reducing degradation in visual images, comprising:

a smoothed image updater generating a temporally-smoothed image from images close to an input image in time;

an histogram analyzer generating histograms of the input image, the temporally-smoothed image and an object-removed image in respective windows centered at positions to be compared each other and generating similarity information of images based on difference of the histograms;

a region divider dividing the input image, depending on the similarity information, into a region of moving object, a region of background and a region of quasi-moving object where through an moving object just passed;

an object-removed image updater generating the object-removed image by moving-averaging only in the region of background and a region of quasi moving object of the input images or by choosing a median of pixel values from the images close to the input image; and

an image corrector generating a synthesized image by combining the input image, the temporally-smoothed image and the object-removed image chosen according to the regions and providing a corrected image by applying adaptive filter in space domain to the synthesized image.

15. The image processing device of claim 14, wherein;

the region divider divides the image with two threshold of higher and lower values in a selective manner, wherein the lower threshold is applied around a region of pixels with values higher than the higher threshold and the higher threshold is applied to others or wherein the higher threshold is applied around a region of pixels with values lower than the lower threshold and the lower threshold is applied to others.

16. The image processing device of claim 14, wherein:

the histogram analyzer generates the similarity information including a first evaluation cost based on difference between the histogram of the input image and the histogram of the temporally-smoothed image and a second evaluation cost based on difference between the histogram of the input image and the histogram of the object-removed image; and

the region divider classifies a region where both the first evaluation cost and the second evaluation cost are higher than a threshold as the region of moving object, a region where the second evaluation cost is higher than the first evaluation cost as the region of background and the rest as the region of quasi moving object.

17. The image processing device of claim 14, wherein:

the adaptive filter of the image corrector calculates edge strength of the input image or the synthesized image, classifies the image into three categories of frontier region, texture region and flat region depending on comparison of the edge strength with a first threshold and a second threshold and applies different filter processing for each categorized region to the synthesized image.

18. The image processing device of claim 17, wherein:

the adaptive filter of the image corrector applies to the frontier region a edge-smoothing filter smoothening contours, to the texture region a bilateral filter suit for smoothing while saving contours and to the flat region a Gaussian filter defocusing softly.

19. The image processing device of claim 16, wherein:

the image corrector chooses pixels of the input image in corresponding positions for the region of moving object, pixels of the temporally-smoothed image in corresponding positions for the region of background, and pixels of the object-removed image in corresponding positions for the region of quasi moving object.

20. The image processing device of claim 16, further comprising:

a distortion calculator calculating strength of local turbulence in the input image; and

a parameter optimizer controlling, depending on the strength of local turbulence, at least one of an interval of the positions to be compared, a size of the window and the threshold.

21. The image processing device of claim 14, further comprising:

a super-resolution processor locally aligning to and overlaying with the corrected image the images close to the input image in time,

wherein the locally aligning is only processed within a same region and is not processed between different regions in the region of moving object, the region of background and the region of quasi-moving object.

22. The image processing device of claim 17, further comprising:

a super-resolution processor locally aligning to and overlaying with the corrected image the images close to the input image in time,

wherein the locally aligning is only processed within a same region in the region of moving object, the region of background and the region of quasi-moving object and is not processed in the flat region where the edge strength becomes weaker.

23. The image processing device of claim 14, wherein the histogram analyzer calculates the difference of the histograms in a manner of summing plural version of difference of the histograms with different number of tones.

25. The image processing method of claim 23, wherein;

the dividing step includes dividing the image with two threshold of higher and lower values in a selective manner, wherein the lower threshold is applied around a region of pixels with values higher than the higher threshold and the higher threshold is applied to others or wherein the higher threshold is applied around a region of pixels with values lower than the lower threshold and the lower threshold is applied to others.

24. An image processing method for reducing degradation in visual images by a computer, comprising the steps of:

updating wherein the computer generates a temporally-smoothed image from images close to an input image in time;

analyzing wherein the computer generates histograms of the input image, the temporally-smoothed image and a object-removed image in respective windows centered at positions to be compared each other and generates similarity information of images based on difference of the histograms;

dividing the input image, depending on the similarity information, into a region of moving object, a region of background and a region of quasi-moving object where through an moving object just passed;

updating wherein the computer generates the object-removed image by moving-averaging only in the region of background and a region of quasi moving object of the input images or by choosing a median of pixel values from the images close to the input image; and

correcting wherein the computer generates synthesized image by combining the input image, temporally-smoothed image and the object-removed image which are chosen according to the regions and provides a corrected image by applying adaptive filter in space domain to the synthesized image.