Image processing device, imaging device, and image processing method

• US 10,217,034 B2
• Filed: 10/22/2015
• Issued: 02/26/2019
• Est. Priority Date: 01/08/2015
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Inventors: Hoshino, Kazuhiro
• Primary Examiner(s): Nakhjavan, Shervin
• Application Number:
  US15/540,597
• Publication Number:
  US 20170357881A1
• Time in Patent Office: 1,223 Days
• Field of Search: None
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/04 : using optical or ultrasonic detectors
• CPC Class Codes: H04N 2209/045 : using mosaic colour filter
• CPC Class Codes: H04N 2209/046 : Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/56 : provided with illuminating means
• CPC Class Codes: H04N 25/131 : including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 : based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/60 : Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 5/144 : Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/208 : for compensating for attenuation of high frequency components, e.g. crispening, aperture distortion correction
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 9/646 : for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

Visibility of a license plate and color reproducibility of a vehicle body are improved in a monitoring camera.

A vehicle body area detection unit detects a vehicle body area of a vehicle from an image signal. A license plate area detection unit detects a license plate area of the vehicle from the image signal. A vehicle body area image processing unit performs processing of the image signal corresponding to the detected vehicle body area. A license plate area image processing unit performs processing different from the processing of the image signal corresponding to the vehicle body area on the image signal corresponding to the detected license plate area. A synthesis unit synthesizes the processed image signal corresponding to the vehicle body area and the processed image signal corresponding to the license plate area.

Claims

1. An image processing device comprising:

a vehicle body area detection unit which detects a vehicle body area of a vehicle from an image signal;

a license plate area detection unit which detects a license plate area of the vehicle from the image signal;

a vehicle body area image processing unit which performs processing of the image signal corresponding to the detected vehicle body area;

a license plate area image processing unit which performs processing different from the processing of the image signal corresponding to the vehicle body area on the image signal corresponding to the detected license plate area; and

a synthesis unit which synthesizes the processed image signal corresponding to the vehicle body area and the processed image signal corresponding to the license plate area.

2. The image processing device according to claim 1, wherein the license plate area image processing unit performs luminance signal conversion processing to convert the image signal corresponding to the license plate area to a luminance signal corresponding to the license plate area and edge enhancement processing to enhance an edge portion of an image in the luminance signal.

3. The image processing device according to claim 1, wherein the license plate area image processing unit performs luminance signal conversion processing to convert the image signal corresponding to the license plate area to a luminance signal corresponding to the license plate area and two-dimensional noise reduction processing to remove noise of the luminance signal by attenuating high-frequency components of luminance signals belonging to the same frame.

4. The image processing device according to claim 1, wherein the vehicle body area image processing unit performs luminance/chrominance signal conversion processing to convert the image signal corresponding to the vehicle body area to a luminance signal and a chrominance signal corresponding to the vehicle body area and three-dimensional noise reduction processing to remove noise of the luminance signal and the chrominance signal by using luminance signals and chrominance signals belonging to a plurality of continuous frames.

5. The image processing device according to claim 1, wherein

the image signal includes image signals corresponding to red light, green light, blue light, and infrared light,

the vehicle body area image processing unit performs processing of the image signals corresponding to the red light, the green light, and the blue light included in the image signal corresponding to the vehicle body area, and

the license plate area image processing unit performs processing of an image signal corresponding to the infrared light included in the image signal corresponding to the license plate area.

6. The image processing device according to claim 5, wherein

the image signal further includes an image signal corresponding to white light, and

the vehicle body area image processing unit performs processing of the image signals corresponding to the red light, the green light, the blue light, and the white light included in the image signal corresponding to the vehicle body area.

7. The image processing device according to claim 1, wherein

the image signal includes image signals corresponding to red light, green light, blue light, and infrared light,

the image processing device further comprising a saturation detection unit which detects whether an image signal corresponding to the infrared light included in the image signal is saturated, and

the license plate area image processing unit performs processing of the image signal corresponding to the infrared light included in the image signal corresponding to the license plate area in a case where the saturation detection unit detects that the image signal corresponding to the infrared light is not saturated and performs processing of the image signals corresponding to the red light, the green light, and the blue light included in the image signal corresponding to the license plate area in a case where the saturation detection unit detects that the image signal corresponding to the infrared light is saturated.

8. An imaging device comprising:

an image sensor which generates an image signal;

a vehicle body area detection unit which detects a vehicle body area of a vehicle from the image signal;

a license plate area detection unit which detects a license plate area of the vehicle from the image signal;

a vehicle body area image processing unit which performs processing of the image signal corresponding to the detected vehicle body area;

a license plate area image processing unit which performs processing different from the processing of the image signal corresponding to the vehicle body area on the image signal corresponding to the detected license plate area; and

a synthesis unit which synthesizes the processed image signal corresponding to the vehicle body area and the processed image signal corresponding to the license plate area.

9. An image processing method comprising:

a vehicle body area detecting procedure to detect a vehicle body area of a vehicle from an image signal;

a license plate area detecting procedure to detect a license plate area of the vehicle from the image signal;

a vehicle body area image processing procedure to perform processing of the image signal corresponding to the detected vehicle body area;

a license plate area image processing procedure to perform processing different from the processing of the image signal corresponding to the vehicle body area on the image signal corresponding to the detected license plate area; and

a synthesis procedure to synthesize the processed image signal corresponding to the vehicle body area and the processed image signal corresponding to the license plate area.