IMAGE PROCESSING DEVICE, ALARMING APPARATUS, IMAGE PROCESSING SYSTEM, AND IMAGE PROCESSING METHOD

• US 20170236015A1
• Filed: 01/18/2017
• Published: 08/17/2017
• Est. Priority Date: 02/16/2016
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: HATTORI, Hideharu, IZUMI, Yoshifumi
• Granted Patent: US 10,303,960 B2
• Time in Patent Office: Days
• CPC Class Codes: B60Q 5/005 automatically actuated
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence

Abstract

An image processing device includes a contrasting correction unit, a color analysis unit, an object detection unit, and an object relation analysis unit. The contrasting correction unit creates a contrasting corrected image by correcting contrasting of an input image captured by a vehicle camera. The color analysis unit creates a color corrected image by correcting colors of the input image. The object detection unit detects a main sign included in the input image based on the contrasting corrected image and detects an auxiliary sign included in the input image based on the color corrected image. The object relation analysis unit recognizes a traffic sign as a combination of a main sign and auxiliary sign by associating the main sign and the auxiliary sign with each other based on a positional relationship between the main and auxiliary signs, which are detected by the object detection unit, in the input image.

Claims

1. An image processing device comprising:

a contrasting correction unit that creates a contrasting corrected image by correcting contrasting of an input image captured by a vehicle camera installed on a vehicle;

a color analysis unit that creates a color corrected image by correcting colors of the input image;

an object detection unit that detects a main sign included in the input image based on the contrasting corrected image and detects an auxiliary sign included in the input image based on the color corrected image; and

an object relation analysis unit that recognizes a traffic sign as a combination of the main sign and the auxiliary sign by associating the main sign and the auxiliary sign with each other based on a positional relationship between the main sign and the auxiliary sign, which have been detected by the object detection unit, in the input image.

2. The image processing device according to claim 1, further comprising:

an orientation determination unit that respectively determines orientations of the main sign and the auxiliary sign based on ratios of widths and heights of the main sign and the auxiliary sign in the input image.

3. The image processing device according to claim 1, further comprising:

a reliability determination unit that obtains a position of the traffic sign and determines reliability of the traffic sign based on a result of accumulating the obtained position of the traffic sign.

4. The image processing device according to claim 1,

wherein the contrasting correction unit sets a target region in a predetermined range in the input image and creates the contrasting corrected image based on an average value of color brightness in a region, which corresponds to the target region, in a past image captured before the input image and an average value of color brightness in the target region in the input image.

5. The image processing device according to claim 1,

wherein the color analysis unit sets a target region in a predetermined range in the input image and creates the color corrected image based on a color difference value in the target region in the input image.

6. The image processing device according to claim 1,

wherein the object detection unit respectively detects the main sign and the auxiliary sign by using preset reference images.

7. The image processing device according to claim 1,

wherein the object relation analysis unit associates the main sign and the auxiliary sign with each other in a case where the auxiliary sign is present in a region set in an arbitrary predetermined range in the main sign or in a case where the main sign is present in a region set in an arbitrary predetermined range in the auxiliary sign in accordance with a preset positional relationship between the main sign and the auxiliary sign.

8. An alarming apparatus comprising:

the image processing device according to claim 1; and

an alarm output unit that outputs an alarm to a driver of the vehicle based on a result of recognizing the traffic sign by the image processing device.

9. An image processing system comprising:

a terminal device that is installed on a vehicle; and

a server device that performs communication with the terminal device,

wherein the terminal device includes

a contrasting correction unit that creates a contrasting corrected image by correcting contrasting an input image captured by a vehicle camera installed on a vehicle,

a color analysis unit that creates a color corrected image by correcting colors in the input image,

an object detection unit that detects a main sign included in the input image based on the contrasting corrected image and detects an auxiliary sign included in the input image based on the color corrected image, and

an image transmission unit that transmits a post-detection image indicating a result of detecting the main sign and the auxiliary sign by the object detection unit to the server device, and

wherein the server device includes

an image receiving unit that receives the post-detection image transmitted from the terminal device,

a second contrasting correction unit that creates a contrasting corrected image by correcting contrasting of the post-detection image received by the image receiving unit,

a second color analysis unit that creates a color corrected image by correcting colors of the post-detection image received by the image receiving unit,

a second object detection unit that detects a main sign included in the post-detection image based on the contrasting corrected image and detects an auxiliary sign included in the post-detection image based on the color corrected image, and

an object relation analysis unit that recognizes a traffic sign as a combination of the main sign and the auxiliary sign by associating the main sign and the auxiliary sign with each other based on a positional relationship between the main sign and the auxiliary sign in the post-detection image received by the image receiving unit.

10. The image processing system according to claim 9,

wherein the server device further includes a second image transmission unit that transmits a post-recognition image related to the traffic sign recognized by the object relation analysis unit to the terminal device, and

wherein the terminal device further includes

a second image receiving unit that receives the post-recognition image transmitted from the server device, and

a notification unit that provides information related to the traffic sign to a driver of the vehicle based on the post-recognition image received by the second image receiving unit.

11. The image processing system according to claim 9,

wherein the server device further includes an alarm information transmission unit that transmits alarm information based on a result of recognizing the traffic sign by the object relation analysis unit to the terminal device, and

wherein the terminal device further includes

an alarm information receiving unit that receives the alarm information transmitted from the server device, and

an alarm output unit that outputs an alarm to the driver of the vehicle based on the alarm information received by the alarm information receiving unit.

12. The image processing system according to claim 9,

wherein the server device further includes a navigation information transmission unit that transmits navigation information for navigating the vehicle to a predetermined destination to the terminal device based on a result of recognizing the traffic sign by the object relation analysis unit, and

wherein the terminal device further includes

a navigation information receiving unit that receives the navigation information transmitted from the server device, and

a vehicle navigation unit that navigates the vehicle to the destination based on the navigation information received by the navigation information receiving unit.

13. An image processing method for processing an input image captured by a vehicle camera installed on a vehicle and input to a computer, the method comprising causing the computer to:

create a contrasting corrected image by correcting contrasting of the input image;

create a color corrected image by correcting colors of the input image;

detect a main sign included in the input image based on the contrasting corrected image;

detect an auxiliary sign included in the input image based on the color corrected image; and

recognize a traffic sign as a combination of the main sign and the auxiliary sign by associating the main sign and the auxiliary sign with each other based on a positional relationship between the detected main sign and the auxiliary sign in the input image.