Image processing device, alarming apparatus, image processing system, and image processing method

• US 10,303,960 B2
• Filed: 01/18/2017
• Issued: 05/28/2019
• Est. Priority Date: 02/16/2016
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Hattori, Hideharu, Izumi, Yoshifumi
• Primary Examiner(s): Rudolph, Vincent
• Assistant Examiner(s): Patel, Pinalben
• Application Number:
  US15/409,325
• Publication Number:
  US 20170236015A1
• Time in Patent Office: 860 Days
• CPC Class Codes: B60Q 5/005 automatically actuated
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence

Abstract

An image processing device includes a contrasting correction unit, a color analysis unit, an object detection unit, and an object relation analysis unit. The contrasting correction unit creates a contrasting corrected image by correcting contrasting of an input image captured by a vehicle camera. The color analysis unit creates a color corrected image by correcting colors of the input image. The object detection unit detects a main sign included in the input image based on the contrasting corrected image and detects an auxiliary sign included in the input image based on the color corrected image. The object relation analysis unit recognizes a traffic sign as a combination of a main sign and auxiliary sign by associating the main sign and the auxiliary sign with each other based on a positional relationship between the main and auxiliary signs, which are detected by the object detection unit, in the input image.

Claims

1. An image processing system comprising:

a terminal device that is installed on a vehicle; and

a server device that performs communication with the terminal device,

wherein the terminal device includes

a contrasting correction unit that creates a contrasting corrected image by correcting contrasting an input image captured by a vehicle camera installed on a vehicle,

a color analysis unit that creates a color corrected image by correcting colors in the input image,

an object detection unit that detects a main sign included in the input image based on the contrasting corrected image and detects an auxiliary sign included in the input image based on the color corrected image, and

an image transmission unit that transmits a post-detection image indicating a result of detecting the main sign and the auxiliary sign by the object detection unit to the server device, and

wherein the server device includes

an image receiving unit that receives the post-detection image transmitted from the terminal device,

a second contrasting correction unit that creates a contrasting corrected image by correcting contrasting of the post-detection image received by the image receiving unit,

a second color analysis unit that creates a color corrected image by correcting colors of the post-detection image received by the image receiving unit,

a second object detection unit that detects a main sign included in the post-detection image based on the contrasting corrected image and detects an auxiliary sign included in the post-detection image based on the color corrected image, and

an object relation analysis unit that recognizes a traffic sign as a combination of the main sign and the auxiliary sign by associating the main sign and the auxiliary sign with each other based on a positional relationship between the main sign and the auxiliary sign in the post-detection image received by the image receiving unit.

2. The image processing system according to claim 1,

wherein the server device further includes a second image transmission unit that transmits a post-recognition image related to the traffic sign recognized by the object relation analysis unit to the terminal device, and

wherein the terminal device further includes

a second image receiving unit that receives the post-recognition image transmitted from the server device, and

a notification unit that provides information related to the traffic sign to a driver of the vehicle based on the post-recognition image received by the second image receiving unit.

3. The image processing system according to claim 1,

wherein the server device further includes an alarm information transmission unit that transmits alarm information based on a result of recognizing the traffic sign by the object relation analysis unit to the terminal device, and

wherein the terminal device further includes

an alarm information receiving unit that receives the alarm information transmitted from the server device, and

an alarm output unit that outputs an alarm to the driver of the vehicle based on the alarm information received by the alarm information receiving unit.

4. The image processing system according to claim 1,

wherein the server device further includes a navigation information transmission unit that transmits navigation information for navigating the vehicle to a predetermined destination to the terminal device based on a result of recognizing the traffic sign by the object relation analysis unit, and

wherein the terminal device further includes

a navigation information receiving unit that receives the navigation information transmitted from the server device, and

a vehicle navigation unit that navigates the vehicle to the destination based on the navigation information received by the navigation information receiving unit.