Vehicle peripheral obstacle notification system

• US 9,463,741 B2
• Filed: 03/19/2015
• Issued: 10/11/2016
• Est. Priority Date: 04/25/2014
• Status: Active Grant

• Current Assignee: Hitachi Construction Machinery Company Limited (Hitachi, Ltd.)
• Original Assignee: Hitachi Construction Machinery Company Limited (Hitachi, Ltd.)
• Inventors: Oota, Moritaka, Kawamata, Yukihiro, Fukuda, Yoshibumi, Onuma, Chieko, Ishimoto, Hidefumi
• Primary Examiner(s): PARK, SOO JIN
• Application Number:
  US14/662,746
• Publication Number:
  US 20150307024A1
• Time in Patent Office: 572 Days
• US Class Current: 1/1
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: E02F 9/261 Surveying the work-site to be treated
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vehicle peripheral obstacle notification system, used on a construction machine, detects a moving obstacle, gives composite display of the detected moving obstacle, and notifies a user thereof, the moving obstacle being at a location not appearing in a composite bird'"'"'s-eye view obtained by onboard cameras acquiring images of the surroundings of the vehicle including an image immediately under the vehicle body. The system has a composite image formation part that extracts composite image formation areas from the surrounding images to compose a composite bird'"'"'s-eye image. A moving obstacle is detected using the surrounding images and it is decided whether the detected position of the moving obstacle is inside or outside the composite image formation areas. If the detected position is determined to be outside the composite image formation areas, information about the detected position is extracted and an image thereof is output for superimposition onto the composite bird'"'"'s-eye image.

Claims

1. A vehicle peripheral obstacle notification system comprising:

a surrounding image input part that includes a plurality of cameras mounted on a vehicle and configured to acquire images of the surroundings of a vehicle;

a composite image formation part that extracts a part of each of the surrounding images outputted from the surrounding image input part as composite image formation areas and composes a composite bird'"'"'s-eye image from the extracted composite image formation areas; and

an output part that presents a user with the composite bird'"'"'s-eye image that the composite image formation part outputted,

wherein the vehicle peripheral obstacle notification system further comprises;

an object detection part that detects whether a moving obstacle exists in the surrounding images;

said composite image formation part calculating the coordinate information of the moving obstacle on the composite bird'"'"'s-eye image based on the detection result of the object detection part;

an area determination part that, when the object detection part detects the moving obstacle, determines whether the position at which the moving obstacle is detected, on the basis of the coordinate information of the moving obstacle, is inside the composite image formation areas or outside the composite image formation areas and inside the surrounding images; and

an image information extraction part which, if the area determination part determines that the detected position of the moving obstacle is outside the composite image formation areas and inside the surrounding images, extracts information about the detected position of the moving obstacle and extracts an image of the detected moving obstacle,

wherein the composite image formation part superimposes the image of the moving obstacle on the detection position of the moving obstacle to the composite bird'"'"'s-eye image based on the information about the detected position of the moving obstacle and the image of the moving obstacle output from the image information extraction part, and outputs the composite bird'"'"'s eye image having the image of the moving obstacle superimposed thereon to the output part.

2. The vehicle peripheral obstacle notification system according to claim 1, wherein the composite image formation part highlights the detected position of the moving obstacle and displays the image of the moving obstacle at a position different from the detected position of the moving obstacle based on the information about the detected position of the moving obstacle and the image of the moving obstacle output from the image information extraction part.

3. The vehicle peripheral obstacle notification system according to claim 1, wherein, if the area determination part determines that the detected position of the moving obstacle is inside the composite image formation areas, the image information extraction part extracts the information about the detected position of the moving obstacle and extracts the image of the detected moving obstacle for output to the composite image formation part for superimposition onto the composite bird'"'"'s-eye image.

4. The vehicle peripheral obstacle notification system according to claim 1, wherein, if the area determination part determines that the detected position of the moving obstacle is outside the composite image formation areas and inside the surrounding images,

the composite image formation part provides a translucent display of the areas other than those indicated by the composite bird'"'"'s-eye image and highlights the detected position of the moving obstacle when composing the surrounding images.

5. The vehicle peripheral obstacle notification system according to claim 1,

wherein if the area determination part determines that the detected position of the moving obstacle has changed from inside the composite image formation areas to a detected position that is outside the composite image formation areas and inside the surrounding images, the image information extraction part retains the image of the moving obstacle acquired inside the composite image formation areas, and

wherein, while the detected position of the moving obstacle remains outside the composite image formation areas and inside the surrounding images, the image information extraction part outputs, the information about the detected position of the moving obstacle and the retained image of the moving obstacle acquired inside the composite image formation areas, to the composite image formation part for superimposition onto the composite bird'"'"'s-eye image.