Excavation device using stereo camera and angle sensors to position bucket

• US 10,118,553 B2
• Filed: 04/24/2014
• Issued: 11/06/2018
• Est. Priority Date: 04/23/2014
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Matsuo, Shigeru, Hasejima, Noriyasu, Kurata, Kenichirou
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US15/303,937
• Publication Number:
  US 20170028922A1
• Time in Patent Office: 1,657 Days
• CPC Class Codes: B60R 1/002 specially adapted for covering the peripheral part of the vehicle, e.g. for viewing tyres, bumpers or the like integrated in the windows B60R1/001; combined with rear-view mirrors B60R1/08
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: E02F 9/205 Remotely operated machines, e.g. unmanned vehicles E02F3/8866 takes precedence
• CPC Class Codes: E02F 9/261 Surveying the work-site to be treated
• CPC Class Codes: E02F 9/262 with follow-up actions to control the work tool, e.g. controller
• CPC Class Codes: E02F 9/264 Sensors and their calibration for indicating the position of the work tool
• CPC Class Codes: E02F 9/265 with follow-up actions (e.g. control signals sent to actuate the work tool)
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Excavation is performed using an excavation device so that positions of a bucket and of an excavated material are measured with high accuracy. Safe excavation work is realized by having the excavation device include a stereo camera, and by recognizing a bucket from an image photographed by the camera, measuring a position of the bucket, recognizing an excavated material to be separated from the ground from the image photographed by the camera, measuring a position of the excavated material, and measuring a positional relationship between the bucket and the excavated material in the same screen.

Claims

1. An excavation device comprising:

an upper swing body supported by a base,

a boom interconnected with the upper swing body,

a bucket configured to perform excavation,

an arm interconnecting the bucket and the boom such that the bucket is vertically movable by the arm and the boom,

angle sensors to measure a position of the bucket, and

an external recognition device located between the arm and the upper swing body, the external recognition device including;

a stereo camera;

a means for recognizing the bucket from an image photographed by the stereo camera and measuring a position of the bucket;

a means for recognizing an excavated material from the image photographed by the stereo camera and measuring a position of the excavated material;

a means for measuring a positional relationship between the bucket and the excavated material in an identical screen; and

an angle sensor measurement unit receiving input from the angle sensors, wherein

when the bucket is present outside the photographing range of the stereo camera, position information measured using a sensor other than the stereo camera is used as the position of the bucket, and the bucket is moved using the input from the angle sensors to a position at which the bucket can be photographed, and

when the bucket is present inside the photographing range of the stereo camera, position information measured by the stereo camera is used as the position of the bucket, and the bucket is moved using the input from the angle sensors.

2. The excavation device according to claim 1, wherein

the position of the excavated material is set to a point which exceeds a predetermined length from the upper swing body of the excavation device.

3. The excavation device according to claim 1, wherein

the means for measuring the positional relationship between the bucket and the excavated material recognizes the excavated material to be separated from a ground from the image photographed by the stereo camera.

4. The excavation device according to claim 1, wherein

an edge image of the excavated material is created, a boundary between rocks as the excavated material is extracted from the edge image, and the position of the excavated material is selected among points at the boundary.

5. The excavation device according to claim 1, wherein

the positional relationship between the bucket and the excavated material, which is measured by the stereo camera, is superimposed on the image photographed by the stereo camera and displayed.

6. The excavation device according to claim 1, wherein

the positional relationship between the bucket and the excavated material includes any one or both of the position of the excavated material and a length between the bucket and the excavated material.