Moving object detecting device and mobile robot

• US 8,218,816 B2
• Filed: 08/22/2008
• Issued: 07/10/2012
• Est. Priority Date: 08/24/2007
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Sonoura, Takafumi
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US12/196,376
• Publication Number:
  US 20090052740A1
• Time in Patent Office: 1,418 Days
• Field of Search: 382/103, 382/104, 382/106, 382/153, 348/135, 348/169, 700/245, 700253-255, 700/259
• US Class Current: 382/103
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A moving object detecting device measures a congestion degree of a space and utilizes the congestion degree for tracking. In performing the tracking, a direction measured by a laser range sensor is heavily weighted when the congestion degree is low. When the congestion degree is high, a sensor fusion is performed by heavily weighting a direction measured by a image processing on a captured image to obtain a moving object estimating direction, and obtains a distance by the laser range sensor in the moving object estimating direction.

Claims

1. A moving object detecting device comprising:

a stereo camera apparatus that takes a set of images of an object and determines a parallax between the object and the stereo camera apparatus;

a range sensor that scans the object and utilizes the parallax to measure a distance from the device to an object;

an image processing component that performs an image processing on the images captured by the stereo camera apparatus to extract a tracking target, which is a moving object to be tracked, from the image and to estimate a distance and a direction from the device to the tracking target based on the image;

a congestion degree measuring component that sets a congestion degree measuring region around a moving object estimated point where a moving object is previously detected and measures a congestion degree represented by a ratio C of the number of detection points detected by the range sensor in the congestion degree measuring region to a total number of scanning lines of the range sensor passing through within the congestion degree measuring region; and

a calculation component that obtains a fusion estimation result by performing a calculation using (a) a first moving object tracking result obtained by an output from the range sensor, (b) a second moving object tracking result obtained by the image processing by the image processing component, and (c) a fusion ratio W₁ that is represented by the following expression (1) where α

, β

, and n are arbitrary constants;

(1) W₁=α

Cⁿ+β

.

2. The device according to claim 1,

wherein the calculation component obtains a distance measured by the range sensor in the fusion estimating direction as a fusion estimating distance.

3. The device according to claim 1, wherein the first tracking algorithm includes:

setting a moving object detection region around the moving object estimated point obtained in a previous detection step; and

setting detection points obtained by the range sensor to be within the moving object detection region as estimating position candidate points representing a position of the moving object in a current detection step.

4. The device according to claim 3, wherein the first tracking algorithm includes:

setting a region obtained by expanding the moving object detection region by a given distance L outward in a normal direction as the congestion degree measuring region; and

measuring the congestion degree utilizing remaining scanning lines other than scanning lines that are detecting any of the estimating position candidate points having a presence probability exceeding a given threshold value δ

, which satisfies 0≦

δ

≦

1, the threshold value δ

being determined in accordance with the position, from among the estimating position candidate points that are within the congestion degree measuring region.

5. The device according to claim 3, wherein, for a case where no estimating position candidate points are detected within the moving object detection region, the first tracking algorithm includes:

setting a prediction position where the moving object is predicted from the previous moving object estimated point as a provisional current estimation position of the tracking target;

adding a lost counter for recording non-detection of the estimating position candidate points; and

continuously performing the tracking of the target object.

6. The device according to claim 5, wherein the first tracking algorithm includes stopping a follow-up operation and transitioning to other operation mode when determined that the tracking target is lost in a case where no estimating position candidate points are detected within the moving object detection region.

7. A mobile robot comprising:

the moving object detecting device comprising;

a stereo camera apparatus that takes a set of images of an object and determines a parallax between the object and the stereo camera apparatus;

a range sensor that scans the object and utilizes the parallax to measure a distance from the device to an object;

an image processing component that performs an image processing on the images captured by the stereo camera apparatus to extract a tracking target, which is a moving object to be tracked, from the image and to estimate a distance and a direction from the device to the tracking target based on the image;

a congestion degree measuring component that sets a congestion degree measuring region around a moving object estimated point where a moving object is previously detected and measures a congestion degree represented by a ratio C of the number of detection points detected by the range sensor in the congestion degree measuring region to a total number of scanning lines of the range sensor passing through within the congestion degree measuring region; and

a calculation component that obtains a fusion estimation result by performing a calculation using (a) a first moving object tracking result obtained by an output from the range sensor, (b) a second moving object tracking result obtained by the image processing by the image processing component, and (c) a fusion ratio W₁ that is represented by the following expression (1) where α

, β

, and n are arbitrary constants;

(1) W₁=α

Cⁿ+β

;

a drive component that drives the robot to move on a floor; and

a controller that controls the drive unit to move the robot to follow the tracking target by referring to a tracking target estimating position that is obtained as the fusion estimation result by the calculation unit of the moving object detecting device,

wherein the calculation component dynamically changes a follow-up gain G for tracking the tracking target to satisfy the following equation (2), where R (0<

R<

1) represents a reflection intensity of a scanning beam of the range sensor, and second fusion ration W₂ satisfies the following equation (3), where α

₂, β

₂, and n₂ are arbitrary constants;

G∝

W₁W₂ 

 

(2)

W₂=α

₂Rⁿ²+β

₂. 

 

(3)