Device and method for detection of moving obstacles

• US 5,777,690 A
• Filed: 01/19/1996
• Issued: 07/07/1998
• Est. Priority Date: 01/20/1995
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Watanabe, Mutsumi, Takeda, Nobuyuki
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): DIEP, NHON THANH
• Application Number:
  US08/588,845
• Time in Patent Office: 900 Days
• Field of Search: 348/61, 348/113, 348/118, 348/119, 348/143, 348/148, 348/169, 348/699, 348/700, 348/154, 348/155, 348/150, 348/151, 348/152, 382/103, 382/107, 382/236
• US Class Current: 348/699
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A moving obstacle detecting device detects moving objects, may be applied in the field of autonomous vehicles, intelligent robots, video monitoring system and similar systems. The device includes an optical flow extracting unit 1 for extracting optical flows for the respective local regions in the measured images, an focus of expansion (F.O.E.) calculating unit for calculating an F.O.E. of a straight line extended by the extracted optical flows, and a moving obstacle detecting unit for analyzing a temporal change of the calculated F.O.E. to judge the presence of the moving obstacle when the temporal positional change is larger than a predetermined variation quantity.

Claims

1. A moving object detecting device comprising:

an image taking means for taking in a plurality of images sequentially picked up while moving,

a motion vector extracting means for extracting a plurality of motion vectors representing a position change of the object between the images taken by said image taking means,

a region setting means for setting a plurality of small regions within the respective images,

an intersection estimating means for extracting a plurality of motion vectors designating the small regions set by said region setting means from said motion vector extracting means and extending the respective motion vectors and for sequentially estimating intersections within the respective small regions which are intersections of the extended motion vectors,

a small region estimation residual calculating means for calculating residual of estimation in the respective small regions in view of the respective intersections within the small regions sequentially estimated by said intersection estimating means, and

a moving object detecting means for detecting a moving object picked up within the image based on the residual of estimation calculated by said small region estimation residual calculating means.

7. A moving object detecting device as set forth in any of claims 1, 2, or 3, wherein said moving object detecting means determines that there exists a moving object when residual of estimation or total residual of estimation is larger than a threshold value.

2. A moving object detecting device comprising:

an image taking means for taking in a plurality of images of a background region containing a moving object which are sequentially picked up while moving,

a motion vector extracting means for extracting a plurality of motion vectors representing a position change of the object between the images taken by said image taking means,

a motion vector selecting means for selecting a plurality of background region motion vectors each representing a position change of the background region other than said moving object already known from the motion vectors extracted by the motion vector extracting means,

a rotation movement estimating means for estimating a rotation component of a motion of the image taking means based on the background region motion vectors selected by said motion vector selecting means,

a motion vector correcting means for deleting the rotation component of the rotary motion of said image taking means based on the motion vectors extracted by said motion vector extracting means employing the rotation component estimated by said rotation movement estimating means,

a region setting means for setting a plurality of small regions within the respective images,

an intersection estimating means for extracting a plurality of motion vectors designating the small regions set by said region setting means from said motion vector correcting means and extending the respective motion vectors and for sequentially estimating intersections within the respective small regions which are intersections of the extended motion vectors,

a small region estimation residual calculating means for calculating residual of estimation in the respective small regions in view of the respective intersections within the small regions sequentially estimated by said intersection estimating means, and

a moving object detecting means for detecting a moving object picked up within the image based on the residual of estimation calculated by said small region estimation residual calculating means.

3. A moving object detecting device comprising

an image taking means for taking in a plurality of images sequentially picked up while moving,

a motion vector extracting means for extracting a plurality of motion vectors representing a position change of the object between the images taken by said image taking means,

a region setting means for setting a plurality of small regions within the respective images,

an intersection estimating means for acquiring a plurality of motion vectors designating the small regions set by said region setting means from said motion vector extracting means and extending the respective motion vectors and for sequentially estimating intersections within the respective small regions which are intersections of the extended motion vectors,

a small region estimation residual calculating means for calculating residual of estimation in the respective small regions in view of the respective intersections within the small regions sequentially estimated by said intersection estimating means,

a total estimation residual calculating means for integrating residual of estimation of a plurality in the small regions of the same image calculated by said small region estimation residual calculating means to calculate all or a part of total residual of estimation on a plane of said image, and

a moving object detecting means for detecting a moving object picked up within the image based on the distribution of the total residual of estimation in the images calculated by said total estimation residual calculating means.

4. A moving object detecting method comprising the steps of:

taking in a plurality of images sequentially picked up while moving,

extracting a plurality of motion vectors representing a position change of an object between the taken images,

setting a plurality of small regions within said respective images,

selecting a plurality of motion vectors designating the set small regions from said extracted motion vectors and extending said respective motion vectors and sequentially estimating intersections within said respective small regions which are intersections of the extended motion vectors,

calculating residual of estimation in the respective small regions in view of the respective intersections within said sequentially estimated small regions, and

detecting a moving object picked up within the image based on the calculated residual of estimation.

8. A moving object detecting method as set forth in any of claims 4, 5, or 6, which further includes a step of determining that a moving object exists when a calculated residual of estimation or calculated total residual of estimation is larger than a threshold value.

5. A moving object detecting method comprising the steps of:

taking in a plurality of images of a background region containing a moving object which are sequentially picked up while moving,

extracting a plurality of motion vectors representing a position change of an object between the taken images,

selecting a plurality of background region motion vectors each representing a position change of the background region other than said moving obstacle already known from the extracted motion vectors,

estimating a rotation component of a motion when said image is taken in from the selected background region motion vectors,

derotating the extracted motion vectors employing said estimated rotation component,

setting a plurality of small regions within said respective images,

extracting a plurality of motion vectors designating said set small regions from the derotated motion vectors and extending the respective motion vectors and sequentially estimating intersections within the respective small regions which are intersections of the extended motion vectors,

calculating residual of estimation in the respective small regions in view of the respective intersections within said sequentially estimated small regions, and

detecting a moving object picked up within the image based on the calculated residual of estimation.

6. A moving object detecting method comprising the steps of:

taking in a plurality of images sequentially picked up while moving,

extracting a plurality of motion vectors representing a position change of an object between the taken images,

setting a plurality of small regions within said respective images,

acquiring a plurality of motion vectors designating said set small regions from said extracted motion vectors and extending the respective motion vectors and sequentially estimating intersections within the respective small regions which are intersections of the extended motion vectors,

calculating residual of estimation in the respective small regions in view of the respective sequentially estimated intersections within the small regions,

integrating the calculated residual of estimation of a plurality of the small regions of the same image to calculate all or a part of total residual of estimation on a plane of said image, and

detecting a moving object picked up within the image based on the distribution of the calculated total residual of estimation in the images.