Object detection device, object detection method, and object detection system

• US 10,157,468 B2
• Filed: 04/30/2015
• Issued: 12/18/2018
• Est. Priority Date: 05/07/2014
• Status: Active Grant

• Current Assignee: FlowBiz Research Inc., National University Corporation Hokkaido University, NEC Corporation
• Original Assignee: FlowBiz Research Inc., National University Corporation Hokkaido University, NEC Corporation
• Inventors: Kumeno, Hiroyuki, Murai, Yuichi, Takeda, Yasushi
• Primary Examiner(s): Williams, Jeffery A
• Application Number:
  US15/308,753
• Publication Number:
  US 20170186175A1
• Time in Patent Office: 1,328 Days
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/752 Contour matching
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An object detection device of an object detection system includes an object detection determination unit including a bird candidate determination unit for determining, from an image acquired by imaging a moving object, whether or not the moving object is a specific detection subject (bird in this case), and a bird type/attitude estimation unit for estimating, based on a contour of the moving object determined to be the bird by the bird candidate determination unit and a WFM library storing contour shape information, which is a contour prepared for each of types of the specific detection subject (types of the bird in this case), a type of the moving object out of the types of the specific detection subject.

Claims

1. An object detection device, comprising:

a primary determination unit comprising processing circuitry configured to determine, from an image acquired by imaging a moving object, whether or not the moving object is a specific detection subject;

an object detection determination unit comprising processing circuitry configured to estimate, based on a contour of the moving object determined to be the specific detection subject by the primary determination unit and contour shape information, which is a contour prepared for each of types of the specific detection subject, a type of the moving object out of the types of the specific detection subject;

the specific detection subject comprises a bird;

the object detection device further comprises a bird image candidate extraction unit comprising processing circuitry configured to extract a contour of the bird from the image;

the object detection determination unit comprises a bird detection determination unit comprising processing circuitry configured to estimate a type of the bird from the contour;

the bird image candidate extraction unit is configured to store information including a contour line of the bird extracted from the image as moving object information; and

the bird detection determination unit comprises;

a wireframe model library storing information on a wireframe of simulating a standard body shape and a wing flap for each of the types of the bird and plane projection contour shape information when rotational angles about three axes and a wing flap angle are each changed at certain rates for each of the types of the bird; and

a bird type/attitude estimation unit comprising processing circuitry configured to compare the moving object information with the wireframe model library to estimate the type and an attitude of the bird.

2. An object detection device according to claim 1, wherein:

the moving object information and the plane projection contour shape information include a vector having a distance from a reference point to the contour as a parameter; and

the bird detection determination unit is configured to estimate the moving object as a corresponding bird type when a cross-correlation of the vector is more than a threshold set for each of the bird types.

3. An object detection device according to claim 1, wherein the bird type/attitude estimation unit is configured to employ, when the moving object information and the wireframe model library are compared with each other, general features relating to an image pickup device for imaging the moving object and the moving object as prior information, and to compare, based on the prior information, the plane projection contour shape information with the moving object information in descending order of a probability of correlation with the contour of the moving object, to thereby estimate the type of the bird.

4. An object detection device according to claim 1, further comprising a flight path estimation unit comprising processing circuitry configured to estimate a flight path of the bird from the type and the attitude of the bird estimated by the bird type/attitude estimation unit.

5. An object detection device according to claim 4, further comprising a bird detection final determination unit comprising processing circuitry configured to confirm, from the type and the attitude of the bird estimated by the bird type/attitude estimation unit and the flight path estimated by the flight path estimation unit, consistency of the attitude and a flight direction of the moving object, and consistency of a flight speed and a wing flap frequency unique to the bird type, to thereby finally determine whether or not the moving object is a bird.

6. An object detection device according to claim 4, further comprising an alarm notification unit comprising processing circuitry configured to determine, from the flight path, whether or not likelihood that the bird intrudes into a region set in advance exists, and generating an alarm when the likelihood exists.

7. A program stored on a non-transitory computer-readable medium for causing a computer to operate as an object detection device, the object detection device comprising:

a primary determination unit comprising processing circuitry configured to determine, from an image acquired by imaging a moving object, whether or not the moving object is a specific detection subject;

an object detection determination unit comprising processing circuitry configured to estimate, based on a contour of the moving object determined to be the specific detection subject by the primary determination unit and contour shape information, which is a contour prepared for each of types of the specific detection subject, a type of the moving object out of the types of the specific detection subject;

the specific detection subject comprises a bird;

the object detection device further comprises a bird image candidate extraction unit comprising processing circuitry configured to extract a contour of the bird from the image;

the object detection determination unit comprises a bird detection determination unit comprising processing circuitry configured to estimate a type of the bird from the contour;

the bird image candidate extraction unit is configured to store information including a contour line of the bird extracted from the image as moving object information; and

the bird detection determination unit comprises;

a wireframe model library storing information on a wireframe of simulating a standard body shape and a wing flap for each of the types of the bird and plane projection contour shape information when rotational angles about three axes and a wing flap angle are each changed at certain rates for each of the types of the bird; and

a bird type/attitude estimation unit comprising processing circuitry configured to compare the moving object information with the wireframe model library to estimate the type and an attitude of the bird.

8. An object detection method, comprising:

(a) imaging a moving object to acquire an image;

(b) primarily determining, from the image, whether or not the moving object is a specific detection subject;

(c) estimating, based on a contour of the moving object determined to be the specific detection subject by the primary determination and contour shape information, which is a contour prepared for each of types of the specific detection subject, a type of the moving object out of the types of the specific detection subject, wherein the specific detection object comprises a bird

(d) extracting a contour of the bird from the image;

(e) estimating a type of the bird from the contour;

(f) storing information including a contour line of the bird extracted from the image as moving object information;

(g) storing information on a wireframe of simulating a standard body shape and sing flap for each of the types of the bird and plane projection contour shape information when rotational angles about three axes and a wing flap angle are each changed at certain rates for each of the types of the bird; and

(h) comparing the moving object information with the wireframe model library to estimate the type and an attitude of the bird.

9. An object detection system, comprising:

an image pickup device for imaging a moving object to acquire an image;

a primary determination unit comprising processing circuitry configured to determine determining, from the image, whether or not the moving object is a specific detection subject; and

an object detection determination unit comprising processing circuitry configured to estimate, based on a contour of the moving object determined to be the specific detection subject by the primary determination unit and contour shape information, which is a contour prepared for each of types of the specific detection subject, a type of the moving object out of the types of the specific detection subject;

the specific detection subject comprises a bird;

the object detection device further comprises a bird image candidate extraction unit comprising processing circuitry configured to extract a contour of the bird from the image;

the object detection determination unit comprises a bird detection determination unit comprising processing circuitry configured to estimate a type of the bird from the contour;

the bird image candidate extraction unit is configured to store information including a contour line of the bird extracted from the image as moving object information; and

the bird detection determination unit comprises;

a wireframe model library storing information on a wireframe of simulating a standard body shape and a wing flap for each of the types of the bird and plane projection contour shape information when rotational angles about three axes and a wing flap angle are each changed at certain rates for each of the types of the bird; and

a bird type/attitude estimation unit comprising processing circuitry configured to compare the moving object information with the wireframe model library to estimate the type and an attitude of the bird.

10. An object detection system according to claim 9, wherein the image pickup device comprises a single monitoring camera.