MOTION DETECTION METHOD, APPARATUS AND SYSTEM

• US 20110044502A1
• Filed: 12/02/2009
• Published: 02/24/2011
• Est. Priority Date: 04/28/2009
• Status: Active Grant

• Current Assignee: Hisense Co., Ltd. (Qingdao Huatong State-Owned Capital Operation Company Limited)
• Original Assignee: Hisense State Key Laboratory of Digital Multi-Media Technology Co., Ltd.
• Inventors: Hu, Shuo, Liu, Wei, Liu, Shao
• Granted Patent: US 8,385,595 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs

Abstract

A motion detection method, apparatus and system are disclosed in the present invention, which relates to the video image processing field. The present invention can effectively overcome the influence of the background on motion detection and the problem of object “conglutination” to avoid false detection, thereby accomplishing object detection in complex scenes with a high precision. The motion detection method disclosed in embodiments of the present invention comprises: acquiring detection information of the background scene and detection information of the current scene, wherein the current scene is a scene comprising an object(s) to be detected and the same background scene; and calculating the object(s) to be detected according to the detection information of the background scene and the detection information of the current scene. The present invention is applicable to any scenes where moving objects need to be detected, e.g., automatic passenger flow statistical systems in railway, metro and bus sectors, and is particularly applicable to detection and calibration of objects in places where brightness varies greatly.

Claims

1. A motion detection method, comprising:

acquiring detection information of a background scene and detection information of a current scene, wherein the current scene is a scene comprising an object(s) to be detected and the same background scene;

calculating the object(s) to be detected according to the detection information of the background scene and the detection information of the current scene.

2. The motion detection method of claim 1, wherein:

calculating the object(s) to be detected is accomplished by subtracting the detection information of the background scene from the detection information of the current scene;

or

setting a first weight value, a second weight value and a compensation factor, subtracting a product of the detection information of the background scene and the second weight value from a product of the detection information of the current scene and the first weight value to obtain initial object detection information, and calculating the object(s) to be detected according to the initial object detection information and the compensation factor.

3. The motion detection method of claim 2, wherein the detection information is a disparity image, and the method further comprises:

acquiring a first image and a second image of the background scene and/or the current scene;

extracting key points from the first image and the corresponding second image respectively to obtain first image key points and second image key points;

performing stereo matching on the first image key points and the second image key points to obtain a matching point of each of the first image key points in the second image key points; and

calculating a disparity of each of the first image key points and acquiring a disparity image of the background scene and/or the current scene according to the disparity.

4. The motion detection method of claim 3, wherein performing stereo matching on the first image key points and the second image key points to obtain a matching point of each of the first image key point in the second image key points comprises:

acquiring key points within a predetermined range that starts from a b^th column in an a^th row of the second image towards the first image, wherein a and b represent a row coordinate and a column coordinate of each of the key points in the first image respectively;

calculating matching values of the first image key point (a, b) and the key points in the second image corresponding to the first image key point (a, b); and

determining a matching point of the first image key point (a, b) in the second image according to the matching values.

5. The motion detection method of claim 3, wherein calculating a disparity of each of the first image key points and acquiring a disparity image according to the disparity comprises:

taking a scanning point next to the first image key point as a reference point according to a scanning sequence, wherein the key point and the reference point have row coordinates and column coordinates of (a, b) and (a, d) respectively in the first image;

acquiring a matching point of the reference point within a search range of the second image, wherein the search range is composed of a (b-DIF)^th column to a d^th column in the a^th row, and DIF represents a disparity of the first image key point; and

calculating a disparity between the reference point and the matching point of the reference point and taking the reference point as a key point.

6. The motion detection method of claim 3, wherein calculating a disparity of each of the first image key points and acquiring a disparity image according to the disparity comprises:

selecting a neighboring key point (o, p) corresponding to a non-key point (m, n) of the first image;

acquiring a matching point of the non-key point from a second search range of the second image, wherein the second search range is composed of a (n-DIF)^th column to a p^th column of a m^th row, and DIF represents a disparity of the neighboring key point in the first image; and

calculating a disparity between the non-key point and the matching point of the non-key point.

7. The motion detection method of claim 3, further comprising:

extracting image edge points from the first image and the second image respectively as the key points, wherein the first image and the second image are binocular images of the scene.

8. The motion detection method of claim 3, further comprising:

performing stereo matching between the first image key point and the second image key point based on the Census criterion; and

acquiring the disparity image according to the disparity of the first image key point and based on the normalized cross correlation criterion.

9. The motion detection method of claim 1, wherein the detection information is a depth image, and the method further comprises:

calculating a depth image(s) of the background scene and/or a depth image(s) of the current scene according to the disparity image(s) of the background scene and/or the disparity image(s) of the current scene acquired;

or

performing an edge detection on images of the background scene and/or the current scene captured at a same view point; and

calculating the depth image(s) of the background scene and/or the current scene according to the edge detection.

10. The motion detection method of claim 2, wherein obtaining the object to be detected further comprises:

determining an initial object(s) to be detected; and

determining a valid object(s) to be detected by removing false objects from the initial object(s) to be detected according to a falseness-removing strategy.

11. The motion detection method of claim 10, wherein the detection information is a disparity image or a depth image, and the method further comprises:

acquiring extreme points of a horizontal projection and a vertical projection respectively of the object disparity/depth image; and

pairing the extreme points of the horizontal projection and those of the vertical projection in one-to-one correspondence to determine the initial objects to be detected,

wherein the object disparity/depth image is calculated according to the detection information of the background scene and the detection information of the current scene.

12. The motion detection method of claim 10, wherein the falseness-removing strategy comprises:

determining whether an average disparity/depth value within a predetermined window of the object disparity/depth image centering around each of the initial objects to be detected is greater than a disparity/depth threshold;

if the answer is yes, then the initial object to be detected is a valid object, and otherwise, the initial object to be detected is a false object; and

for initial objects to be detected that haven'"'"'t been subjected to the falseness-removing process, acquiring an object objPoint[maxN] that has the greatest average disparity/depth value within a predetermined window of the object disparity/depth image centering around the initial object to be detected, calculating an Euclidean distance of each of the initial objects to be detected from the object objPoint[maxN];

if the Euclidean distance of the initial object to be detected from the object objPoint[maxN] is no smaller than a distance threshold, then the initial object to be detected is a valid object to be detected, and otherwise, the initial object to be detected is a false object, wherein maxN is a serial No. of the object that has the greatest average disparity/depth value; and

acquiring a minimum average gray-scale value of the initial objects to be detected within a predetermined window of the corresponding image of the current scene centering around the objects, and calculating a ratio of an average gray-scale value of each of the initial objects to be detected to the minimum average gray-scale value;

if the ratio corresponding to the initial object to be detected is no greater than a ratio threshold and the average gray-scale value corresponding to the initial object to be detected is no greater than a control threshold, then the initial object to be detected is a valid object to be detected, and otherwise, the initial object to be detected is a false object.

13. A motion detection apparatus, comprising:

a detection information acquisition unit, configured to acquire detection information of a background scene and detection information of a current scene, wherein the current scene is a scene comprising an object(s) to be detected and the same background scene;

an object detection unit, configured to calculate the object(s) to be detected according to the detection information of the background scene and the detection information of the current scene.

14. The motion detection apparatus of claim 13, wherein:

the object detection unit is further configured to calculate the object(s) to be detected by subtracting the detection information of the background scene from the detection information of the current scene;

or

the object detection unit is further configured to set a first weight value, a second weight value and a compensation factor, subtract a product of the detection information of the background scene and the second weight value from a product of the detection information of the current scene and the first weight value to obtain initial detection information of the object to be detected, and calculate the object(s) to be detected according to the initial detection information of the object to be detected and the compensation factor.

15. The motion detection apparatus of claim 13, wherein the detection information is a disparity image, and the detection information acquisition unit further comprises:

an image acquisition module, configured to acquire a first image and a second image of the background scene or the current scene;

a key point extracting module, configured to extract key points from the first image and the second image respectively to obtain first image key points and second image key points;

a key point matching module, configured to perform stereo matching on the first image key points and the second image key points to obtain a matching point of each of the first image key points in the second image key points; and

a disparity image acquisition module, configured to calculate a disparity of each of the first image key points and acquire a disparity image of the background scene or the current scene according to the disparity.

16. The motion detection apparatus of claim 13, wherein the detection information is a depth image, and the detection information acquisition unit comprises:

a first depth image calculation module, configured to calculate a depth image(s) of the background scene and/or the current scene according to the disparity image(s) of the background scene and/or the current scene acquired;

or

a second depth image calculation module, configured to perform an edge detection on images of the background scene and/or the current scene captured at a same view point, and calculate the depth image(s) of the background scene and/or the current scene according to the edge detection.

17. The motion detection apparatus of claim 15, wherein the key point matching module is further configured to acquire key points within a predetermined range that starts from a b^th column in an a^th row of the second image towards the first image, wherein a and b represent a row coordinate and a column coordinate of each of the key points in the first image respectively;

calculate matching values of the first image key point (a, b) and key points in the second image corresponding to the first image key point (a, b); and

determine a matching point of the first image key point (a, b) in the second image according to the matching values.

18. The motion detection apparatus of claim 15, wherein the disparity image acquisition module comprises:

a reference point disparity acquisition module, configured to take a scanning point next to the first image key point as a reference point according to a scanning sequence, wherein the key point and the reference point have row coordinates and column coordinates of (a, b) and (a, d) respectively in the first image;

acquire a matching point of the reference point within a search range of the second image, wherein the search range is composed of a (b-DIF)^th column to a d^th column in the a^th row, and DIF represents a disparity of the first image key point; and

calculate a disparity between the reference point and the matching point of the reference point and take the reference point as a key point; and

a non-key point disparity acquisition module, configured to select a neighboring key point (o, p) corresponding to a non-key point (m, n) of the first image;

acquire a matching point of the non-key point from a second search range of the second image, wherein the second search range is composed of a (n-DIF)^th column to a p^th column of a m^th row, and DIF represents a disparity of the neighboring key point in the first image; and

calculate a disparity between the non-key point and the matching point of the non-key point.

19. The motion detection apparatus of claim 13, wherein the detection information is a disparity image or a depth image, and the object detection unit comprises:

an initial object acquisition module, configured to, from the object disparity/depth image calculated according to the detection information of the background scene and the detection information of the current scene, determine an initial object(s) to be detected;

an object validity distinguishing module, configured to determine a valid object(s) to be detected by removing false objects from the initial objects to be detected according to a falseness-removing strategy;

wherein, the falseness-removing strategy comprises;

determining whether an average disparity/depth value within a predetermined window of the object disparity/depth image centering around each of the initial objects to be detected is greater than a disparity/depth threshold;

if the answer is yes, then the initial object to be detected is a valid object, and otherwise, the initial object to be detected is a false object; and

for initial objects to be detected that haven'"'"'t been subjected to the falseness-removing process, acquiring an object objPoint[maxN] that has the greatest average disparity/depth value within a predetermined window of the object disparity/depth image centering around the initial object to be detected, calculating an Euclidean distance of each of the initial objects to be detected from the object objPoint[maxN];

if the Euclidean distance of the initial object to be detected from the object objPoint[maxN] is no smaller than a distance threshold, then the initial object to be detected is a valid object to be detected, and otherwise, the initial object to be detected is a false object, wherein maxN is a serial No. of the object that has the greatest average disparity/depth value; and

acquiring a minimum average gray-scale value of the initial objects to be detected within a predetermined window of the corresponding image of the current scene centering around the objects, and calculating a ratio of an average gray-scale value of each of the initial objects to be detected to the minimum average gray-scale value;

if the ratio corresponding to the initial object to be detected is no greater than a ratio threshold and the average gray-scale value corresponding to the initial object to be detected is no greater than a control threshold, then the initial object to be detected is a valid object to be detected, and otherwise, the initial object to be detected is a false object.

20. A passenger flow detection system comprising a motion detection apparatus and a counter apparatus, the counter apparatus being configured to calculate a passenger flow volume according to objects to be detected that are acquired by the motion detection apparatus, wherein,

the motion detection apparatus comprises:

a detection information acquisition unit, configured to acquire detection information of a background scene and detection information of a current scene, wherein the current scene comprises an object(s) to be detected and the same background scene; and

an object detection unit, configured to calculate the object(s) to be detected according to the detection information of the background scene and the detection information of the current scene.