MOVING OBJECT DETECTION APPARATUS AND MOVING OBJECT DETECTION METHOD

• US 20120206597A1
• Filed: 04/24/2012
• Published: 08/16/2012
• Est. Priority Date: 07/27/2010
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Komoto, Ayako, Nobori, Kunio, Iwasaki, Masahiro
• Granted Patent: US 8,599,252 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A moving object detection apparatus includes: a stationary measure calculation unit calculating, for each of trajectories, a stationary measure representing likelihood that the trajectory belongs to a stationary object; a distance calculation unit calculating a distance representing similarity between trajectories; and a region detection unit (i) performing a transformation based on the stationary measures and the distances between the trajectories, so that a ratio of a distance between a trajectory on stationary object and a trajectory on moving object, to a distance between trajectories both belonging to stationary object becomes greater than a ratio obtained before the transformation and (ii) detecting the moving object region by separating the trajectory on the moving object from the trajectory on the stationary object, based on a geodesic distance between the trajectories.

Claims

1. A moving object detection apparatus which detects a moving object region using trajectories which correspond to a region included in a video, said moving object detection apparatus comprising:

a stationary measure calculation unit configured to calculate, for each of the trajectories, a stationary measure representing likelihood that the trajectory belongs to a stationary object, the trajectory indicating a motion of blocks between two or more pictures included in the video, and each of the blocks being included in a picture and having one or more pixels;

a distance calculation unit configured to calculate a distance between each pair of trajectories, the distance representing similarity between the trajectories; and

a region detection unit configured (i) to perform a transformation based on the distances between trajectories and the stationary measure of these trajectories, so that a ratio of a distance between two trajectories that one belongs to stationary object and another belongs to moving object, to the distance between trajectories both belonging to stationary object, becomes greater than the ratio obtained before the transformation, and (ii) to detect the trajectories belonging to the moving object as the moving object region by separating the trajectories on moving object from those on stationary object, based on the transformed distances between trajectories.

2. The moving object detection apparatus according to claim 1,

wherein said distance calculation unit is configured to calculate, based on the distances between trajectories, a geodesic distance being a distance of a path that passes through, as a relay point, a trajectory other than the first trajectory and the second trajectory, to reach the second trajectory from the first trajectory, and

said region detection unit is configured (i) to perform a transformation based on the stationary measure of trajectories and the geodesic distances between the trajectories, so that a ratio of a geodesic distance between two trajectories that one belongs to stationary object and another belongs to moving object, to the distance between trajectories both belonging to stationary object, becomes greater than the ratio obtained before the transformation, and (ii) to detect the trajectories belonging to the moving object as the moving object region by separating the trajectories of the moving object from those of the stationary object, based on each of the transformed geodesic distances between trajectories.

3. The moving object detection apparatus according to claim 1,

wherein said stationary measure calculation unit is configured to estimate, based on the trajectories, a geometric constraint that holds when the trajectory belongs to the stationary object, and to calculate, as the stationary measure, a degree at which the trajectory satisfies the estimated geometric constraint.

4. The moving object detection apparatus according to claim 3,

wherein said stationary measure calculation unit is configured to estimate, based on the trajectories, any of following geometric constraints, an epipolar constraint, a homography constraint, a trilinear constraint, and a structure consistency constraint, and calculate, as the stationary measure, a degree at which the trajectory satisfies the estimated geometric constraint.

5. The moving object detection apparatus according to claim 1,

wherein said region detection unit includes:

a weighted distance calculation unit configured to (i) calculate a geodesic distance between each pair of trajectories based on the distance between these trajectories, and to (ii) calculate a weighted geodesic distance by assigning a weight based on the stationary measure of these trajectories, to the calculated geodesic distance, so that a ratio of a geodesic distance between two trajectories that one belongs to stationary object and another belongs to moving object, to the geodesic distance between trajectories both belonging to stationary object, becomes greater than the ratio obtained before the weight assignment; and

a segmentation unit configured to segment the picture into stationary object region and moving object region by classifying the trajectories into different classes if the weighted geodesic distance calculated by said weighted distance calculation unit is greater than or equal to a predetermined threshold, and into the same class if the distance is below the threshold.

6. The moving object detection apparatus according to claim 5,

wherein said distance calculation unit is configured to calculate a geodesic distance between each pair of trajectories, the geodesic distance representing similarity between the trajectories, and

said weighted distance calculation unit is configured to calculate the weighted geodesic distance by assigning a weight based on the stationary measures to the geodesic distance calculated by said distance calculation unit, so that a ratio of a geodesic distance between two trajectories that one belongs to stationary object and another belongs to moving object, to the geodesic distance between trajectories both belonging to stationary object becomes greater than the ratio obtained before the weight assignment.

7. The moving object detection apparatus according to claim 5,

wherein said distance calculation unit is configured to calculate a linear distance between each pair of trajectories, the linear distance representing similarity between the trajectories, and

said weighted distance calculation unit is configured to (i) calculate a weighted linear distance by assigning a weight based on the stationary measures of the trajectories to the linear distance calculated by said distance calculation unit, so that a ratio of a linear distance between two trajectories that one belongs to stationary object and another belongs to moving object, to a linear distance between trajectories both belonging to stationary object, becomes greater than the ratio obtained before the weight assignment, and to (ii) calculate the weighted geodesic distance based on the calculated weighted linear distance.

8. The moving object detection apparatus according to claim 1,

wherein said region detection unit includes:

a weighted distance calculation unit configured to calculate a geodesic distance between each pair of trajectories based on the distance between the trajectories; and

a segmentation unit configured to segment the picture into stationary object region and moving object region, by separating trajectories that belong to the moving object, and those belong to the stationary object into different classes, based on a result whether or not the geodesic distance calculated by said weighted distance calculation unit is greater than a weighted geodesic distance threshold, which is obtained by assigning a weight based on the stationary measures to a geodesic distance threshold, the threshold applied to determine whether or not to classify the trajectories on stationary object and moving object into different classes.

9. The moving object detection apparatus according to claim 8,

wherein said segmentation unit is configured to perform the weight assignment so that the weight assigned to the geodesic distance threshold is greater when the stationary measures of the pair of trajectories both indicates “

stationary”

, than when the stationary measures of the pair of trajectories indicate “

moving” and

“

stationary”

, respectively.

10. The moving object detection apparatus according to claim 5, further comprising

a subclass classification unit configured to classify the trajectories into subclasses each of which is a subset of similar trajectories out of the trajectories,

wherein said weighted distance calculation unit is configured to calculate, based on stationary measures of the trajectories, the distances between trajectories, and classified subclasses, the weighted geodesic distance assigned with the weight based on the stationary measures, and

said segmentation unit is configured to segment the picture into regions of stationary object and those of moving object, by separating the trajectories on the moving object from the trajectories on the stationary object, based on the weighted geodesic distance calculated by said weighted distance calculation unit.

11. The moving object detection apparatus according to claim 5, further comprising

a subclass classification unit configured to classify the trajectories into subclasses each of which is a subset of similar trajectories out of the trajectories,

wherein said weighted distance calculation unit is configured to calculate, based on stationary measures of the trajectories, the distances between trajectories, and classified subclasses, the weighted inter-subclass geodesic distance assigned with the weight based on stationary measures, and

said segmentation unit is configured to segment the picture into regions of stationary object and those of moving object, by separating the subclasses on the moving object from the subclasses on the stationary object, based on the weighted inter-subclass geodesic distance calculated by said weighted distance calculation unit.

12. The moving object detection apparatus according to claim 11,

wherein said distance calculation unit is configured to calculate a geodesic distance between each pair of trajectories, the geodesic distance representing similarity between the trajectories, and

said weighted distance calculation unit is configured to calculate the weighted inter-subclass geodesic distance by assigning, to a representative value of inter-subclass geodesic distances, a weight based on the representative value of stationary measures of trajectories within the subclass.

13. The moving object detection apparatus according to claim 11,

wherein said distance calculation unit is configured to calculate a linear distance between each pair of trajectories, the linear distance representing similarity between the trajectories, and

said weighted distance calculation unit is configured to assign, to a representative value of inter-subclass linear distances, a weight based on a representative value of stationary measures of the trajectories within the subclass, and calculate the weighted inter-subclass geodesic distance based on the weighted representative value of the inter-subclass linear distances.

14. The moving object detection apparatus according to claim 11,

wherein, if a subclass has a representative stationary measure indicating “

moving” and

another has that of “

stationary”

, said weighted distance calculation unit is configured to set a value greater than 1 as the weight to be assigned to the inter-subclass geodesic distance between such pair of subclasses.

15. The moving object detection apparatus according to claim 11,

wherein, when the pair of the subclasses both have representative stationary measures indicating “

stationary”

, said weighted distance calculation unit is configured to set a value smaller than 1 as the weight to be assigned to the inter-subclass geodesic distance.

16. The moving object detection apparatus according to claim 14,

wherein, when the pair of subclasses both have representative stationary measures indicating “

moving”

, said weighted distance calculation unit is configured to set a value 1 as the weight to be assigned to the inter-subclass geodesic distance.

17. The moving object detection apparatus according to claim 10,

wherein said subclass classification unit is configured to classify trajectories into different subclasses, based on the similarity between the trajectories.

18. The moving object detection apparatus according to claim 10,

wherein said subclass classification unit is configured to classify trajectories into different subclasses, based on similarity in luminance between blocks respectively belonging to the trajectories.

19. The moving object detection apparatus according to claim 10,

wherein said subclass classification unit includes:

a second distance calculation unit configured to calculate a geodesic distance between each pair of trajectories; and

a clustering unit configured to perform a dimensionality reduction on the geodesic distance between the trajectories calculated by said second distance calculation unit, and classify the trajectories into the subclasses based on the lower-dimensional geodesic distance between the trajectories.

20. The moving object detection apparatus according to claim 10,

wherein said subclass classification unit is configured to classify trajectories into different subclasses by:

obtaining the geodesic distance between a pair of trajectories by calculating a shortest path from one trajectory (A) to another (B) after (i) selecting a distance smaller than or equal to a predetermined distance threshold from the distances between pairs including former trajectory (A), and (ii) performing nonlinear processing to change an unselected distances between pairs including former trajectory (A) into an infinite distance; and

classifying, into the same subclass, if a set of trajectories between each pair of which the geodesic distance is finite.

21. The moving object detection apparatus according to claim 1,

wherein said stationary measure calculation unit is configured to estimate a geometric constraint from a trajectory used for estimating the geometric constraint that holds when the trajectory belongs to the stationary object, and calculate, based on the estimated geodesic distance, the stationary measures of the trajectories to be used by said distance calculation unit to calculate the distance.

22. The moving object detection apparatus according to claim 5, further comprising

a camera motion obtaining unit configured to obtain motion information on a camera capturing the video,

wherein said weighted distance calculation unit is configured to adjust the weight which is to be assigned to the stationary measure, based on a magnitude indicated by the motion information of the camera.

23. The moving object detection apparatus according to claim 22,

wherein, when the magnitude indicated by the motion information on the camera is greater, said weighted distance calculation unit is configured to increase the weight that is to be assigned to the distance between the trajectory on stationary object and the trajectory on the moving object.

24. The moving object detection apparatus according to claim 11, further comprising

a camera motion obtaining unit configured to obtain motion information of a camera capturing the video,

wherein, if a subclass has a representative stationary measure indicating “

moving” and

another has that of “

stationary”

, said weighted distance calculation unit is configured to perform a weight assignment on an inter-subclass geodesic distance between such pair of subclasses, and

a relationship expressed as WB>

WS>

1 is satisfied, where WB represents a weight of a case where a value of the motion information on the camera is greater than or equal to a predetermined threshold and WS represents a weight of a case where the value of the motion information on the camera is smaller than the predetermined threshold.

25. The moving object detection apparatus according to claim 11, further comprising

a camera motion obtaining unit configured to obtain motion information on a camera capturing the video,

wherein, when the pair of subclasses both has representative stationary measures indicating “

stationary”

, said weighted distance calculation unit is configured to perform a weight assignment on an inter-subclass geodesic distance between such pair of subclasses, and

a relationship expressed as WB<

WS<

1 is satisfied, where WB represents a weight of a case where a value of the motion information on the camera is greater than or equal to a predetermined threshold and WS represents a weight of a case where the value of the motion information on the camera is smaller than the predetermined threshold.

26. The moving object detection apparatus according to claim 22,

wherein said camera motion obtaining unit is configured to obtain the motion information on the camera from an operation control signal of the camera.

27. The moving object detection apparatus according to claim 22,

wherein said camera motion obtaining unit is configured to obtain the motion information on the camera from an in-car sensor.

28. The moving object detection apparatus according to claim 5,

wherein said weighted distance calculation unit is further configured to compare the stationary measure of the trajectory with a stationary measure threshold, and to determine that if the stationary measure is smaller than or equal to the stationary measure threshold, the trajectory is on the stationary object, and that if the stationary measure is greater than the stationary measure threshold, the trajectory is on the moving object.

29. The moving object detection apparatus according to claim 28,

wherein said weighted distance calculation unit includes a threshold receiving unit configured to receive a stationary measure threshold, and is configured to compare the stationary measure of the trajectory with the stationary measure threshold received by said threshold receiving unit, and to determine that if the stationary measure is smaller than or equal to the stationary measure threshold, the trajectory belongs to the stationary object, and that if the stationary measure is greater than the stationary measure threshold, the trajectory belongs to the moving object, and

said region detection unit is configured to cause a display unit to display the detected moving object region.

30. The moving object detection apparatus according to claim 1,

wherein said stationary measure calculation unit is further configured to cause a display unit to display each of the blocks included in the picture as to reflect the calculated stationary measure.

31. A moving object detection method for detecting a moving object region using trajectories which correspond to a region included in a video, said moving object detection method comprising:

calculating, for each of the trajectories, a stationary measure representing likelihood that the trajectory belongs to a stationary object, the trajectory indicating a motion of blocks between two or more pictures included in the video, and each of the blocks being included in a picture and having one or more pixels;

calculating a distance between each pair of trajectories, the distance representing similarity between the trajectories; and

(i) performing a transformation based on the stationary measures of the trajectories, and the distances between the trajectories, so that a ratio of a distance between a trajectory on stationary object and another on moving object, to a distance between trajectories both belonging to stationary object becomes greater than the ratio obtained before the transformation and (ii) detecting the trajectories on the moving object as the moving object region corresponding to the trajectory belonging to the moving object by separating the trajectory on the moving object from the trajectory on the stationary object, based on each of the transformed distances between pairs of trajectories.

32. A non-transitory computer-readable recording medium for use in a computer, said recording medium having a computer program recorded thereon for causing the computer to execute the moving object detection method according to claim 31 to detect a moving object region using trajectories which correspond to a region included in a video.