MOVING OBJECT DETECTION APPARATUS AND MOVING OBJECT DETECTION METHOD

• US 20130051624A1
• Filed: 10/31/2012
• Published: 02/28/2013
• Est. Priority Date: 03/22/2011
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: IWASAKI, Masahiro, NOBORI, Kunio, KOMOTO, Ayako
• Granted Patent: US 8,605,946 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A moving object detection apparatus includes: a trajectory calculating unit which calculates a plurality of trajectories for each image subset; a subclass classification unit which classifies the trajectories into subclasses for each of the image subsets, an inter-subclass similarity calculating unit which calculates a trajectory share ratio which indicates the degree of sharing of the same trajectory between arbitrary two of the subclasses and calculates the similarity between the subclasses based on the share ratio; a moving object detection unit which detects the moving object in video by classifying the subclasses into classes such that the subclasses between which a higher similarity is present are more likely to be classified into the same class.

Claims

1. A moving object detection apparatus which detects a moving object in video by performing segmentation to identify an entire or partial region of the moving object in the video, the moving object detection apparatus comprising:

an image input unit configured to receive at least three pictures included in the video;

a trajectory calculating unit configured to (i) extract, from among the at least three pictures, image subsets each including at least two of the pictures, and (ii) calculate, for each of the image subsets, trajectories each of which is a trajectory of motion of a block, between the pictures included in the image subset, the block including one or more pixels and being included in each of the pictures;

a subclass classification unit configured to classify, for each of the image subsets, the trajectories into subclasses, the trajectories being included in the image subset and calculated by the trajectory calculating unit;

an inter-subclass similarity calculating unit configured to (i) calculate a trajectory share ratio which indicates a degree of sharing a same trajectory between two of the subclasses, and (ii) calculate a similarity between the two of the subclasses based on the trajectory share ratio; and

a moving object detection unit configured to detect the moving object in the video by classifying the subclasses into classes such that the subclasses between which the similarity calculated by the inter-subclass similarity calculating unit is higher are more likely to be classified into the same class, to identify, as the region of a moving object, the blocks which correspond to the trajectories included in a same class.

2. The moving object detection apparatus according to claim 1,

wherein the subclass classification unit includes a subclass label assigning unit configured to (i) classify, for each of the image subsets, the trajectories which are included in the image subset and calculated by the trajectory calculating unit, into the subclasses, and (ii) assign, to each of the trajectories, a subclass label that is an identifier of a corresponding one of the subclasses into which the trajectory is classified,

the inter-subclass similarity calculating unit includes a label propagation unit configured to propagate the subclass label between the subclasses by (i) calculating the trajectory share ratio which indicates a degree of sharing a trajectory indicating motion of a same block between a first subclass and a second subclass which are arbitrarily selected from among all of the subclasses, and (ii) reassigning the subclass label such that the first subclass and the second subclass between which the trajectory share ratio is higher are more likely to be reassigned with the same subclass label, and

the moving object detection unit is configured to detect the moving object in the video by classifying, into the same class, the subclasses to which the same subclass label is assigned, to identify, as the region of the moving object, the blocks which correspond to the trajectories included in the same class.

3. The moving object detection apparatus according to claim 2,

wherein the trajectory calculating unit is configured to (i) extract the image subsets from among the at least three pictures such that some of the pictures are shared between temporally adjacent image subsets, and (ii) calculate, for each of the image subsets, trajectories each of which is a trajectory of motion of a block, between the pictures included in the image subset, the block including one or more pixels and being included in the pictures, and

the label propagation unit is configured to propagate the subclass label between the subclasses by (I) determining that the trajectory share ratio between the first subclass and the second subclass is higher as a ratio of a trajectory passing a same position in the pictures shared between the first subclass and the second subclass to the trajectories included in the first subclass and the second subclass is higher, and (ii) reassigning the subclass label such that the first subclass and the second subclass between which the trajectory share ratio is higher are more likely to be reassigned with the same subclass label.

4. The moving object detection apparatus according to claim 2,

wherein the label propagation unit is further configured to propagate the subclass label between the subclasses by performing reassigning such that the same subclass label as the subclass label assigned by the subclass label assigning unit is more likely to be reassigned to the subclass which is higher in reliability that is a value corresponding to the number of trajectories included in the subclass or a spatial size of a region including trajectories included in the subclass.

5. The moving object detection apparatus according to claim 2, further comprising:

a holding unit configured to hold, from among the trajectories calculated by the trajectory calculating unit, trajectories which are calculated using fewer pictures than the pictures included in the image subset; and

a trajectory recovery unit configured to classify the trajectories held in the holding unit into a same class as a class to which other trajectories continuous to the trajectories held in the holding unit belong,

wherein the moving object detection unit is further configured to detect the moving object in the video by identifying, as the region of the moving object, blocks which correspond to the trajectories included in the same class, the trajectories including the trajectories classified by the trajectory recovery unit.

6. The moving object detection apparatus according to claim 2,

wherein the subclass label assigning unit is configured to classify, for each of the image subsets, the trajectories which are included in the image subset and calculated by the trajectory calculating unit, into subclasses by classifying similar trajectories from among the trajectories into the same subclass, and assign, to each of the trajectories, a subclass label which is an identifier of the subclass into which the trajectory is classified.

7. The moving object detection apparatus according to claim 6,

wherein the subclass label assigning unit is configured to classify, for each of the image subsets, the trajectories into the subclasses, by performing a process of classifying, into a same subclass, a combination of most similar trajectories from among the trajectories which are included in the image subset and calculated by the trajectory calculating unit, repeatedly for predetermined times or until the trajectories are classified into a predetermined number of subclasses, and assign, to each of the trajectories, a subclass label which is an identifier of the subclass into which the trajectory is classified.

8. The moving object detection apparatus according to claim 6,

wherein the subclass label assigning unit is configured to (i) classify the trajectories calculated by the trajectory calculating unit into subclasses, by (a) calculating a distance that indicates the similarity between the trajectories, (b) combining, from among the calculated distances, distances each of which is shorter than a predetermined threshold, to transform the calculated distances into geodetic distances, and (c) calculating, for each of the image subsets, a geodetic distance of two arbitrary trajectories from among the trajectories which are included in the image subset and calculated by the trajectory calculating unit, detecting a discontinuity point in a distribution of the obtained geodetic distances, sorting out, as one cluster, trajectories which are spaced apart from each other by the geodetic distance shorter than the detected discontinuity point, to classify the trajectories into subclasses, and (ii) assign, for each of the trajectories, a subclass label that is an identifier of the subclass into which the trajectory is classified, the geodetic distance being a distance of a path that passes through, as a relay point, a trajectory other than the arbitrary two trajectories, from one of the two trajectories to the other.

9. The moving object detection apparatus according to claim 2,

wherein the subclass label assigning unit is configured to divide, for each of the image subsets, at least one image into a plurality of regions, using at least one of edge, color, and brightness of the at least one image, to generate subclasses, and assign a subclass label to each of the trajectories that pass through the regions resulting from the division, the at least one image being included in the image subsets.

10. The moving object detection apparatus according to claim 2, further comprising

an output unit configured to perform image processing on at least one of the pictures received by the image input unit, such that each of moving object region classified into the same class by the moving object detection unit is displayed in a display mode different from a display mode of other moving object regions, and output the picture on which the image processing is performed.

11. The moving object detection apparatus according to claim 2, further comprising

a recording and transmitting unit configured to write or transmit a result of detecting a moving object performed by the moving object detection unit, into a recording device or to an external device via a transmission path, respectively.

12. The moving object detection apparatus according to claim 2, further comprising

a motion prediction unit configured to calculate, from among the trajectories which are identified by the moving object detection unit and included in the class, a representative trajectory that represents the class, and predict, according to the calculated representative trajectory, motion of the moving object by predicting that a region of the moving object corresponding to the class moves between the pictures.

13. The moving object detection apparatus according to claim 1,

wherein the inter-subclass similarity calculating unit is configured to calculate a similarity between the subclasses by repeatedly multiplying a matrix W in which each of elements indicates the trajectory share ratio between the subclasses, by a matrix Z in which each of the elements indicates the similarity between the subclasses, to update the matrix Z in which each of the elements indicates the similarity between the subclasses.

14. The moving object detection apparatus according to claim 13,

wherein the inter-subclass similarity calculating unit is further configured to calculate the similarity between the subclasses by weighting the updated matrix Z with a matrix Y in which each of the elements indicates a subclass reliability that is a value corresponding to the number of trajectories included in the subclass or a spatial size of a region including trajectories included in the subclass.

15. A method of detecting a moving object in video by performing segmentation to identify an entire or partial region of the moving object in the video, the method comprising:

receiving at least three pictures included in the video;

extracting, from among the at least three pictures, image subsets each including at least two of the pictures, and calculating, for each of the image subsets, trajectories each of which is a trajectory of motion of a block, between the pictures included in the image subset, the block including one or more pixels and being included in each of the pictures;

classifying, for each of the image subsets, the calculated trajectories into subclasses, the calculated trajectories being included in the image subset;

calculating a trajectory share ratio which indicates a degree of sharing a same trajectory between two of the subclasses, and calculating a similarity between the two of the subclasses based on the trajectory share ratio; and

detecting the moving object in the video by classifying the subclasses into classes such that the subclasses between which the calculated similarity is higher are more likely to be classified into the same class, to identify, as the region of a moving object, the blocks which correspond to the trajectories included in a same class.

16. A non-transitory computer-readable recording medium for use in a computer, the recording medium having a computer program recorded thereon for causing the computer to execute all of the steps included in the moving object detection method according to claim 15.