ARTICULATED OBJECT REGIONARTICULATED OBJECT REGION DETECTION APPARATUS AND METHOD OF THE SAME

• US 20110255748A1
• Filed: 06/28/2011
• Published: 10/20/2011
• Est. Priority Date: 12/28/2009
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Iwasaki, Masahiro, Nobori, Kunio, Komoto, Ayako
• Granted Patent: US 8,401,243 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An articulated object region detection apparatus includes: a subclass classification unit which classifies trajectories into subclasses; a distance calculating unit which calculates, for each of the subclasses, a point-to-point distance and a geodetic distance between the subclass and another subclass; and a region detection unit which detects, as a region having an articulated motion, two subclasses to which trajectories corresponding to two regions connected via the same articulation and indicating the articulated motion belong, based on a temporal change in the point-to-point distance and a temporal change in the geodetic distance between two given subclasses.

Claims

1. An articulated object region detection apparatus which detects a region having an articulated motion, using trajectories which correspond, in one-to-one relationship, to regions included in a moving object in a video, said articulated object region detection apparatus comprising:

a subclass classification unit configured to classify trajectories into subclasses each of which is a group of similar trajectories, each of the trajectories being a trajectory of a motion of a block and being over pictures included in the video, the block including one or more pixels each of which constitutes the pictures;

a distance calculating unit configured to calculate, for each of the subclasses, a point-to-point distance and a geodetic distance, the point-to-point distance being a distance directly connecting a first trajectory that belongs to the subclass and a second trajectory that belongs to another one of the subclasses, the geodetic distance being a distance of a path that passes through, as a relay point, a trajectory other than the first trajectory and the second trajectory, to reach the second trajectory from the first trajectory; and

a region detection unit configured to detect, as a region having the articulated motion, two subclasses that include trajectories corresponding to two regions connected via a same articulation and including the articulated motion, based on a temporal change in the point-to-point distance and a temporal change in the geodetic distance between two given subclasses among the subclasses.

2. The articulated object region detection apparatus according to claim 1,

wherein said region detection unit is configured to detect, as the region having an articulated motion, a subclass that is one of two subclasses between which the temporal change in the point-to-point distance is larger than a predetermined threshold of the point-to-point distance and the temporal change in the geodetic distance is equal to or smaller than a predetermined threshold of the geodetic distance.

3. The articulated object region detection apparatus according to claim 2,

wherein said region detection unit is configured to detect, from the subclasses classified by said subclass classification unit, a subclass that is one of two subclasses between which the temporal change in the point-to-point distance is larger than the predetermined threshold of the point-to-point distance, and detect from the detected subclasses, as the region having an articulated motion, a subclass that is one of two subclasses between which the temporal change in the geodetic distance is equal to or smaller than the predetermined threshold of the geodetic distance.

4. The articulated object region detection apparatus according to claim 2,

wherein said region detection unit is configured to detect, from the subclasses classified by said subclass classification unit, a subclass that is one of two subclasses between which the temporal change in the geodetic distance is equal to or smaller than the predetermined threshold of the geodetic distance, and detect from the detected subclasses, as the region having an articulated motion, a subclass that is one of two subclasses between which the temporal change in the point-to-point distance is larger than the predetermined threshold of the point-to-point distance.

5. The articulated object region detection apparatus according to claim 2,

wherein said region detection unit is configured to perform a first detection and a second detection in parallel, and detect, as the region having an articulated motion, a subclass detected in both of the first detection and the second detection, the first detection being a detection in which a subclass that is one of two subclasses between which the temporal change in the point-to-point distance is larger than the predetermined threshold of the point-to-point distance is detected from the subclasses classified by said subclass classification unit, and the second detection being a detection in which a subclass that is one of two subclasses between which the temporal change in the geodetic distance is equal to or smaller than the predetermined threshold of the geodetic distance is detected from the subclasses classified by said subclass classification unit.

6. The articulated object region detection apparatus according to claim 2,

wherein said region detection unit is further configured to detect, as the region having an articulated motion, a subclass that is one of two subclasses between which the temporal change in the point-to-point distance is larger than the predetermined threshold of the point-to-point distance and the temporal change in the geodetic distance is equal to or smaller than the predetermined threshold of the geodetic distance, only when a minimum value of the point-to-point distance between the subclasses is equal to or smaller than a predetermined threshold of a minimum value.

7. The articulated object region detection apparatus according to claim 1,

wherein said region detection unit is configured to detect, as regions having the articulated motion, two subclasses that include trajectories corresponding to two regions connected via a same articulation and indicating the articulated motion, based on a temporal change in a representative value of the point-to-point distances and a temporal change in a representative value of the geodetic distances between two given subclasses.

8. The articulated object region detection apparatus according to claim 7,

wherein the representative value of the point-to-point distances between the two subclasses is an average of the point-to-point distances between the two subclasses, and

the representative value of the geodetic distances between the two subclasses is an average of the geodetic distances between the two subclasses.

9. The articulated object region detection apparatus according to claim 7,

wherein the representative value of the point-to-point distances between the two subclasses is a median of the point-to-point distances between the two subclasses, and

the representative value of the geodetic distances between the two subclasses is a median of the geodetic distances between the two subclasses.

10. The articulated object region detection apparatus according to claim 7,

wherein the representative value of the point-to-point distances between the two subclasses is a mode value of the point-to-point distances between the two subclasses, and

the representative value of the geodetic distances between the two subclasses is a mode value of the geodetic distances between the two subclasses.

11. The articulated object region detection apparatus according to claim 1,

wherein said subclass classification unit is configured to classify the trajectories into subclasses based on the point-to-point distance between the trajectories.

12. The articulated object region detection apparatus according to claim 1,

wherein said subclass classification unit is configured to classify the trajectories into subclasses based on a color similarity between pixels that belong to respective trajectories or blocks to which the pixels belong between the trajectories.

13. The articulated object region detection apparatus according to claim 1,

wherein said subclass classification unit is configured to perform dimensionality reduction on the geodetic distance between the trajectories, and classify the trajectories into the subclasses based on the geodetic distance between the trajectories on which dimensionality reduction is performed.

14. The articulated object region detection apparatus according to claim 1,

wherein said distance calculating unit is configured to select, for each of the trajectories, a point-to-point distance equal to or smaller than a predetermined threshold of a distance, from among point-to-point distances from the trajectory to other trajectories, and calculate the geodetic distance by obtaining a shortest path from the trajectory to the other trajectories after performing non-linearization to change unselected point-to-point distances into infinite.

15. The articulated object region detection apparatus according to claim 14,

wherein said distance calculating unit is configured to select, for each of the trajectories, the point-to-point distance equal to or smaller than the predetermined threshold of a distance, from among the point-to-point distances from the trajectory to the other trajectories, calculate the geodetic distance by obtaining the shortest path from the trajectory to the other trajectories after performing non-linearization to change the unselected point-to-point distances into infinite, and classify the trajectories into the subclasses by classifying, into a same subclass, a group of trajectories between which the geodetic distances indicate finite values.

16. The articulated object region detection apparatus according to claim 1, further comprising an output unit configured to perform image processing on at least one of the pictures so as to be displayed in a different mode per region detected by said region detection unit, and output the picture including the region on which the image processing is performed.

17. The articulated object region detection apparatus according to claim 1,

wherein the video is captured to image a person as an object, and

said articulated object region detection apparatus further comprising an output unit configured to perform image processing on at least one of the pictures so as to be displayed in a display mode in which a region of the person can be visually recognized, per region detected by said region detection unit, and output the picture including the region on which the image processing is performed.

18. The articulated object region detection apparatus according to claim 1,

wherein the video is captured to image a person as an object, and

said articulated object region detection apparatus further comprising an action recognition unit configured to calculate an angle between subclasses which are included in the region detected by said region detection unit and which indicate an articulated motion, and recognize an action of the object by comparing the calculated angle with an angle which is determined in advance for each action pattern of the person.

19. The articulated object region detection apparatus according to claim 18, further comprising an output unit configured to perform image processing on at least one of the pictures, based on the action of the object recognized by said action recognition unit, so as to be displayed in a display mode in which the action pattern of the person can be visually recognized, and output the picture including the region on which the image processing is performed.

20. The articulated object region detection apparatus according to claim 1, further comprising an action detecting unit configured to detect a change in the number of connections of a moving object included in the video, based on the number of articulations between the subclasses which are included in the region detected by said region detection unit and which indicate the articulated motion.

21. The articulated object region detection apparatus according to claim 20,

wherein said action detecting unit is configured to determine that an item is abandoned by the moving object when detecting a decrease in the number of connections and determine that an item is taken away by the moving object when detecting an increase in the number of connections.

22. The articulated object region detection apparatus according to claim 21, further comprising an output unit configured to perform image processing on at least one of the pictures, based on a result of the determination performed by said action detecting unit, so as to be displayed in a display mode in which abandonment or taking away of the item can be visually recognized, and output the picture including the region on which the image processing is performed.

23. The articulated object region detection apparatus according to claim 1, further comprising:

an image input unit configured to capture pictures included in the video; and

a trajectory calculating unit configured to detect, per block including one or more pixels and constituting the captured pictures, a motion of the block between two temporally neighboring pictures included in the video, and calculate the trajectories by concatenating the detected motion for the pictures.

24. The articulated object region detection apparatus according to claim 23,

wherein said movement trajectory calculating unit is configured to change the number of pictures used for calculating trajectories, according to the number of the trajectories to be calculated, and calculate the trajectories.

25. The articulated object region detection apparatus according to claim 24,

wherein said trajectory calculating unit is configured to increase the number of the pictures used for calculating the trajectories, when the number of the trajectories to be calculated exceeds a predetermined upper limit, and calculate the trajectories.

26. The articulated object region detection apparatus according to claim 24,

wherein said trajectory calculating unit is configured to decrease the number of the pictures used for calculating the trajectories, when the number of the trajectories to be calculated falls below a predetermined lower limit, and calculate the trajectories.

27. The articulated object region detection apparatus according to claim 1,

wherein the articulated motion is a motion of an object connected via an articulation.

28. The articulated object region detection apparatus according to claim 1,

wherein the articulated motion is a motion of a string-shape object.

29. An articulated object region detection method of detecting a region having an articulated motion, using trajectories which correspond, in one-to-one relationship, to regions included in a moving object in a video, said articulated object region detection method comprising:

classifying trajectories into subclasses each of which is a group of similar trajectories, each of the trajectories being a trajectory of a motion of a block and being over pictures included in the video, the block including one or more pixels each of which constitutes the pictures;

calculating, for each of the subclasses, a point-to-point distance and a geodetic distance, the point-to-point distance being a distance directly connecting a first trajectory that belongs to the subclass and a second trajectory that belongs to another one of the subclasses, the geodetic distance being a distance of a path that passes through, as a relay point, a trajectory other than the first trajectory and the second trajectory, to reach the second trajectory from the first trajectory; and

detecting, as a region having the articulated motion, two subclasses that include trajectories corresponding to two regions connected via a same articulation and including the articulated motion, based on a temporal change in the point-to-point distance and a temporal change in the geodetic distance between two given subclasses among the subclasses.

30. A non-transitory computer-readable recording medium on which a program is recorded, the program being for detecting a region having an articulated motion, using trajectories which correspond, in one-to-one relationship, to regions included in a moving object in a video, and

the program causing a computer to execute the articulated object region detection method according to claim 29.

31. An articulated object region detection apparatus which detects a region having an articulated motion, using trajectories which correspond, in one-to-one relationship, to regions included in a moving object in a video, said articulated object region detection apparatus comprising:

a subclass obtaining unit configured to obtain classification information that indicates a result of classifying trajectories into subclasses each of which is a group of similar trajectories, each of the trajectories being a trajectory of a motion of a block and being over pictures included in the video, the block including one or more pixels each of which constitutes the pictures;

a distance calculating unit configured to calculate, for each of the subclasses, a point-to-point distance and a geodetic distance, based on the obtained classification information, the point-to-point distance being a distance directly connecting a first trajectory that belongs to the subclass and a second trajectory that belongs to another one of the subclasses, the geodetic distance being a distance of a path that passes through, as a relay point, a trajectory other than the first trajectory and the second trajectory, to reach the second trajectory from the first trajectory; and

a region detection unit configured to detect, as a region having the articulated motion, two subclasses that include trajectories corresponding to two regions connected via a same articulation and including the articulated motion, based on a temporal change in the point-to-point distance and a temporal change in the geodetic distance between two given subclasses among the subclasses.