METHODS FOR RECOGNIZING POSE AND ACTION OF ARTICULATED OBJECTS WITH COLLECTION OF PLANES IN MOTION

• US 20100303303A1
• Filed: 05/28/2010
• Published: 12/02/2010
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: University of Central Florida Research Foundation Inc. (State University System of Florida)
• Original Assignee: University of Central Florida Research Foundation Inc. (State University System of Florida)
• Inventors: Foroosh, Hassan, Shen, Yuping
• Granted Patent: US 8,755,569 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The invention comprises an improved system, method, and computer-readable instructions for recognizing pose and action of articulated objects with collection of planes in motion. The method starts with a video sequence and a database of reference sequences corresponding to different known actions. The method identifies the sequence from the reference sequences such that the subject in performs the closest action to that observed. The method compares actions by comparing pose transitions. The cross-homography invariant may be used for view-invariant recognition of human body pose transition and actions.

Claims

1. A computer-implemented method for recognizing pose and action of articulated objects, comprising:

receiving by a computing device action sequence information as a set of points for each of a plurality of images, wherein the set of points represents a pose P defined by M body points obtained by using an articulated object tracking technique;

processing the action sequence information by solving via the computing device an action recognition problem to find one or more probable matches by comparing based on pose transition the action sequence information to a reference sequence, wherein the action recognition problem decomposes motions of the set of points into a collection of rigid motions of planes defined by triplets of points in the set; and

outputting at least one of the one or more probable matches.

2. The computer-implemented method of claim 1 wherein the action sequence information is received in real time or from previously recorded action sequence information.

3. The computer-implemented method of claim 1 wherein the reference sequence comprises a sequence of two-dimensional (2D) poses for each of a known action maintained in an action database of known actions.

4. The computer-implemented method of claim 1 wherein the pose P is represented by M body points such that P={mi=1 . . . M}.

5. The computer-implemented method of claim 1 wherein the plurality of images is obtained from a single camera view.

6. The computer-implemented method of claim 1 further comprising calculating and outputting by the processing device confidence level information for the one or more probable matches.

7. The computer-implemented method of claim 1 wherein the action recognition problem uses action sequence information comprising a set of non-rigid pose transitions defined by the triplets of points, such that each pose is decomposed into the collection of rigid motions of planes defined by triplets of points in the set, wherein the triplets of points represent planes and invariants of the planes are found for a rigid motion of the planes defined by the triplets across two frames.

8. The computer-implemented method of claim 7 wherein the invariants of the planes are found by one or both of a vector of fundamental ratios and an equality of eigenvalues of cross-homography, thereby providing a view-invariant recognition of actions.

9. The computer-implemented method of claim 8 wherein the vector of fundamental ratios defines three projective invariants for a configuration involving a moving plane as a moving triplet of points as observed in two frames by a stationary camera.

10. The computer-implemented method of claim 1 wherein the output of at least one of the one or more probable matches is utilized in one or more of a video surveillance system or a gesture-based computer interface.

11. A computer readable medium containing program instructions for execution on a computing device, which when executed by the computing device, causes the computing device to:

receive action sequence information as a set of points for each of a plurality of images, wherein the set of points represents a pose P defined by M body points obtained by using an articulated object tracking technique;

process the action sequence information by solving an action recognition problem to find one or more probable matches by comparing based on pose transition the action sequence information to a reference sequence, wherein the action recognition problem decomposes motions of the set of points into a collection of rigid motions of planes defined by triplets of points in the set; and

output at least one of the one or more probable matches.

12. The computer readable medium of claim 11 further comprising calculating and outputting by the processing device confidence level information for the one or more probable matches.

13. The computer readable medium of claim 11 wherein the action recognition problem uses action sequence information comprising a set of non-rigid pose transitions defined by the triplets of points, such that each pose is decomposed into the collection of rigid motions of planes defined by triplets of points in the set, wherein the triplets of points represent planes and invariants of the planes are found for a rigid motion of the planes defined by the triplets across two frames.

14. The computer readable medium of claim 13 wherein the invariants of the planes are found by one or both of a vector of fundamental ratios and an equality of eigenvalues of cross-homography, thereby providing a view-invariant recognition of actions.

15. The computer readable medium of claim 14 wherein the vector of fundamental ratios defines three projective invariants for a configuration involving a moving plane as a moving triplet of points as observed in two frames by a stationary camera.

16. A system for recognizing pose and action of articulated objects, comprising:

a computing device having at least one processing device, a memory coupled to the processing device, the computing device in communication with a data source comprising action sequence information of an articulated object and a database of known actions;

wherein the processing device executes a program residing in the memory to;

receive the action sequence information from the data source as a set of points for each of a plurality of images, wherein the set of points represents a pose P defined by M body points obtained by using an articulated object tracking technique;

process the action sequence information by solving an action recognition problem to find one or more probable matches by comparing based on pose transition the action sequence information to a reference sequence from the database of known actions, wherein the action recognition problem decomposes motions of the set of points into a collection of rigid motions of planes defined by triplets of points in the set; and

output at least one of the one or more probable matches.

17. The system of claim 16 further comprising calculating and outputting by the processing device confidence level information for the one or more probable matches.

18. The system of claim 16 wherein the action recognition problem uses action sequence information comprising a set of non-rigid pose transitions defined by the triplets of points, such that each pose is decomposed into the collection of rigid motions of planes defined by triplets of points in the set, wherein the triplets of points represent planes and invariants of the planes are found for a rigid motion of the planes defined by the triplets across two frames.

19. The system of claim 18 wherein the invariants of the planes are found by one or both of a vector of fundamental ratios and an equality of eigenvalues of cross-homography, thereby providing a view-invariant recognition of actions.

20. The system of claim 19 wherein the vector of fundamental ratios defines three projective invariants for a configuration involving a moving plane as a moving triplet of points as observed in two frames by a stationary camera.