Apparatus for behavior analysis and method thereof

• US 20100278391A1
• Filed: 10/12/2006
• Published: 11/04/2010
• Est. Priority Date: 10/12/2006
• Status: Abandoned Application

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Hsu, Yung-Tai, Hsieh, JunWei, Liao, HongYuan
• Application Number:
  US11/546,400
• Publication Number:
  US 20100278391A1
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/424 Syntactic representation, e.g. by using alphabets or grammars
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 30/1985 Syntactic analysis, e.g. using a grammatical approach syntactic image representation G06V30/18171
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

In the present invention, an apparatus for behavior analysis and method thereof is provided. In this apparatus, each behavior is analyzed and has its corresponding posture sequence through a triangulation-based method of triangulating the different triangle meshes. The two important posture features, the skeleton feature and the centroid context, are extracted and complementary to each other. The outstanding ability of posture classification can generate a set of key postures for coding a behavior sequence to a set of symbols. Then, based on the string representation, a novel string matching scheme is proposed to analyze different human behaviors even though they have different scaling changes. The proposed method of the present invention has been proved robust, accurate, and powerful especially in human behavior analysis.

Claims

1. An apparatus for posture recognition comprising:

a triangulation unit for dividing a posture of a body into a plurality of triangular meshes; and

a recognition unit for forming a spanning tree correspond to the meshes to recognize the posture.

2. The apparatus for posture recognition according to claim 1 further comprises a background subtraction unit to extract and define a boundary of the body.

3. The apparatus for posture recognition according to claim 2, wherein the background subtraction unit is a video.

4. The apparatus for posture recognition according to claim 1, wherein the meshes are triangles.

5. The apparatus for posture recognition according to claim 1, wherein the recognition unit is achieved via a skeleton analysis or a centroid context analysis.

6. The apparatus for posture recognition according to claim 5, wherein the skeleton analysis is defined via a graph search scheme.

7. An apparatus for behavior analysis comprising:

a clustering unit for key postures selection via merging a plurality of postures iteratively;

a coding unit for translating all the input postures into a plurality of correspondent symbols according to the selected key postures; and

a matching unit, which takes advantages of the coding unit, for unscrambling the correspondent symbols to distinguish a behavior.

8. The apparatus for behavior analysis according to claim 7, wherein the clustering unit is programmable.

9. The apparatus for behavior analysis according to claim 7, wherein the clustering unit is user-defined.

10. The apparatus for behavior analysis according to claim 7, wherein the postures are obtained from an apparatus for posture recognition comprising:

a triangulation unit for dividing a body posture into a plurality of triangular meshes; and

a recognition unit for forming a spanning tree from the meshes to recognize the posture.

11. The apparatus for behavior analysis according to claim 10, wherein the apparatus for posture recognition further comprises a background subtraction unit to extract and define boundaries of the body.

12. The apparatus for behavior analysis according to claim 10, wherein the meshes can be defined via a triangle-mesh algorithm.

13. The apparatus for behavior analysis according to claim 10, wherein the recognition unit is achieved via a skeleton analysis or a centroid context analysis.

14. The apparatus for behavior analysis according to claim 13, wherein the skeleton analysis is defined via a graph search scheme.

15. The apparatus for behavior analysis according to claim 13, wherein the centroid context is formed by labeling each mesh with a number.

16. The apparatus for behavior analysis according to claim 7, wherein the correspondent symbols are unscrambling via a string matching method.

17. A method for posture recognition, comprising the steps of:

triangulating a posture of a body into a plurality of triangular meshes;

forming a skeleton analysis and a centroid context analysis correspond to the triangulated meshes; and

recognizing and analyzing the posture.

18. The method for posture recognition according to claim 17, wherein extracting and defining a boundary of the body is done via a background subtraction.

19. The method for posture recognition according to claim 17, wherein forming the meshes is based on a general triangulation algorithm.

20. The method for posture recognition according to claim 17, wherein the skeleton analysis comprises the steps of:

inputting a set of the triangle meshes extracted from the posture;

constructing a graph from the set of triangle meshes according to connectivity of a plurality of nodes in the triangle meshes;

applying a depth first search to the graph for finding a correspondent spanning tree;

extracting the skeleton feature from the spanning tree; and

outputting the skeleton feature of the posture.

21. The method for posture recognition according to claim 17, wherein a distance between the two different postures, P and D, defined via the skeleton analysis is satisfied with:

22. The method for posture recognition according to claim 17, wherein the centroid context analysis comprises the steps of:

finding the spanning tree of the posture;

tracing the spanning tree via using the depth first search recursively until the spanning tree is empty;

removing a plurality of branch nodes from the spanning tree;

finding and collecting a plurality of visited nodes from a set of paths;

defining a centroid histogram of each path centroid via using;

h_r(k)=#{q|q≠

r, (q−

r)ε

bin^k}, where bin^k is a kth bin of log-polar coordinate; and

collecting all the histograms as the output of the centroid context extraction of the posture.

23. The method for posture recognition according to claim 17, wherein a distance between the two different postures, P and Q, defined via the centroid context analysis is satisfied with:

24. The method for posture recognition according to claim 17, wherein a distance between the two different postures, P and Q, defined via both the skeleton analysis and the centroid context analysis is satisfied with:

Error(P,Q)=w d_skeleton(P,Q)+(1−

w)d_CC(P,Q),

where Error(P,Q) is the integrated distance between the postures P and Q and w is a weight for balancing the two distances d_skeleton(P,Q) and d_CC(P,Q).

25. A method for behavior analysis, comprising the steps of:

selecting a plurality of key postures;

coding the input postures into a plurality of correspondent symbols according to the selected key postures; and

matching the correspondent symbols to distinguish a behavior.

26. The method for behavior analysis according to claim 25, wherein selecting the key postures is programmable.

27. The method for behavior analysis according to claim 25, wherein selecting the key postures is user-defined.

28. The method for behavior analysis according to claim 25, wherein selecting the key postures is via clustering a plurality of time-varied postures.

29. The method for behavior analysis according to claim 28, wherein a distance between the two selected cluster elements, z_i and z_j, is satisfied with:

30. The method for behavior analysis according to claim 28, wherein the key posture is satisfied with:

31. The method for behavior analysis according to claim 28, wherein the matching step is based on a string matching method.

32. The method for behavior analysis according to claim 31, wherein the string matching method comprises inserting, deleting and replacing.

33. The method for behavior analysis according to claim 31, wherein an edit distance between S_Q[0 . . . i] and S_D[0 . . . j] of two strings S_Q and S_D based on the string matching method is defined as:

D_SQ,SD^e(i,j)=min[D_SQ,SD^e(i−

1,j)+C_i,j^I), D_SQ,SD^e(i,j−

1)+C_i,j^D), D_SQ,SD^e(i−

1,j−

1)+α

(i−

1,j−

1)],

where C_i,j^I=ρ

+(1−

ρ

)α

(i−

1,j), C_i,j^D=ρ

+(1−

ρ

)α

(i,j−

1) and ρ

is smaller than 1.

34. A system for irregular human action analysis comprising:

an action recognition apparatus for integrating a plurality of postures to define an action; and

a judging apparatus for identifying whether the action is irregular.

35. The system for irregular human action analysis according to claim 34, wherein the analyzing apparatus comprises:

a posture recognition apparatus for recognizing the individual posture; and

a behavior recognition apparatus for distinguishing a behavior via a plurality of key postures selected from the postures.

36. The system for irregular human action analysis according to claim 35, wherein the posture recognition apparatus comprises:

a triangulation unit for dividing the posture of a body into a plurality of triangular meshes; and

a recognition unit for forming a spanning tree correspond to the meshes to recognize the posture.

37. The system for irregular human action analysis according to claim 35, the posture recognition apparatus further comprises a background subtraction unit to extract and define a boundary of the body.

38. The system for irregular human action analysis according to claim 37, wherein the background subtraction unit is a video.

39. The system for irregular human action analysis according to claim 35, wherein the meshes are triangles.

40. The system for irregular human action analysis according to claim 35, wherein the recognition unit is achieved via a skeleton analysis or a centroid context analysis.

41. The system for irregular human action analysis according to claim 35, wherein the skeleton analysis is defined via a graph search scheme.

42. The system for irregular human action analysis according to claim 34, wherein the behavior recognition apparatus comprises:

a clustering unit for selecting the key postures via clustering the postures iteratively for defining the various regular behaviors/actions;

a coding unit for translating the input postures into a plurality of correspondent symbols according to the selected key postures; and

a matching unit for unscrambling the correspondent symbols to distinguish the irregular/suspicious behavior.

43. The system for irregular human action analysis according to claim 42, wherein the clustering unit is programmable.

44. The system for irregular human action analysis according to claim 42, wherein the clustering unit is user-defined.

45. The system for irregular human action analysis according to claim 42, wherein the correspondent symbols are unscrambling via a string matching method.

46. The system for irregular human action analysis according to claim 42, wherein the matching unit is achieved via a symbol counting method for finding a series of irregular/suspicious posture patterns from input video sequences using the set of key postures.

47. The system for irregular human action analysis according to claim 34 further comprises a warning unit for sending an alarm if the behavior is irregular.

48. The system for irregular human action analysis according to claim 47, wherein the alarm is sent to a surveillance system.

49. The system for irregular human action analysis according to claim 47, wherein the warning unit is selected from an audio media, a color-highlighted video media or a light-emitted media.

50. A method for irregular human action analysis, comprising the steps of:

calculating the distance between a posture P and a set K of a plurality of selected key postures with;

51. The method for irregular human action analysis according to claim 50, wherein the threshold is programmable.

52. The method for irregular human action analysis according to claim 50, wherein the threshold is user-defined.

53. The method for irregular human action analysis according to claim 50, wherein defining the distance, dis(P,Q), between the two different postures, P and Q, is selected from the methods of a skeleton analysis or a centroid context analysis.

54. The method for irregular human action analysis according to claim 50, wherein the distance, dis(P,Q), between the two different postures, P and Q, defined via the skeleton analysis is satisfied with:

55. The method for irregular human action analysis according to claim 50, wherein the distance dis(P,Q), between the two different postures, P and Q, defined via the centroid context analysis is satisfied with:

56. The method for irregular human action analysis according to claim 50, wherein the distance dis(P,Q), between the two different postures, P and Q, defined via both the skeleton analysis and the centroid context analysis is satisfied with:

Error(P,Q)=wd_skeleton(P,Q)+(1−

w)d_CC(P,Q),

where Error(P,Q) is the integrated distance between the postures P and Q and w is a weight for balancing the two distances d_skeleton(P,Q) and d_CC(P,Q).