ACTIVITY RECOGNITION METHOD

• US 20130182105A1
• Filed: 07/18/2012
• Published: 07/18/2013
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: National Taiwan University of Science and Technology
• Original Assignee: National Taiwan University of Science & Technology (Government of Taiwan)
• Inventors: Fahn, Chin-Shyurng, Liu, Shih-Min
• Granted Patent: US 8,928,816 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 10/7625 Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An activity recognition method, for recognizing continuous activities of several moving objects in the foreground of a video, includes: capturing and processing a training video to get a contour of a moving object; extracting a minimum bounding box of the contour in order to get parameters then transfer to feature vectors; constructing a decision tree model based on support vector machines (SVMs), for classifying the activities of the moving object according to the parameter and the feature vector of the training video; capturing and processing a testing video to get other parameters and using several formulas to generate feature vectors, and executing an algorithm for recognizing the activities of several moving objects in the foreground of the testing video. Said feature vectors are transformed from the parameters that in the testing and training videos. Via above descriptions, the method can recognize activities of foreground objects in the testing video.

Claims

1. An activity recognition method, comprising steps of:

capturing a training video having a first foreground moving object and a first background, wherein the first foreground moving object has a first contour, the steps of capturing the training video comprising;

processing the training video to distinguish the first contour from the first background, wherein the first foreground moving object has a plurality of activities;

defining a first minimum bounding box for the first contour;

calculating a first parameter according to the first minimum bounding box; and

transforming the first parameter into a first feature vector;

constructing a decision tree model having a plurality of support vector machines (SVMs) for classifying the activities of the first foreground. moving object according to the first parameter and the first feature vector in one of the support vector machines;

capturing a testing video having a second foreground moving object and. a second background, wherein the second foreground moving object has a second contour, the steps of capturing the testing video comprising;

processing the testing video to distinguish the second contour from the second background;

defining a second minimum bounding box of the second contour;

calculating a second parameter according to the second minimum bounding box, wherein the second parameter comprises a center value of the second minimum bounding box;

providing an algorithm to judge whether the second foreground moving object is the same as the first foreground moving object according to a trajectory in form of the center value varying with the time; and

transforming the second parameter into a second feature vector; and

each of the support vector machines comparing the first feature vector and the second feature vector in sequence according to the, decision tree model to recognize an activity of the second foreground moving object.

5. The activity recognition method of claim 1, wherein the testing video comprises a plurality of frames including a first frame, a second frame and a third frame, which appear in sequence in the testing video, the steps of processing the testing video comprising:

providing a temporary moving object in the testing video;

providing an averaging background method to distinguish the temporary moving object from each of the first background and the second background;

executing the averaging background method to calculate a first absolute difference value between each of the three frames and the first frame respectively;

providing a maximum variance between clusters method to generate a noisy moving object according to the first absolute difference value; and

providing a logic operation to combine the temporary moving object and the noisy moving object into each of the first foreground moving object and the second foreground moving object.

6. The activity recognition method of claim 5, wherein the steps of executing the averaging background method comprise:

calculating a second absolute difference value between the first frame and the second frame, and a third absolute difference value between the second frame and the third frame respectively;

providing an accumulation step for accumulating the second and the third absolute difference values in sequence to calculate an average value thereof, and for generating a number of accumulating times;

judging whether the number of accumulating times reaches a threshold; and

if the number of accumulating times reaches a threshold, constructing a statistic model based on the second and the third absolute difference values.

7. The activity recognition method of claim 5, wherein each of the first foreground moving object and the second foreground moving object has a plurality of noise pixels, and the steps of providing the logic operation comprise:

providing an erosion operation to remove the noise pixels in one of the first foreground moving object and the second foreground moving object;

providing a dilation operation to dilate one of the first foreground moving object and the second foreground moving object after removing the noise pixels thereof; and

contouring one of the first foreground moving object and the second foreground moving object after the erosion operation and the dilation operation to generate one of the first contour and the second contour.

8. The activity recognition method of claim 1, wherein the steps of providing the algorithm comprise:

initializing the second foreground moving object to create a buffer space with an variable;

judging whether the variable is empty;

if the variable is empty, setting the variable equal to the center value of the second minimum bounding box; and

assigning an identification to the second foreground moving object.

9. The activity recognition method of claim 8, wherein the step of judging whether the variable is empty comprises:

if the variable is not empty, judging whether the variable in the buffer is space is equal to the center value of the second minimum bounding box to determine if the second foreground moving object is moving or doing fixed-point activity; and

if the variable in the buffer space is equal to the center value of the second minimum bounding box, assigning the identification to the second foreground moving object, and recognizing the second foreground moving object is doing the fixed-point activity.

10. The activity recognition method of claim 9, wherein the step of judging whether the variable in the buffer space is equal to the center value of the second minimum bounding box comprises:

if the variable in the buffer space is not equal to the center value of the second minimum bounding box, judging the absolute value of the center value subtracting the variable in the buffer space reaches a minimum value;

if the absolute value of the center value subtracting the variable in the buffer space is equal to the minimum value, resetting the variable in the buffer space as the center value and assigning the identification to the second foreground moving object; and

if the absolute value of the center value subtracting the variable in the buffer space is not equal to the minimum value, executing the algorithm again to assign another identification.

11. The activity recognition method of claim 1, wherein each of the first parameter and the second parameter comprises an initial value, a minimum width, a minimum height and a center value.

12. The activity recognition method of claim 11, wherein each of the first feature vector and the second feature vector comprises a ratio, a displacement value, a speed value and a delay value, and all of the ratio, the displacement value, the speed value and the delay value are calculated according to the initial value, the minimum width, the minimum height and the center value.

13. The activity recognition method of claim 12, wherein the activity of the second foreground moving is selected from the group consisting of running, walking, jumping, sitting, squatting and lying.

2. The activity recognition method of Mimi, wherein the training video comprises a plurality of frames including a first frame, a second frame and a third frame, which appear in sequence in the training video, the steps of processing the training video comprising:

providing a temporary moving object in the training video;

providing an averaging background method to distinguish the temporary moving object from each of the first background and the second background;

executing the averaging background method to calculate a first absolute difference value between each of the three frames and the first frame respectively;

providing a maximum variance between clusters method to generate a noisy moving object according to the first absolute difference value; and

providing a logic operation to combine the temporary moving object and the noisy moving object into each of the first foreground moving object and the second foreground moving object.

3. The activity recognition method of claim 2, wherein the steps of executing the averaging background method comprise:

calculating a second absolute difference value between the first frame and the second frame, and a third absolute difference value between the second frame and the third frame respectively;

providing an accumulation step for accumulating the second and the third absolute difference values in sequence to calculate an average value thereof, and for generating a number of accumulating times;

judging whether the number of accumulating times reaches a threshold; and

if the number of accumulating times reaches a threshold, constructing a statistic model based on the second and the third absolute difference values.

4. The activity recognition method of claim 2, wherein each of the first foreground moving object and the second foreground moving object has a plurality of noise pixels, and the steps of providing the logic operation comprise:

providing an erosion operation to remove the noise pixels in one of the first foreground moving object and the second foreground moving object;

providing a dilation operation to dilate one of the first foreground moving object and the second foreground moving object after removing the noise pixels thereof; and

contouring one of the first foreground moving object and the second foreground moving object after the erosion operation and the dilation operation to generate one of the first contour and the second contour.