Classifying an object in a video frame

• US 8,134,596 B2
• Filed: 02/06/2006
• Issued: 03/13/2012
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Lei, Bangjun, Xu, Li-Qun
• Primary Examiner(s): Tran, Philip B
• Application Number:
  US11/883,420
• Publication Number:
  US 20100045799A1
• Time in Patent Office: 2,227 Days
• Field of Search: 348/169, 348/739, 382/103, 382/224
• US Class Current: 348/169
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

In a digital video surveillance system, a number of processing stages are employed to identify foreground regions representing moving objects in a video sequence. An object tracking stage (5) is also provided in order to identify a correspondence between candidate objects in a current frame and those that have already been identified in one or more previous frames. In this way, it is possible to calculate the path taken by the or each foreground object and to record this path information in a trajectory database. In order to improve tracking performance, the object tracking stage (5) employs a state transitional object management scheme which determines whether or not a particular object is tracked. The path information generated by the object tracking stage (5) can be displayed on a video monitor (15) of the surveillance system and/or made available to high level applications (7).

Claims

1. A method for selectively tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the method comprising:

(a) identifying a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identifying a plurality of candidate objects in a subsequent frame and determining whether there is a correspondence between any one of the candidate objects and the first object;

(c) in the event of correspondence between a candidate object and the first object, repeating step (b) for that candidate object over subsequent frames until a predetermined number of sequential correspondences are identified;

(d) changing the first status parameter to a second status parameter when said predetermined number of sequential correspondences are identified; and

(e) in response to the change from first to second status parameter, recording the intra-frame position of said object for subsequent frames;

wherein correspondence is determined in step (b) by;

calculating a first similarity score between each candidate object and the first object using a first cost function, calculating a second similarity score using a second cost function which is different from the first cost function, grouping the calculated similarity scores in accordance with the status parameter of the first object, analyzing each group in a predetermined order and selecting the closest match between the candidate objects and the first object.

2. The method according to claim 1, wherein, following step (d), step (b) is repeated for a candidate object in the next frame and, in the event that no correspondence is identified between the first object and said candidate object, the second status parameter is changed to one of a plurality of further parameters in accordance with a transition rule associated with said second status parameter.

3. The method according to claim 1, wherein the transition rule determines whether the first object has disappeared or merged with another candidate object in the frame.

4. The method according to claim 1, further comprising, in step (e), modifying the appearance of said subsequent frames to provide a visual indication of the intra-frame position of said object.

5. The method according to claim 4, wherein subsequent frames are modified so as to display each intra-frame position recorded in step (e) so as to provide a trail line indicating movement of said object in said subsequent frames.

6. The method according to claim 1, wherein, step (d) comprises changing the first status parameter to the second status parameter when (i) said sequential correspondences are identified and (ii) position characteristics of the object meet a set of predefined criteria.

7. The method according to claim 6, wherein step (ii) comprises assigning a motion factor ζ

_m to the first object based on its position characteristics over a plurality of frames, and classifying said first object as meeting the predefined criteria if the motion factor is above a predetermined threshold T_ζ

.

8. The method according to claim 7, wherein the motion factor ζ

_m is given by;

9. The method according to claim 1, further comprising providing a first set of data representing appearance features f_i^o of the first object, and extracting, for the candidate object in the subsequent frame, a second set of data representing appearance features f_i^b of that object, step (b) comprising combining the first set of appearance data with the second set of appearance data in a cost function C_ob thereby to generate a numerical parameter indicating the degree of correspondence between the first object and the or each new candidate region.

10. The method according to claim 9, wherein the cost function is given by the following expression:

11. The method according to claim 9, wherein the appearance features of the first object and candidate object include features relating to the frame position of the object/candidate object, and features relating to the shape of the object/candidate object.

12. The method according to claim 11, wherein in the event of a correspondence being identified in step (b), the appearance features f_i^o of the first object are updated using the appearance features f_i^b of the candidate object to which the first object corresponds.

13. A non-transitory computer-readable storage medium storing a set of instructions to cause a computer to perform steps for selectively tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the steps comprising:

(a) identifying a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identifying a plurality of candidate objects in a subsequent frame and determining whether there is a correspondence between any one of the candidate objects and the first object,

(c) in the event of correspondence between a candidate object and the first object, repeating step (b) for that candidate object over subsequent frames until a predetermined number of sequential correspondences are identified;

(d) changing the first status parameter to a second status parameter when said predetermined number of sequential correspondences are identified; and

(e) in response to the change from first to second status parameter, recording the intra-frame position of said object for subsequent frames;

wherein correspondence is determined in step (b) by;

calculating a first similarity score between each candidate object and the first object using a first cost function, calculating a second similarity score using a second cost function which is different from the first cost function, grouping the calculated similarity scores in accordance with the status parameter of the first object, analysing each group in a predetermined order and selecting the closest match between the candidate objects and the first object.

14. A video processing system for selectively tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the system being arranged, in use, to:

(a) identify a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identify a plurality of candidate objects in a subsequent frame and determining whether there is a correspondence between any one of the candidate objects and the first object;

(c) in the event of correspondence between a candidate object and the first object, repeat step (b) for that candidate object over subsequent frames until a predetermined number of sequential correspondences are identified;

(d) change the first status parameter to a second status parameter when said predetermined number of sequential correspondences are identified;

(e) in response to the change from first to second status parameter, record the intra-frame position of said object for subsequent frames; and

calculate a first similarity score between each candidate object and the first object using a first cost function, calculate a second similarity score using a second cost function which is different from the first cost function, group the calculated similarity scores in accordance with the status parameter of the first object, analyze each group in a predetermined order and select the closest match between the candidate objects and the first object.

15. A method of classifying an object in a video sequence comprising a plurality of frames, the method comprising:

(a) identifying a first object in a first frame and associating therewith a status parameter having one of a plurality of predetermined states, each state having a different transition rule associated therewith;

(b) identifying at least one candidate object in a subsequent frame;

(c) comparing the at least one candidate object with the first object to determine if there is a correspondence therebetween; and

(d) updating the status parameter of the first object in accordance with its associated transition rule, said transition rule indicating which of the predetermined states the status parameter should be transited to dependent on whether a correspondence was identified in step (c)

wherein correspondence in step (c) is determined by;

calculating a first similarity score between each candidate object and the first object using a first cost function, calculating a second similarity score using a second cost function which is different from the first cost function, grouping the calculated similarity scores in accordance with the status parameter of the first object, analyzing each group in a predetermined order and selecting the closest match between the candidate objects and the first object.

16. The method according to claim 15, further comprising repeating steps (b) to (d) for a plurality of subsequent frames of the video sequence.

17. The method according to claim 15, wherein the transition rule associated with the state causes the status parameter to maintain its current state if there is no correspondence identified in step (c).

18. The method according claim 15, wherein the status parameter has a new state and a real state, the transition rule associated with the new state causing the status parameter to be changed to the real state in the event that a correspondence is identified in step (c), the method further comprising the step of recording the position change between the first object and the corresponding candidate region only when the status parameter is in the real state.

19. The method according to claim 18, wherein the status parameter is changed to the real state only if a correspondence is identified in a plurality of sequential frames in step (c).

20. A video processing apparatus comprising:

an input for receiving frames of a video sequence;

an object queue arranged to store data representing objects identified in one or more frames and, associated with each object, a status parameter representing one of a plurality of predetermined states;

a video analysis unit arranged to receive a frame having one or more candidate objects therein, and to determine whether the or each candidate object matches an object in the object queue; and

a state transition controller arranged to update the status parameter of each object in the object queue in accordance with a transition rule corresponding to the state of the object'"'"'s status parameter, said transition rule indicating which of the predetermined states the status parameter should be transited to;

wherein the video analysis unit is further arranged to;

calculate a first similarity score between each candidate object and the first object using a first cost function, calculate a second similarity score using a second cost function which is different from the first cost function, group the calculated similarity scores in accordance with the status parameter of the first object, analyse each group in a predetermined order and select the closest match between the candidate objects and the first object.

21. Apparatus according to claim 20, wherein the status parameter represents one of a new and real state, the state transition controller being arranged to transit the status parameter from the new state to the real state if the video analysis unit determines that a match is made between an object in the object queue and a candidate object in the received frame.

22. Apparatus according to claim 21, wherein the video analysis unit is further arranged to record the change in frame position of an object in the object queue and a candidate object to which it is matched in the event that the state transition controller updates the status parameter, associated with said object, to the real state.

23. Apparatus according to claim 22, further comprising a display controller arranged to display, on a video display unit, a path line indicative of the change in frame position.

24. A method for tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the method comprising:

(a) identifying a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identifying a candidate object in a subsequent frame and determining whether there is a correspondence between the candidate object and the first object;

(c) in the event of correspondence, repeating steps (b) and (c) for further subsequent frames until a predetermined number of sequential correspondences are identified;

(d) changing the first status parameter to a second status parameter when said sequential correspondences are identified; and

(e) in response to the change from first to second status parameter, recording the intra-frame position of said object for subsequent frames;

wherein, step (d) comprises changing the first status parameter to the second status parameter when (i) said sequential correspondences are identified and (ii) position characteristics of the object meet a set of predefined criteria; and

wherein step (ii) comprises assigning a motion factor ζ

_m to the first object based on its position characteristics over a plurality of frames, and classifying said first object as meeting the predefined criteria if the motion factor is above a predetermined threshold T_ζ

.

25. A method for tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the method comprising:

(a) identifying a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identifying a candidate object in a subsequent frame and determining whether there is a correspondence between the candidate object and the first object;

(c) in the event of correspondence, repeating steps (b) and (c) for further subsequent frames until a predetermined number of sequential correspondences are identified;

(d) changing the first status parameter to a second status parameter when said sequential correspondences are identified;

(e) in response to the change from first to second status parameter, recording the intra-frame position of said object for subsequent frames; and

providing a first set of data representing appearance features f_i^o of the first object, and extracting, for the candidate object in the subsequent frame, a second set of data representing appearance features f_i^b of that object, step (b) comprising combining the first set of appearance data with the second set of appearance data in a cost function C_ob thereby to generate a numerical parameter indicating the degree of correspondence between the first object and the or each new candidate region.