Classifying an Object in a Video Frame

• US 20100045799A1
• Filed: 02/06/2006
• Published: 02/25/2010
• Est. Priority Date: 02/04/2005
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Xu, Li-Qun, Lei, Bangjun
• Granted Patent: US 8,134,596 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

In a digital video surveillance system, a number of processing stages are employed to identify foreground regions representing moving objects in a video sequence. An object tracking stage (5) is also provided in order to identify a correspondence between candidate objects in a current frame and those that have already been identified in one or more previous frames. In this way, it is possible to calculate the path taken by the or each foreground object and to record this path information in a trajectory database. In order to improve tracking performance, the object tracking stage (5) employs a state transitional object management scheme which determines whether or not a particular object is tracked. The path information generated by the object tracking stage (5) can be displayed on a video monitor (15) of the surveillance system and/or made available to high level applications (7).

Claims

1. A method for tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the method comprising;

(a) identifying a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identifying a candidate object in a subsequent frame and determining whether there is a correspondence between the candidate object and the first object;

(c) in the event of correspondence, repeating steps (b) and (c) for further subsequent frames until a predetermined number of sequential correspondences are identified;

(d) changing the first status parameter to a second status parameter when said sequential correspondences are identified; and

(e) in response to the change from first to second status parameter, recording the intra-frame position of said object for subsequent frames.

2. A method according to claim 1, wherein, following step (d), step (b) is repeated for a candidate object in the next frame and, in the event that no correspondence is identified between the first object and said candidate object, the second status parameter is changed to one of a plurality of further parameters in accordance with a transition rule associated with said second status parameter.

3. A method according to claim 1, wherein the transition rule determines whether the first object has disappeared or merged with another candidate object in the frame.

4. A method according to claim 1, further comprising, in step (e), modifying the appearance of said subsequent frames to provide a visual indication of the intra-frame position of said object.

5. A method according to claim 4, wherein subsequent frames are modified so as to display each intra-frame position recorded in step (e) so as to provide a trail line indicating movement of said object in said subsequent frames.

6. A method according to claim 1, wherein, step (d) comprises changing the first status parameter to the second status parameter when (i) said sequential correspondences are identified and (ii) position characteristics of the object meet a set of predefined criteria.

7. A method according to claim 6, wherein step (ii) comprises assigning a motion factor ζ

_m to the first object based on its position characteristics over a plurality of frames, and classifying said first object as meeting the predefined criteria if the motion factor is above a predetermined threshold T_ζ

.

8. A method according to claim 7, wherein the motion factor ζ

_m is given by;

${\mathrm{Ϛ<br><br>}}_m=\left(\frac{{\mathrm{\sigma <br><br>}}_{\mathrm{cx}}^2}{{\mathrm{\sigma <br><br>}}_{\mathrm{vx}}^2+\mathrm{\tau <br><br>}}+\frac{{\mathrm{\sigma <br><br>}}_{\mathrm{cy}}^2}{{\mathrm{\sigma <br><br>}}_{\mathrm{vy}}^2+\mathrm{\tau <br><br>}}\right)/2$

where σ

_cx² and σ

_cy² are the positional variances of the first object in the x and y directions, respectively, σ

_vx² and σ

_vy² are the velocity variances in x and y directions, respectively, and τ

is a predetermined constant.

9. A method according to claim 1, further comprising providing a first set of data representing appearance features f_i^o of the first object, and extracting, for the candidate object in the subsequent frame, a second set of data representing appearance features f_i^b of that object, step (b) comprising combining the first set of appearance data with the second set of appearance data in a cost function C_ob thereby to generate a numerical parameter indicating the degree of correspondence between the first object and the or each new candidate region.

10. A method according to claim 11, wherein the cost function is given by the following expression:

$C_{\mathrm{ob}}=\sqrt{\underset{i=1}{\overset{n}{\sum <br><br>}}<br><br>\frac{{\left(f_i^o-f_i^b\right)}^2}{{\mathrm{\sigma <br><br>}}_i^2}}$

where f_i^o represents an appearance feature of the first object, f_i^b represents an appearance feature of the candidate region, σ

_i² is the variance of f_i^o over a predetermined number of frames and n is the number of appearance features in the first and second data sets.

11. A method according to claim 9, wherein the appearance features of the first object and candidate object include features relating to the frame position of the object/candidate object, and features relating to the shape of the object/candidate object.

12. A method according to claim 11, wherein in the event of a correspondence being identified in step (b), the appearance features f_i^o of the first object are updated using the appearance features f_i^b of the candidate object to which the first object corresponds.

13. A computer program stored on a computer-readable medium and comprising a set of instructions to cause a computer to perform the method according to claim 1.

14. A video processing system for selectively tracking an object appearing in a video sequence comprising a plurality of frames, each frame comprising a plurality of pixels, the system being arranged, in use, to;

(a) identify a first object in a first frame and associating therewith a first status parameter indicative of a non-tracking condition;

(b) identify a candidate object in a subsequent frame and determining whether there is a correspondence between the candidate object and the first object;

(c) in the event of correspondence, repeat steps (b) and (c) for further subsequent frames until a predetermined number of sequential correspondences are identified;

(d) change the first status parameter to a second status parameter when said sequential correspondences are identified; and

(e) in response to the change from first to second status parameter, record the intra-frame position of said object for subsequent frames.

15. A method of classifying an object in a video sequence comprising a plurality of frames, the method comprising;

(a) identifying a first object in a first frame and associating therewith a status parameter having one of a plurality of predetermined states, each state having a different transition rule associated therewith;

(b) identifying at least one candidate object in a subsequent frame;

(c) comparing the or each candidate object with the first object to determine if there is a correspondence therebetween; and

(d) updating the status parameter of the first object in accordance with its associated transition rule, said transition rule indicating which of the predetermined states the status parameter should be transited to dependent on whether a correspondence was identified in step (c).

16. A method according to claim 15, further comprising repeating steps (b) to (d) for a plurality of subsequent frames of the video sequence.

17. A method according to claim 15, wherein the transition rule associated with the a state causes the status parameter to maintain its current state if there is no correspondence identified in step (c).

18. A method according to claim 15, wherein the status parameter has a new state and a real state, the transition rule associated with the new state causing the status parameter to be changed to the real state in the event that a correspondence is identified in step (c), the method further comprising the step of recording the position change between the first object and the corresponding candidate region only when the status parameter is in the real state.

19. A method according to claim 18, wherein the status parameter is changed to the real state only if a correspondence is identified in a plurality of sequential frames in step (c).

20. Video processing apparatus comprising:

an input for receiving frames of a video sequence;

an object queue arranged to store data representing objects identified in one or more frames and, associated with each object, a status parameter representing one of a plurality of predetermined states;

video analysis means arranged to receive a frame having one or more candidate objects therein, and to determine whether the or each candidate object matches an object in the object queue; and

a state transition controller arranged to update the status parameter of each object in the object queue in accordance with a transition rule corresponding to the state of the object'"'"'s status parameter, said transition rule indicating which of the predetermined states the status parameter should be transited to.

21. Apparatus according to claim 20, wherein the status parameter represents one of a new and real state, the state transition controller being arranged to transit the status parameter from the new state to the real state if the video analysis means determines that a match is made between an object in the object queue and a candidate object in the received frame.

22. Apparatus according to claim 21, wherein the video analysis means is further arranged to record the change in frame position of an object in the object queue and a candidate object to which it is matched in the event that the state transition controller updates the status parameter, associated with said object, to the real state.

23. Apparatus according to claim 22, further comprising a display controller arranged to display, on a video display unit, a path line indicative of the change in frame position.

24. A state transition database for use with a video processing system storing information representing one or more objects identified in a video frame, each object being classified to one of a plurality of predefined object states, the database defining a plurality of different transition rules for each respective object state, the transition rules defining two or more updated states to which the object is transited depending on whether the object is matched with an object appearing in a subsequent frame.