Salient motion detection system, method and program product therefor

• US 7,747,075 B2
• Filed: 06/20/2005
• Issued: 06/29/2010
• Est. Priority Date: 06/20/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Tian, Ying-Li
• Primary Examiner(s): Seth; Manav
• Application Number:
  US11/156,508
• Publication Number:
  US 20060285724A1
• Time in Patent Office: 1,835 Days
• Field of Search: 382/100, 382/103, 382106-107, 382171-173, 382/180, 382/190, 382/195, 37524001-24029, 375347-350, 348135-159
• US Class Current: 382/173
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An apparatus and method of identifying objects with salient motion in a scene and program product therefor. A region of change is identified within a scene. Optical flow is determined for the region of change. Pixels moving in a common direction are identified as seed pixels. Seed pixels are expanded to form pixel clusters. Temporal imaging results are fused to identify those objects with salient motion.

Claims

1. A method of identifying objects with salient motion in a scene, said method comprising of the steps of:

a) identifying a region of change within a scene, wherein identifying said region of change develops a temporal filter;

b) determining optical flow for said region;

c) identifying seed pixels moving in a common direction in said region;

d) expanding identified said seed pixels, pixel clusters being formed from expanded said seed pixels; and

e) fusing results of steps (a), (b) and (c), fused said results identifying objects having salient motion.

2. A method as in claim 1, wherein said scene is a video stream including a plurality of frames, I(x, y, t).

3. A method as in claim 2, wherein the step (a) of identifying said region of change comprises subtracting a subsequent image.

4. A method as in claim 2, wherein the step (a) of identifying said region of change for each image comprises finding a difference image I_difference(x, y, t) using a weighted accumulation, I_accum(x, y, t), with a fixed weight for each new observation.

5. A method as in claim 2, wherein said temporal filter is determined by calculating optical flow for said plurality of frames over a given period of time, [t, t+n].

6. A method as in claim 5, wherein calculating said optical flow comprises identifying each of a plurality of pixels having a monotonously changing position.

7. A method as in claim 6, wherein said plurality of said identified pixels in [t, t+n] comprise positive X-component pixels with positive X-component optical flow (P_iX), negative X-component with negative optical flow (N_iX), positive Y-component pixels with positive Y-component optical flow (P_iY) and negative Y-component with negative optical flow (N_iY).

8. A method as in claim 2, wherein the step (d) of fusing results comprises combining an image of temporal difference (I_differnce(x, y, t)) from step (a), temporal filtered images (I_X-temporal(x, y, t) and I_Y-temporal(x, y, t)) from step (b) and a region motion information (I_RegionMotion(x, y, t)) from step (c), a salient motion image (I_salient(x, y, t)) resulting from said combination having the form:

I_salient(x, y, t)=I_difference(x, y, t)I [I_X-temporal(x, y, t)Y I_Y-temporal(x, y, t)Y I_RegionMotion(x, y, t)].

9. A computer program product for identifying objects with salient motion in a scene, said computer program product comprising a computer usable medium having computer readable program code thereon, said computer readable program code comprising:

computer readable program code means for identifying a region of change within a plurality of sequential frames, I(x, y, t), wherein identifying said region of change develops a temporal filter;

computer readable program code means for determining optical flow for said region of change;

computer readable program code means for identifying seed pixels in said region moving in a common direction;

computer readable program code means for expanding identified said seed pixels, pixel clusters being formed from expanded said seed pixels; and

computer readable program code means for fusing temporal imaging results.

10. A computer program product as in claim 9, wherein the computer readable program code means for identifying said region of change for each image comprises computer readable program code means for subtracting a subsequent image.

11. A computer program product as in claim 9, wherein the computer readable program code means for identifying said region of change for each image comprises computer readable program code means for finding a difference image I_difference(x, y, t) using a weighted accumulation, I_accum(x, y, t), with a fixed weight for each new observation.

12. A computer program product as in claim 10, wherein computer readable program code means for developing said temporal filter comprises computer readable program code means for calculating optical flow for said plurality of frames over a given period of time, [t, t+n].

13. A computer program product as in claim 12, wherein computer readable program code means for calculating said optical flow comprises computer readable program code means for identifying each of a plurality of pixels having a monotonously changing position.

14. A computer program product as in claim 13, wherein said plurality of said identified pixels in [t, t+n] comprise positive X-component pixels with positive X-component optical flow (P_iX), negative X-component with negative optical flow (N_iX), positive Y-component pixels with positive Y-component optical flow (P_iY) and negative Y-component with negative optical flow (N_iY).

15. A computer program product as in claim 14, wherein said computer readable program code means for identifying each of a plurality of pixels identifies pixels as belonging to an object with salient motion for a selected threshold T₁ and comprises computer readable program code means for:

representing optical flow of frames I_t, I_t+1, . . . I_t+n in [t, t+n] as F₁, F₂, . . . F_n having an X-component, F_1,x, F_2,x, . . . F_n,x and a Y-component is F_1,y, F_2,y, . . . F_n,y, respectively, for i ⊂

[1,n]; and

, initially, P_iX=0, and N_iX=0; and

otherwise

$P_{\mathrm{iX}}=\{\begin{array}{c}{P}_{\left(i-1\right)<br><br>X}+1,<br><br>\mathrm{if}<br><br><br><br>\left(F_{i,X}><br><br>0\right)<br><br>\\ P_{\left(i-1\right)<br><br>X},<br><br>\mathrm{otherwise}\end{array};<br><br><br><br>\mathrm{and}<br><br><br><br>N_{\mathrm{iX}}=\{\begin{array}{c}{N}_{\left(i-1\right)<br><br>X}+1,<br><br>\mathrm{if}<br><br><br><br>\left(F_{i,X}<<br><br>0\right)\\ N_{\left(i-1\right)<br><br>X},<br><br><br><br>\mathrm{otherwise}\end{array};<br><br>$

 

said identified pixels being identified by P_nX≧

T₁, N_nX≧

T₁, P_nY≧

T₁ and N_nY≧

T₁.

16. An apparatus for identifying objects with salient motion in a scene, said apparatus comprising:

means for identifying a region of change within a plurality of sequential frames, I(x, y, t), wherein identifying said region of change develops a temporal filter;

means for determining optical flow for said region of change over a given period of time, [t, t+n];

means for identifying seed pixels in said region moving in a common direction;

means for expanding identified said seed pixels, pixel clusters being formed from expanded said seed pixels; and

means for fusing temporal imaging results.

17. A apparatus as in claim 16, wherein the means for identifying said region of change for each image comprises:

means for finding a difference image I_difference(x, y, t) using a weighted accumulation, I_accum(x, y, t), with a fixed weight for each new observation.