Method and apparatus for tracking moving objects in pictures

• US 20030169340A1
• Filed: 03/06/2003
• Published: 09/11/2003
• Est. Priority Date: 03/07/2002
• Status: Active Grant

• Current Assignee: Socionext Inc., The Foundation for the Promotion of Industrial Science
• Original Assignee: Fujitsu Limited
• Inventors: Kamijo, Shunsuke, Sakauchi, Masao
• Granted Patent: US 7,149,328 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

Each frame of time-series pictures shot by a camera is converted to a spatial differential frame picture by the following equations:

H(i, j)=Σ|G(i+di, j+dj)−G(i, j)|/(Gi,j,max/Gmax),

I=Gmax/{1+exp(−0.02(H−80))},

where Σ denotes a sum over di=−1 to 1 and dj=−1 to 1, Gi,j,max denotes the maximum pixel value of 3×3 pixels having its center at the i-th row and j-th column, and Gmax denotes the assumable maximum value 255 of the pixel value G(i, j). Moving objects at a time t is discriminated based on a correlation between spatial differential frame pictures at times (t−1) and t and a discrimination result of moving objects included in the spatial differential frame picture at the time (t−1).

Claims

1. A moving object tracking method of processing first time-series pictures to track moving objects therein, the method comprising the steps of:

(a) generating second time-series pictures by converting the first time-series pictures to respective spatial differential frame pictures; and

(b) discriminating moving objects at a time t2 based on a correlation between one of the spatial differential frame pictures at a time t1 adjacent to the time t2 and another of the spatial differential frame pictures at the time t2, and also based on a discrimination result of moving objects included in said one of the spatial differential frame pictures at the time t1.

2. The moving object tracking method according to claim 1, wherein in the step (a), a value of a pixel of interest in any of the spatial differential frame pictures is determined by converting a value of a corresponding pixel of interest of corresponding one of the first time-series pictures to a value I associated with a differential value between a value of the corresponding pixel of interest and values of its neighboring pixels.

3. The moving object tracking method according to claim 2, wherein in the step (a), the value I associated with the differential value is associated with a value H proportional to a value obtained by dividing the differential value by a maximum value among the corresponding pixel of interest and the neighboring pixels.

4. The moving object tracking method according to claim 3, wherein, if the value H is within a predetermined range, then the value I is determined by an approximate linear transformation of the value H, or else, the value I is determined by converting the value H to a value which is suppressed so as not to exceed an assumable maximum value Gmax of a pixel of the spatial differential frame picture.

5. The moving object tracking method according to claim 4, wherein on the basis of a sigmoid function

f(H)=1/{1+exp[−

β

(H−

α

)]},

the value I=Gmax·

f(H) is determined,

where H is said proportional value, and α and

β

are positive constants.

6. The moving object tracking method according to claim 5, wherein the constant β

is a value in a range of 0.05 to 0.50.

7. The moving object tracking method according to claim 1, wherein in the step (b),

each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

8. The moving object tracking method according to claim 2, wherein in the step (b),

each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

9. The moving object tracking method according to claim 3, wherein in the step (b),

each of the spatial differential frame pictures at the times t1and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

10. The moving object tracking method according to claim 4, wherein the object map at the time t2 is produced by the steps of:

assuming that a block i in the object map at the time t2 has an identification code IDi and a moving vector Vi;

calculating a unit correlation between an image of the block i in the spatial differential frame picture at the time t2 and an image of a box in the spatial differential frame picture at the time t1, the box being determined by moving a box of the block i by a vector −

Vi; and

calculating a total sum of the unit correlations over all blocks at the time t2 so as to produce the object map at the time t2 at a maximum value among the total sums.

11. The moving object tracking method according to claim 5, wherein in the step (b),

each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

12. A moving object tracking apparatus of processing a first time-series pictures to track moving objects therein, the apparatus comprising:

a picture recording section for storing the first time-series pictures;

a picture conversion section for generating second time-series pictures by converting the first time-series pictures to respective spatial differential frame pictures, and for storing the generated second time-series pictures into the picture recording section; and

a picture processing section for discriminating moving objects at a time t2 based on a correlation between one of the spatial differential frame pictures at a time t1 adjacent to the time t2 and another of the spatial differential frame pictures at the time t2, and also based on a discrimination result of moving objects included in said one of the spatial differential frame pictures at the time t1.

13. The moving object tracking apparatus according to claim 12, wherein the picture conversion section determines a value of a pixel of interest in any of the spatial differential frame pictures by converting a value of a corresponding pixel of interest of corresponding one of the first time-series pictures to a value I associated with a value H proportional to a value obtained by dividing a differential value between the value of the corresponding pixel of interest and values of its neighboring pixels by a maximum value among the corresponding pixel of interest and the neighboring pixels.

14. The moving object tracking apparatus according to claim 13, wherein the picture conversion section, determines the value I by an approximate linear transformation of the value H if the value H is within a predetermined range, or else, the picture conversion section determines the value I by converting the value H to a value which is suppressed so as not to exceed an assumable maximum value Gmax of a pixel of the spatial differential frame picture.

15. The moving object tracking apparatus according to claim 13, wherein the picture conversion section, on the basis of a sigmoid function

f(H)=1/{1+exp[β

(H−

α

)]},

determines the value I=Gmax·

f(H), where H is said proportional value, and α and

β

are positive constants.

16. The moving object tracking apparatus according to claim 15, wherein the constant β

is a value in a range of 0.05 to 0.50.

17. The moving object tracking apparatus according to claim 12, wherein in the picture processing section, each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

18. The moving object tracking apparatus according to claim 13, wherein in the picture processing section, each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

19. The moving object tracking apparatus according to claim 14, wherein in the picture processing section,

each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

20. The moving object tracking apparatus according to claim 15, wherein in the picture processing section,

each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

21. The moving object tracking apparatus according to claim 16, wherein in the picture processing section,

each of the spatial differential frame pictures at the times t1 and t2 is divided into blocks each including a plurality of pixels;

the moving object discrimination result at the time t1 is an object map at the time t1 which includes an identification code of a moving object assigned to at least one of the blocks, said one including a portion of the moving object, and also includes a moving vector of said at least one of the blocks; and

an object map at the time t2 is produced to discriminate a moving object at the time t2.

22. The moving object tracking method according to claim 17, wherein the picture processing section:

assumes that a block i in the object map at the time t2 has an identification code IDi and a moving vector Vi;

calculates a unit correlation between an image of the block i in the spatial differential frame picture at the time t2 and an image of a box in the spatial differential frame picture at the time t1, the box being determined by moving a box of the block i by a vector −

Vi; and

calculates a total sum of the unit correlations over all blocks at the time t2 so as to produce the object map at the time t2 at a maximum value among the total sums.