Device for and method of automatically tracking a moving object

• US 6,993,158 B2
• Filed: 08/06/2002
• Issued: 01/31/2006
• Est. Priority Date: 08/07/2001
• Status: Expired due to Term

• Current Assignee: Hanwha Techwin Co., Ltd. (Hanwha Corporation)
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Iizuka, Hiroshi, Cho, Jae-soo, Tanaka, Kazuhiro
• Primary Examiner(s): Patel, Kanjibhai
• Assistant Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US10/212,142
• Publication Number:
  US 20030035051A1
• Time in Patent Office: 1,274 Days
• Field of Search: 382/103, 382/107, 382/236, 348/236, 348/14.1, 348/97, 348/154, 348/155, 348/208.1, 348/208.2, 348/208.4, 348/208.13, 348/252, 348/402.1, 348/451, 348/452, 348/699, 348/700, 356/3, 356/27, 734/88
• US Class Current: 382/103
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

An automatic moving object tracking device and a method thereof are capable of tracking a moving object through a signal process of an input image signal of each frame. The moving object tracking device converts a pre-processed input image signal into a binary disturbance image signal, and calculates a motion value of each pixel through an initial tracking window. After locating the tracking window at the pixel where the motion value is a maximum value, a size of the tracking window is adjusted so that the tracking window can include the moving object therein. Next, based on acquired information about the moving object and the tracking window and previously acquired information about previously frames, a pixel in a following frame is estimated to be a next location of the moving object. The tracking window is positioned at the estimated pixel. If the tracking result is satisfied, the tracking of the moving object continues.

Claims

1. An automatic moving object tracking device, comprising:

a photographing unit photographing a monitoring area;

a binary disturbance image generating unit generating a binary disturbance image signal from an input image signal of each frame acquired through the photographing unit;

a moving object acquiring unit acquiring from the binary disturbance image signal information about a moving object by using a moving window as an initial tracking window having a predetermined size, and adjusting the initial tracking window in size so that the binary disturbance image signal contains the moving object therein;

a moving object automatic tracking unit estimating information about a location in a following frame to which a center of the moving object is to be moved based on currently acquired information and previously acquired information about the center of the moving object, moving a center of the moving window to the estimated location where the center of the moving object is estimated to be moved to, and acquiring actual information about an actual center of the moving object in the moving window and the size of the tracking window; and

a tracking status determining unit comparing the actual information about the moving object with the estimated information about the moving object, and determining a tracking status of the moving object based on a resultant error range of the actual information and the estimated information, the actual information about the moving object being acquired through the following frame.

2. The automatic moving object tracking device of claim 1, further comprising:

an input image pre-processing unit improving a processing speed of the input image signal that is input through a photographing process of the photographing unit and pre-processing the input image signal to eliminate a noise from the input image signal.

3. The automatic moving object tracking device of claim 2, wherein the input image pre-processing unit comprises:

a resolution adjusting unit reducing a resolution of the input image signal; and

a filter eliminating the noise from the input image signal having the reduced resolution.

4. The automatic moving object tracking device of claim 1, wherein the binary disturbance image generating unit comprises:

a disturbance field generating unit generating a disturbance field through a difference between the input image signal of a current frame and an average of previous frames;

a headlight preventing unit comparing respective ones of the input image signal and the average of the previous frames with a threshold, and performing a logic-operation on a result of the comparison; and

an image division unit determining a binary value for each pixel according to a result of comparing an absolute value of each pixel of the disturbance field with a disturbance threshold and also according to a result of the logic-operation of the headlight prevention unit.

5. The automatic moving object tracking device of claim 4, wherein the moving object acquiring unit comprises:

a moving disturbance function (MDF) calculating unit determining a moving disturbance function (MDF) value of a pixel corresponding to a center coordinate by adding up binary values of first through last coordinates in the initial tracking window of the binary disturbance image signal; and

a tracking window adjusting unit adjusting the size of the tracking window by calculating the center coordinate of an actual moving object and a size of the moving object with reference to a pixel having a greatest MDF value.

6. The automatic moving object tracking device of claim 5, wherein the MDF calculating unit calculates MDF values for pixels in parallel with each other.

7. The automatic moving object tracking device of claim 5, wherein the tracking window adjusting unit adjusts the moving window so that the moving object is included in the input image signal with respect to a coordinate of a pixel having a greatest MDF value.

8. The automatic moving object tracking device of claim 7, wherein the tracking window adjusting unit determines that there does not exist the moving object when the greatest value of the MDF value calculated by the MDF calculating unit is below an MDF threshold, and the tracking window adjusting unit does not adjust the moving window upon the determination on the moving object.

9. The automatic moving object tracking device of claim 7, wherein the tracking window adjusting unit adjusts the size of the moving window by using a size rate of a current moving window and a following moving window of the moving object.

10. The automatic moving object tracking device of claim 1, wherein the tracking status determining unit determines a tracking failure when an error of the estimated center value and a measured center value of the moving object equals or is greater than a predetermined range, or when the number of pixels of the binary disturbance image signal in the moving window is below a threshold, and accordingly re-acquires the information of the moving object from an entire image signal through the moving object acquiring unit.

11. The automatic moving object tracking device of claim 1, further comprising a photographing control unit controlling the photographing unit in a manner such that the photographing unit continues photographing in a fixed position when the center and size of the moving object acquired through the moving object acquiring unit is within a predetermined range that does not require a change of a photographing direction and a zoom-in/zoom-out of the photographing unit, and that the change of the photographing direction and the zoom-in/zoom-out are performed when the center and size of the moving object exceeds the predetermined range.

12. The automatic moving object tracking device of claim 11, wherein the photographing control unit controls the zoom-in/zoom-out of the photographing unit so that the size of an actual moving object acquired from the following frame is maintained at a certain rate with respect to the size of the moving window of a current frame.

13. The automatic moving object tracking device of claim 12, wherein the controlling of the photographing unit and the signal processing of the input image signal are performed on a time-division basis.

14. An automatic moving object tracking method, comprising:

photographing a monitoring area using a photographing unit;

generating a binary disturbance image signal from an input image signal acquired through the photographing unit;

acquiring from the binary disturbance image signal information about the moving object by using a moving window as an initial tracking window having a predetermined size;

adjusting a size of the initial tracking window so that the binary disturbance image signal contains the moving object;

estimating information about a location of the moving object in a following frame to which a center of the moving object is to be moved based on currently acquired information and previously acquired information about a center of the moving object;

moving a center of the moving window to the estimated location where the center of the moving object is estimated to be moved to;

acquiring actual information about an actual center of the moving object in the moving window and the size of the moving window; and

comparing the actual information about the moving object with the estimated information about the moving object, and determining tracking status of the moving object based on a resultant error range of the estimated and actual information, the actual information about the moving object being acquired through the following frame.

15. The automatic moving object tracking method of claim 14, further comprising:

improving a speed of processing the input image signal that is input through the photographing unit, and then pre-processing the input image signal to eliminate a noise from the input image signal.

16. The automatic moving object tracking method of claim 15, wherein the pre-processing of the input image signal comprises:

modifying a resolution of the input image signal; and

eliminating the noise from the input image signal having the reduced resolution.

17. The automatic moving object tracking method of claim 14, wherein the generating of the binary disturbance image signal comprises:

generating a disturbance field through a difference between the input image signal and an average of previous frames;

comparing the input image signal with a threshold and the average with the threshold to perform a logic-operation on a result of the comparison; and

determining a binary value for each pixel according to a result of comparing an absolute value of each pixel of the disturbance field with a disturbance threshold and also according to the result of the logic-operation.

18. The automatic moving object tracking method of claim 14, wherein the acquiring of the moving object comprises:

positioning the initial tracking window so that each pixel is recognizable with a center coordinate in the binary disturbance image signal;

determining a moving disturbance function (MDF) value of a pixel by adding up binary values of initial through last coordinates in the moving window; and

adjusting a size of the moving window so that the moving object is included in the input image signal with respect to a coordinate of a pixel having a greatest MDF value in the binary disturbance image signal.

19. The automatic moving object tracking method of claim 18, wherein the calculating of the MDF comprises calculating MDF values for pixels in parallel with each other.

20. The automatic moving object tracking method of claim 18, wherein the acquiring of the moving object determines that there does not exist the moving object when the binary value of the coordinate of the pixel having the greatest MDF value is smaller than an MDF threshold.

21. The automatic moving object tracking method of claim 14, wherein the determining of the tracking status comprises re-acquiring from the entire input image signal the information about the moving object with respect to the following frame when the error range exceeds a predetermined limit.

22. The automatic moving object tracking method of claim 14, further comprising:

controlling the photographing unit to move a photographing direction of the photographing unit to an estimated direction to which the moving object is estimated to be moved when the center of the moving window is located outside of a predetermined range with respect to the entire input image signal of the monitoring area.

23. The automatic moving object tracking method of claim 14, further comprising:

zooming-in/zooming-out of the photographing unit so that the size of the tracking window located at the estimated location and the size of the moving object acquired from the following frame are maintained at a certain rate.

24. The automatic moving object tracking method of claim 23, wherein the photographing control of the photographing unit and the signal processing of the input image signal are both performed on a basis of time-division.

25. The automatic moving object tracking method of claim 23, wherein the controlling of the photographing unit is discontinued when the tracking window is located within the predetermined range with respect to the entire input image area.

26. An automatic moving object tracking device, comprising:

a photographing unit photographing a monitoring area to generate an input image signal of each frame;

a binary disturbance image generating unit generating a binary disturbance image signal from input image signals of current and previous frames;

a moving object acquiring unit generating a moving window, acquiring information about a moving object from the binary disturbance image by using the moving window, and adjusting a size of the moving window in response to the information;

a moving object automatic tracking unit estimating a location of the moving object in a following frame based on the centers of the moving object in the current and previous frame and acquiring an actual center of the moving object in the moving window in the following frame;

a tracking status determining unit generating tracking status upon comparing the actual center of the moving object with the estimated center of the moving object; and

a photographing control unit controlling the photographing unit in response to the tracking status.

27. The automatic moving object tracking device of claim 26, wherein the moving object automatic tracking unit moves a center of the moving window to the estimated location in the following frame to detect the actual center of the moving object in the following frame.

28. The automatic moving object tracking device of claim 26, wherein moving object acquiring unit comprises:

a moving disturbance function (MDF) calculating unit determining a moving disturbance function (MDF) value of a pixel corresponding to a center coordinate by adding up binary values of coordinates in the moving window of the binary disturbance image signal; and

a tracking window adjusting unit adjusting the size of the moving window by calculating the center coordinate of the moving object of the current frame and a size of the moving object with reference to a pixel having a greatest MDF value.

29. The automatic moving object tracking device of claim 26, wherein the MDF value is calculated from a disturbance of the current frame using the following formula:

Δ

_t=I_t−

A<sub>t−

1</sub>

A_t=I_t−

K·

LIM(I_t−

A<sub>t−

1</sub>)

where Δ

_t is the disturbance of the current frame, A_t is either a background image or an average of images of the current frame, A<sub>t−

1</sub> is either a background image or an average of images of a previous frame, I_t is a currently input image, K is a gain, and LIM is a limiter.

30. The automatic moving object tracking device of claim 26, wherein the binary disturbance image generating unit comprises:

a first subtractor subtracting a previous input image signal from a current input image signal;

a second subtractor subtracting a first output of the first subtractor from the current input image signal;

a third subtractor subtracting a second output of the second subtractor from the current input image signal; and

an image division unit comparing a third output of the third subtractor with a reference value to generate the binary disturbance image signal.

31. The automatic moving object tracking device of claim 30, wherein the binary disturbance image generating unit comprises:

a limiter limiting the first output of the first subtractor to generate a fourth output; and

a gain controller controlling a gain of the fourth output of the limiter, the gain-controlled fourth output being subtracted from the current input image signal in the second subtractor to generate the second output.

32. An automatic moving object tracking method, comprising:

photographing a monitoring area to generate an input image signal of each frame using a photographing unit;

generating a binary disturbance image signal from input image signals of current and previous frames;

generating a moving window, acquiring information about a moving object from the binary disturbance image by using the moving window, and adjusting a size of the initial tracking window in response to the information;

estimating a location of the moving object in a following frame based on the centers of the moving object in the current and previous frame;

acquiring an actual center of the moving object in the tracking window in the following frame;

generating tracking status upon comparing the actual center of the moving object with the estimated center of the moving object; and

controlling the photographing unit in response to the tracking status.