Method and system for tracking a fast moving object
First Claim
1. A method of following a detected moving object using a video camera having panning, tilting and/or zooming functions, comprising the steps of:
- a) storing succeeding and preceding frames produced by said camera into first and second memories, respectively, and controlling the panning, tilting and/or zooming functions of said video camera according to a position and a size of said moving object;
b) performing a motion compensation on said preceding frame to compensate for a motion of a background image of said moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame;
c) detecting an image difference between the motion-compensated preceding frame and said succeeding frame and determining from the image difference a position and a size of the moving object; and
d) repeating steps (a) to (c) by using said position and said size of step (c) when the step (b) is repeated.
1 Assignment
0 Petitions
Accused Products
Abstract
A moving object is detected by subtraction between successive frames produced by a video camera and a position and a size of the moving object are determined. Preceding and succeeding frames produced by the video camera are stored into respective memories and the panning, tilting and zooming functions of the video camera are controlled according to the determined position and size of the moving object. A motion compensation is performed on the preceding frame to compensate for a motion of background image caused by a tracking movement of the camera so that coordinates of the motion-compensated frame are transformed to coordinates of the succeeding frame. An image difference between the motion-compensated frame and the succeeding frame is extracted as a moving object and a position and a size of the extracted image are determined, with which the video camera is adaptively controlled.
64 Citations
30 Claims
-
1. A method of following a detected moving object using a video camera having panning, tilting and/or zooming functions, comprising the steps of:
-
a) storing succeeding and preceding frames produced by said camera into first and second memories, respectively, and controlling the panning, tilting and/or zooming functions of said video camera according to a position and a size of said moving object;
b) performing a motion compensation on said preceding frame to compensate for a motion of a background image of said moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame;
c) detecting an image difference between the motion-compensated preceding frame and said succeeding frame and determining from the image difference a position and a size of the moving object; and
d) repeating steps (a) to (c) by using said position and said size of step (c) when the step (b) is repeated. - View Dependent Claims (2, 3, 4, 5, 6)
determining initial values of motion parameters by using an SSD (sum of squared differences) matching method;
performing an affine transformation of the preceding frame to the succeeding frame by using said initial values of motion parameters;
estimating said motion parameters of background image by using the affine-transformed preceding frame under constraint conditions of optical flow; and
repeating the affine transformation and the estimation of the motion parameters until said motion parameters have a sufficient degree of precision.
-
-
5. The method of claim 1, wherein a time interval between the motion-compensated frame and said succeeding frame of step (c) is adaptively varied according to the size of said moving object.
-
6. The method of claim 1, wherein a time interval between the motion-compensated frame and said succeeding frame of step (c) is adaptively varied according to a speed of movement of said moving object.
-
7. A method of following a detected moving object using a video camera having panning, tilting and zooming functions, comprising the steps of:
-
a) producing a template representing the detected moving object;
b) storing succeeding and preceding frames produced by said camera into first and second memories, respectively, and controlling the panning, tilting an zooming functions of said video camera according to a position and a size of the moving object;
c) selecting one of first, second and third tracking modes depending on a movement of the detected moving object;
d) if said first tracking mode is selected, performing a motion compensation on said preceding frame to compensate for a motion of a background image of said moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame;
e) detecting an image difference between the motion-compensated frame and said succeeding frame and determining from the image difference a position and a size of the moving object, and returning to step (a);
f) if said second tracking mode is selected, transforming the size of the template as an updated size of the moving object according to a zooming operation of said video camera caused by step (b), and determining a position of said moving object that matches the transformed template, and returning to step (b);
g) if said third tracking mode is selected, transforming the size of the template as an updated size of the moving object according to a zooming operation of said video camera caused by step (a);
h) performing a motion compensation on the preceding frame to compensate for a motion of a background image of said moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame; and
i) detecting an image difference between the motion-compensated frame and said succeeding frame, determining a position of the moving object by using the transformed template and the image difference and returning to step (a). - View Dependent Claims (8, 9, 10, 11, 12, 13, 14)
determining initial values of motion parameters by using an SSD (sum of squared differences) matching method;
performing an affine transformation of the preceding frame to the succeeding frame by using said initial values of motion parameters;
estimating said motion parameters of background image by using the affine-transformed preceding frame under constraint conditions of optical flow; and
repeating the affine transformation and the estimation of the motion parameters until said motion parameters have a sufficient degree of precision.
-
-
11. The method of claim 7, wherein a time interval between the motion-compensated frame and said succeeding frame of step (e) is adaptively varied according to the size of said moving object.
-
12. The method of claim 7, wherein a time interval between the motion-compensated frame and said succeeding frame of the step (e) is adaptively varied according to a speed of movement of said moving object.
-
13. The method of claim 7, wherein step (f) comprises the step of determining said position of said moving object by using an SAD (sum of absolute differences) matching method.
-
14. The method of claim 7, wherein step (i) comprises the step of determining said position of the moving object by using a linear sum of an SAD (sum of absolute differences) matching value and said image difference.
-
15. An image tracking system for tracking a detected moving object, comprising:
-
a video camera having panning, tilting and/or zooming functions;
first and second memories for storing succeeding and preceding frames produced by said camera, respectively;
tracking control circuitry connected to the video camera for performing a motion compensation on the preceding frame to compensate for a motion of a background image of the detected moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame, detecting an image difference between the motion-compensated frame and the succeeding frame, and determining from the image difference a position and a size of the moving object; and
camera control circuitry for controlling the panning, tilting and/or zooming functions of said video camera according to said position and said size of e moving object. - View Dependent Claims (16, 17, 18, 19, 20, 21)
determining initial values of motion parameters by using an SSD (sum of squared differences) matching method;
performing an affine transformation of the preceding frame to the succeeding frame by using said initial values of motion parameters;
estimating said motion parameters of background image by using the affine-transformed preceding frame under constraint conditions of optical flow; and
repeating the affine transformation and the estimation of the motion parameters until said motion parameters have a sufficient degree of precision.
-
-
20. The image tracking system of claim 15, wherein a time interval between the motion-compensated frame and said succeeding frame is adaptively varied according to the size of said moving object.
-
21. The image tracking system of claim 15, wherein a time interval between the motion-compensated frame and said succeeding frame is adaptively varied according to a speed of movement of said moving object.
-
22. An image tracking system for tracking a detected moving object, comprising:
-
a video camera having panning, tilting and zooming functions;
a template memory;
first and second memories for storing succeeding and preceding frames produced by said camera, respectively; and
tracking control circuitry for performing the functions of;
a) storing the detected moving object as a template in said template memory;
b) controlling the panning, tilting and zooming functions of said video camera according to a position and a size of the moving object;
c) selecting one of first, second and third tracking modes depending on a movement of the detected moving object;
d) if said first tracking mode is selected, performing a motion compensation on said preceding frame to compensate for a motion of a background image of said moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame;
e) detecting an image difference between the motion-compensated frame and said succeeding frame and determining from the image difference a position and a size of the moving object, and returning to the function (a);
f) if said second tracking mode is selected, transforming the size of the template as an updated size of the moving object according to the zooming operation of said video camera caused by the function (b), and determining a position of said moving object that matches the transformed template, and returning to the function (b);
g) if said third tracking mode is selected, transforming the size of the template as an updated size of the moving object according to a zooming operation of said video camera caused by the function (b);
h) performing a motion compensation on the preceding frame to compensate for a motion of a background image of said moving object so that coordinates of said preceding frame are transformed to coordinates of the succeeding frame; and
i) detecting an image difference between the motion-compensated frame and said succeeding frame, determining a position of the moving object by using the transformed template and the image difference and returning to the function (a). - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30)
determining initial values of motion parameters by using an SSD (sum of squared differences) matching method;
performing an affine transformation of the preceding frame to the succeeding frame by using said initial values of motion parameters;
estimating said motion parameters of background image by using the affine-transformed preceding frame under constraint conditions of optical flow; and
repeating the affine transformation and the estimation of the motion parameters until said motion parameters have a sufficient degree of precision.
-
-
27. The image tracking system of claim 22, wherein a time interval between the motion-compensated frame and said succeeding frame is adaptively varied according to the size of said moving object.
-
28. The image tracking system of claim 22, wherein a time interval between the motion-compensated frame and said succeeding frame is adaptively varied according to a speed of movement of said moving object.
-
29. The image tracking system of claim 22, wherein the function (f) comprises determining said position of said moving object by using an SAD (sum of absolute differences) matching method.
-
30. The image tracking system of claim 22, wherein the function (i) comprises determining said position of the moving object by using a liner sum of an SAD (sum of absolute differences) matching value and said image difference.
Specification