Method, system and program product for a camera to track an object using motion vector data

• US 20060126737A1
• Filed: 12/15/2004
• Published: 06/15/2006
• Est. Priority Date: 12/15/2004
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Boice, Charles Edward, Westermann, Edward Francis, Schaefer, Joseph George, Butter, Adrian Stephen
• Application Number:
  US11/012,904
• Publication Number:
  US 20060126737A1
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: G01S 3/7862 mounted on a moving platform, e.g. space vehicle
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

A method, system and program product in accordance with the preferred embodiments use motion vector data to track an object being monitored by a video camera. Motion vector data are used to calculate pan, tilt and/or zoom adjustment data. For example, motion vector data may be provided to a motion tracking processor at a macroblock level by an MPEG compression processor. Alternatively, motion vector data may be provided to a motion tracking processor at a pixel level by a pre-processor. The pan, tilt and/or zoom adjustment data is sent to the camera. For example, the pan, tilt and/or zoom adjustment data may be sent to a PTZ adjustment mechanism of the camera. Because the preferred embodiments use a closed loop system, tracking the object is made easier and does not require a skilled operator.

Claims

1. A method for a camera to track an object using motion vector data, the method comprising the steps of:

a camera with a field of view providing a sequence of video fields;

providing motion vector data for an object in the field of view based on the sequence of video fields provided by the camera;

calculating at least one of pan, tilt and zoom adjustment data for the camera to track the object based on the motion vector data;

sending the at least one of pan, tilt and zoom adjustment data to the camera.

2. The method as recited in claim 1, wherein the step of sending the at least one of pan, tilt and zoom adjustment data to the camera includes the step of sending the at least one of pan, tilt and zoom adjustment data to a PTZ adjustment mechanism of the camera.

3. The method as recited in claim 1, wherein the step of providing motion vector data is performed by an MPEG compression processor that provides the motion vector data at a macroblock level, and wherein the step of calculating the at least one of pan, tilt and zoom adjustment data is performed by a motion tracking processor that receives the motion vector data at a macroblock level from the MPEG compression processor.

4. The method as recited in claim 1, wherein the step of providing motion vector data is performed by a pre-processor that provides the motion vector data at a pixel level, and wherein the step of calculating the at least one of pan, tilt and zoom adjustment data is performed by a motion tracking processor that receives the motion vector data at a pixel level from the pre-processor.

5. The method as recited in claim 1, further comprising the steps of:

detecting whether an event has occurred in the field of view, wherein the object is associated with the event;

tracking the object in the field of view based on the motion vector data.

6. The method as recited in claim 1, wherein the step of calculating the at least one of pan, tilt and zoom adjustment data includes the step of calculating a predicted camera pan and tilt adjustment mv_camera(n+1) between a current field (n) and a future field (n+1).

7. The method as recited in claim 6, wherein the step of calculating a predicted camera pan and tilt adjustment mv_camera(n+1) is performed for a first field as the current field (n=1) by calculating mv_camera(2)=ISF×

loc_oc(1), wherein ISF is a user-specified initial scaling factor and loc_oc(1) is an object center location in current field.

8. The method as recited in claim 7, wherein the user-specified initial scaling factor (ISF) is ½

.

9. The method as recited in claim 7, wherein the step of calculating a predicted camera pan and tilt adjustment mv_camera(n+1) is performed for a second field as the current field (n=2) by calculating mv_camera(3)=[2×

mv_oc(2)]+loc_oc(1)+mv_camera(2), wherein mv_oc(2) is an object center motion vector from a previous field to the current field.

10. The method as recited in claim 9, wherein the step of calculating a predicted camera pan and tilt adjustment mv_camera(n+1) is performed for a post-second field as the current field (n>

2) by calculating mv_camera(n+1)=[3×

mv_oc(n)]+loc_oc(n−

2)+[2×

mv_camera(n)]−

mv_camera(n−

1).

11. The method as recited in claim 10, wherein the values of mv_camera(n+1) and loc_oc(n) are stored for use in calculating future field pan and tilt adjustments.

12. The method as recited in claim 1, wherein the step of calculating the at least one of pan, tilt and zoom adjustment data includes the step of calculating an estimated focal length change Δ

fl_est(n+1) between a current field (n) and a future field (n+1).

13. The method as recited in claim 12, wherein the step of calculating an estimated focal length change Δ

fl_est(n+1) is performed for a second field as the current field (n=2) by calculating the estimated focal length change Δ

fl_est(3) based on object velocity.

14. The method as recited in claim 12, wherein the step of calculating an estimated focal length change Δ

fl_est(n+1) is performed for a post-second field as the current field (n>

2) by calculating the estimated focal length change Δ

fl_est(n+1) based on object velocity and acceleration.

15. The method as recited in claim 1, wherein the step of calculating the at least one of pan, tilt and zoom adjustment data includes the step of calculating predicted zoom adjustment data comprising the steps of:

calculating an ideal focal length fl_ideal(n) for viewing the object in a current field (n);

calculating a change in ideal focal length Δ

fl_ideal(n) from a past field (n−

1) to the current field (n);

calculating an estimated focal length change Δ

fl_est(n+1) between the current field (n) and a future field (n+1);

calculating a current field (n) actual focal length fl_act(n);

calculating a future field (n+1) estimated focal length fl_est(n+1);

calculating a future field (n+1) estimated zoom factor ZF_est(n+1).

16. The method as recited in claim 15, wherein step of calculating predicted zoom adjustment data further comprises the step of storing values of fl_ideal(n), Δ

fl_ideal(n), fl_act(n), and ZF_est(n+1) for use in calculating future field zoom adjustments.

17. The method as recited in claim 16, wherein the step of calculating predicted zoom adjustment data further comprises the step of storing values of y(1), x(1), Δ

AR, os_wa, mv_y-net(n), and mv_y-net(n) for use in calculating future field zoom adjustments, wherein y(1) is a field 1 object height at ideal focal length for optimal viewing, x(1) is a field 1 object width at ideal focal length for optimal viewing, Δ

AR is a permitted deviation of object aspect ratio from an initial value, os_wa is a weighted average object size, mv_y-net(n) is a net object height expansion or contraction from field (n−

1) to field (n), and mv_x-net(n) is a net object width expansion or contraction from field (n−

1) to field (n).

18. The method as recited in claim 17, wherein the step of calculating predicted zoom adjustment data further comprises the step of calculating an ideal zoom factor ZF_ideal(n) for the current field (n) based on a stored value of an estimated zoom factor ZF_est(n) for the current field (n) and a net object expansion or contraction factor.

19. The method as recited in claim 18, wherein the net object expansion or contraction factor is based on the stored values of os_wa, y(1), mv_y-net(n), x(1), and mv_x-net(n).

20. A system for a camera to track an object using motion vector data, comprising:

a camera with a field of view providing a sequence of video fields;

a video data processor providing motion vector data for an object in the field of view based on the sequence of video fields provided by the camera;

a motion tracking processor calculating at least one of pan, tilt and zoom adjustment data for the camera to track the object based on the motion vector data provided by the video data processor and sending the at least one of pan, tilt and zoom adjustment data to the camera.

21. The system as recited in claim 20, wherein the motion tracking processor sends the at least one of pan, tilt and zoom adjustment data to a PTZ adjustment mechanism of the camera.

22. The system as recited in claim 20, wherein the video data processor includes an MPEG compression processor that provides the motion vector data at a macroblock level to the motion tracking processor.

23. The system as recited in claim 20, wherein the video data processor includes a pre-processor that provides the motion vector data at a pixel level to the motion tracking processor.

24. The system as recited in claim 20, wherein the motion tracking processor calculates a predicted camera pan and tilt adjustment mv_camera(n+1) between a current field (n) and a future field (n+1).

25. The system as recited in claim 20, wherein the motion tracking processor calculates an estimated focal length change Δ

fl_est(n+1) between a current field (n) and a future field (n+1).

26. The system as recited in claim 20, wherein the motion tracking processor calculates an estimated focal length change Δ

fl_est(n+1) between a current field (n) and a future field (n+1), a current field actual focal length fl_act(n), a future field (n+1) estimated focal length fl_est(n+1), and a future field (n+1) estimated zoom factor ZF_est(n+1).

27. A program product, comprising:

a tracking mechanism that calculates at least one of pan, tilt and zoom adjustment data for a camera to track an object in a field of view covered by the camera based on motion vector data and sends the at least one of pan, tilt and zoom adjustment data to the camera;

computer-readable signal bearing media bearing the tracking mechanism.

28. The program product as recited in claim 27, wherein the signal bearing media comprises recordable media.

29. The program product as recited in claim 27, wherein the signal bearing media comprises transmission media.

30. The program product as recited in claim 27, wherein the tracking mechanism is executed by a motion tracking processor.

31. The program product as recited in claim 27, wherein the tracking mechanism provides the at least one of pan, tilt and zoom adjustment data to a PTZ adjustment mechanism of the camera system.

32. The program product as recited in claim 27, wherein the tracking mechanism calculates a predicted camera pan and tilt adjustment mv_camera(n+1) between a current field (n) and a future field (n+1).

33. The program product as recited in claim 27, wherein the tracking mechanism calculates an estimated focal length change Δ

fl_est(n+1) between a current field (n) and a future field (n+1).

34. The program product as recited in claim 27, wherein the tracking mechanism calculates an estimated focal length change Δ

fl_est(n+1) between a current field (n) and a future field (n+1), a current field actual focal length fl_act(n), a future field (n+1) estimated focal length fl_est(n+1), and a future field (n+1) estimated zoom factor ZF_est(n+1).