Automatic Multiscale Image Acquisition from a Steerable Camera

• US 20120327246A1
• Filed: 09/04/2012
• Published: 12/27/2012
• Est. Priority Date: 03/07/2005
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Senior, Andrew William, Pankanti, Sharathchandra, Hampapur, Arun, Brown, Lisa Marie, Tian, Ying-Li
• Application Number:
  US13/602,973
• Publication Number:
  US 20120327246A1
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/662 by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A system for automatically acquiring high-resolution images by steering a pan-tilt-zoom camera at targets detected in a fixed camera view is provided. The system uses automatic or manual calibration between multiple cameras. Using automatic calibration, the homography between the cameras in a home position is estimated together with the effects of pan and tilt controls and the expected height of a person in the image. These calibrations are chained together to steer a slave camera. The manual calibration scheme steers a camera to the desired region of interest and calculates the pan, tile and zoom parameters accordingly.

Claims

1. A method for acquiring multi-scale images, the method comprising:

tracking an object of interest moving through a field of view of a master camera;

calibrating a steerable slave camera to obtain multi-scale images of the object of interest;

steering the slave camera in accordance with the calibration; and

obtaining multi-scale video of the object of interest using the slave camera.

2. The method of claim 1, wherein the master camera and the slave camera comprise a single camera.

3. The method of claim 1, wherein the step of identifying an object further comprises identifying a plurality of objects and the step calibrating a steerable slave camera comprises calibrating a plurality of steerable cameras, the method further comprising using a camera control policy to associate steerable cameras with objects of interest so as to optimize at least one objective function.

4. The method of claim 3, wherein the objective function comprises the imaged size of an object, the orientation of an object, the location of an object, the length of time an object is viewed, the number of times an object is viewed, the chance of viewing an object before that object is no longer visible, the speed of the object, the type of the object or any other measured parameter of the object, the time taken to steer an available camera to view the object and combinations thereof.

5. The method of claim 1, wherein the step of calibrating the steerable slave camera comprises generating at least one of the pan, the tilt and the zoom parameters for the steerable camera.

6. The method of claim 1, wherein the step of calibrating the steerable slave camera comprises manually calibrating the steerable slave camera.

7. The method of claim 6, wherein the step of manually calibrating the steerable slave camera comprises:

displaying a field of view of the master camera in a graphical user interface;

identifying the objects of interest in the field of view;

defining at least one region of interest in the field of view;

associating at least one steerable slave camera with each region of interest;

generating pan, tilt and zoom parameters for each steerable camera for objects of interest within the associated region of interest.

8. The method of claim 7, further comprising:

identifying current and predicted regions of interest through which each object of interest tracks;

selecting steerable slave cameras associated with the identified regions of interest; and

steering the selected slave cameras to the associated regions of interest in accordance with the pan, tilt and zoom parameters generated for the objects of interest that track through the regions of interest.

9. The method of claim 1, wherein the step of calibrating the steerable slave camera comprises automatically calibrating the steerable slave camera.

10. The method of claim 9, wherein the step of automatically calibrating the steerable slave camera comprises:

establishing a homography between the master camera and the slave camera for objects of interest;

obtaining a height mapping between the master camera and the slave camera for objects of interest; and

obtaining control parameters for the steerable camera.

11. The method of claim 10, wherein the step of establishing a homography comprises:

directing the master camera and the steerable camera to a common position;

generating a tracking data on a common object for both the master camera and the slave camera;

using the tracking data from the master camera and the slave camera to generate the homography.

12. The method of claim 11, further comprising generating a multiple sets of tracking data, generating at least one homography for each set of tracking data and selecting a best fit homography for use with the steerable slave camera.

13. The method of claim 10, wherein the step of obtaining a height mapping between the master camera and the slave camera comprises determining the height proportionality between the master camera and the slave camera for a common object of interest.

14. The method of claim 10, wherein the step of obtaining control parameters for the steerable camera comprises:

moving the slave camera in accordance with at least one known pattern;

tracking points along the pattern while the slave camera is moving;

estimating the motion of the slave camera using the tracked points;

comparing the estimated motion of the track points to actual point locations from the known pattern; and

deriving the control parameters from the comparison of estimated points to actual points.

15. The method of claim 14, wherein the step of estimating the motion comprises using affine fit or random sample consensus.

17. The computer readable medium of claim 1, wherein the step of identifying an object further comprises identifying a plurality of objects and the step calibrating a steerable slave camera comprises calibrating a plurality of steerable cameras, the method further comprising using a camera control policy to associate steerable cameras with objects of interest so as to optimize at least one objective function.

18. The computer readable medium of claim 1, wherein the step of calibrating the steerable slave camera comprises manually calibrating the steerable slave camera.

19. The computer readable medium of claim 1, wherein the step of calibrating the steerable slave camera comprises automatically calibrating the steerable slave camera.

16. A computer readable medium containing a computer executable code that when read by a computer causes the computer to perform a method for acquiring multi-scale images, the method comprising:

tracking an object of interest moving through a field of view of a master camera;

calibrating a steerable slave camera to obtain multi-scale images of the object of interest;

steering the slave camera in accordance with the calibration; and

obtaining multi-scale video of the object of interest using the slave camera.

20. A system for indexing multi-scale data, the system comprising:

a database comprising a plurality of database entries, each database entry associated with at least one multi-scale image;

a control mechanism in communication with the database and capable of retrieving database entries; and

a graphical user interface capable of displaying the multi-scale images associated with database entries retrieved by the control mechanism.