System and method for analyzing video from non-static camera

• US 7,457,433 B2
• Filed: 01/20/2005
• Issued: 11/25/2008
• Est. Priority Date: 01/20/2005
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Pankanti, Sharathchandra U., Senior, Andrew W.
• Primary Examiner(s): Ahmed; Samir A
• Assistant Examiner(s): Koziol; Stephen R
• Application Number:
  US11/039,497
• Publication Number:
  US 20060159308A1
• Time in Patent Office: 1,405 Days
• Field of Search: 382/103, 382/294, 348/E7.088
• US Class Current: 382/103
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19663 Surveillance related processing done local to the camera
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A novel system and method of treating the output of moving cameras, in particular ones that enable the application of conventional “static camera” algorithms, e.g., to enable the continuous vigilance of computer surveillance technology to be applied to moving cameras that cover a wide area. According to the invention, a single camera is deployed to cover an area that might require many static cameras and a corresponding number of processing units. A novel system for processing the main video sufficiently enables long-term change detection, particularly the observation that a static object has been moved or has appeared, for instance detecting the parking and departure of vehicles in a parking lot, the arrival of trains in stations, delivery of goods, arrival and dispersal of people, or any other application.

Claims

1. A method for analyzing images acquired from a single non-static imaging device whose view of an observed area changes over time comprising the steps of:

establishing movement of said single non-static imaging device to acquire a sequence of single field of view static images covering a larger area to be observed,

processing each acquired single field of view static image and initially determining a number of reference images frames, each reference image frame comprising a single field of view corresponding to a portion of said larger area to be observed, and,

determining, for each reference image frame, one or more reference features, said one or more reference features comprising one or more of;

trackable points, lines, corners, and localizable image features, or combinations thereof;

employing motion estimation for tracking, in subsequent acquired single field of view static images, said reference features that are found in a reference frame as said imaging device moves, said motion estimation for estimating imaging device movement;

obtaining, based on said estimated imaging device movement, a best reference frame corresponding to each subsequent acquired single field of view static image; and

,

comparing each subsequent acquired single field of view static image to a corresponding reference image frame, and geometrically correcting said frame to align said acquired single field of view static image with said reference image frame based on said tracked reference features of said estimating imaging device movement;

generating successive output images corresponding to each of said number of reference image frames of a respective field of view portion to form a corresponding output image sequence for each corresponding field of view portion, each said successive output images corresponding to each of said number of reference images of a respective field of view portion being aligned with its corresponding reference image frame; and

,

analyzing each corresponding output image sequence for detecting a change in a desired feature of the corresponding field of view portions over time, said method further comprising;

providing control information for generating control signals used for controlling said the non-static imaging device to control said view changes of said subsequent acquired images over time, said control information comprising one or more of pan, tilt, zoom, rotation, dolly, and translation control parameters for controlling said non-static imaging device; and

,

recording time stamp information taken for each at least one reference image and successive output images of said output image sequence to thereby relate a time when a detected change in a desired feature of the observed area occurred.