System and method for analyzing video from non-static camera

• US 7,822,228 B2
• Filed: 07/21/2008
• Issued: 10/26/2010
• Est. Priority Date: 01/20/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Pankanti, Sharathchandra U., Senior, Andrew W.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Koziol; Stephen R
• Application Number:
  US12/176,521
• Publication Number:
  US 20090022366A1
• Time in Patent Office: 827 Days
• Field of Search: 382/103, 382/294
• US Class Current: 382/103
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19663 Surveillance related processing done local to the camera
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A system and method of treating the output of moving cameras, in particular ones that enable the application of conventional “static camera” algorithms, e.g., to enable the continuous vigilance of computer surveillance technology to be applied to moving cameras that cover a wide area. A single camera is deployed to cover an area that might require many static cameras and a corresponding number of processing units. A novel system for processing the main video sufficiently enables long-term change detection, particularly the observation that a static object has been moved or has appeared, for instance detecting the parking and departure of vehicles in a parking lot, the arrival of trains in stations, delivery of goods, arrival and dispersal of people, or any other application.

Claims

1. A system for analyzing images acquired from a single non-static imaging device having a field of view that changes over time, the system comprising:

a memory;

a processor in communications with the memory, wherein the system is capable of performing a method comprising;

receiving said images and processing said images to obtain at least one reference image comprising a single field of view corresponding to a portion of a larger area to be observed;

generating successive images corresponding to each known at least one reference image of a field of view to form a corresponding output image sequence for each corresponding portion; and

,

analyzing each at least one output image sequence for detecting a change in a desired feature of the corresponding field of view portions.

2. The system as claimed in claim 1, wherein an acquired image exhibits a slight motion from its corresponding known reference image, said system further comprising means for aligning said successive reference image with its corresponding reference image.

3. The system as claimed in claim 1, wherein said means for receiving said images and processing said images further comprises means for estimating the image motion.

4. The system as claimed in claim 3, wherein said means for estimating the image motion comprises means for determining and tracking features used in motion estimation.

5. The system as claimed in claim 4, wherein said features include one or more of points, lines, corners, and localizable image features, or combinations thereof.

6. The system as claimed in claim 3, further comprising control means for providing control signals to said non-static imaging device used in controlling said view changes over time, said means for estimating the image motion further incorporating information about said control signals.

7. The system as claimed in claim 6, wherein said information about said control signals used in estimating motion comprises one or more of pan, tilt, zoom, rotation, dolly, translate control parameters.

8. The system as claimed in claim 1, wherein said processing means further records time stamp information taken for each at least one reference image and successive output images of said output image sequence for relating a time to a detected change in a desired feature.

9. The system as claimed in claim 1, wherein said means for analyzing each at least one reference image sequence is adapted to determine changes comprising:

arrival and departure of vehicles in parking spaces of an observed parking lot area;

the arrival of trains in stations;

the detection of arrival and departure times of vehicles, shipping containers or deliveries;

the arrival and dispersal of people;

or, the detection of graffiti, litter, dumping, and damage to an observed area or structure.

10. A computer-implemented method for analyzing images acquired from a single non-static imaging device whose view changes over time comprising the steps of:

a) processing received said images to obtain at least one reference image comprising a single field of view corresponding to a portion of a larger area to be observed;

b) generating successive output images corresponding to each known at least one reference image of a portion to form a corresponding output image sequence for each corresponding portion; and

,

c) analyzing each at least one output image sequence for detecting a change in a desired feature of the corresponding field of view portions,

wherein a program using a processor unit executes one or more of said processing, generating and analyzing.

11. The method as claimed in claim 10, wherein an acquired image exhibits a slight motion from its corresponding known reference image, said method further comprising the step of aligning said acquired image with its corresponding reference image before adding the acquired image to the output image sequence.

12. The method as claimed in claim 10, wherein said processing step a) further comprises the step of estimating the image motion.

13. The method as claimed in claim 12, wherein said step of estimating image motion comprises the step of determining and tracking features used in motion estimation, said features comprising one or more of:

points, lines, corners, and localizable image features, or combinations thereof.

14. The method as claimed in claim 12, wherein said step of estimating the image motion comprises the step of:

incorporating information about control signals provided to the non-static imaging device used in controlling said view changes over time, said information comprising one or more of pan, tilt and zoom control parameters.

15. The method as claimed in claim 10, further comprising the step of recording time stamp information for each at least one static reference image and successive generated output images of said image sequence, and relating a time with a detected change in a desired feature.

16. The method as claimed in claim 12, wherein said processing step a) includes the step of creating a first mosaic image of a larger area to be observed from one or more received frames, said means further generating successive mosaic images registered to the first mosaic image, said analyzing step analyzing each successive mosaic image for detecting a change in a desired feature of the observed area.

17. The method as claimed in claim 10, wherein said step of analyzing each at least one reference image sequence further includes determining changes comprising:

arrival and departure of vehicles in parking spaces of an observed parking lot area;

the arrival of trains in stations;

the detection of arrival and departure times of vehicles, shipping containers or, deliveries;

the arrival and dispersal of people;

or, the detection of graffiti, litter, dumping, and damage to an observed area or structure.