Apparatus and a method for detecting motion within an image sequence

• US 6,081,606 A
• Filed: 06/17/1996
• Issued: 06/27/2000
• Est. Priority Date: 06/17/1996
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Burt, Peter Jeffrey, Hansen, Michael Wade
• Primary Examiner(s): BELLA, MATTHEW C
• Application Number:
  US08/664,584
• Time in Patent Office: 1,471 Days
• Field of Search: 382/107, 382/278, 382/240, 382/260, 348/699
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Apparatus and a concomitant method of identifying the direction of motion within a scene that is represented by a sequence of images, e.g., a series of frames in a video sequence. The apparatus contains a flow field generator, a flow field segmentor, and alarm detector and an alarm processor. The flow field generator processes the sequence of images and generates a substantially distortionless flow field representing the motion within a scene. The flow field is a vector representation of the motion of the scene that represents both the magnitude and the direction of the motion. The flow field is generated by correlating at least two frames in the sequence of images. This flow field is analyzed by a flow field segmentor to determine the magnitude and direction of motion with the scene and segment the motion information from the static portions of the scene. An alarm detector then processes the motion information to determine if an alarm should be generated based upon the motion information. The alarm processor activates an appropriate alarm response, e.g., illuminate warning lamps, sound a warning alarm, lock doors, and alert security personnel.

Claims

1. Apparatus for detecting motion within a scene that is represented by a sequence of images, comprising:

a flow field generator for processing said sequence of images to produce a motion flow field, wherein said flow field generator comprises an image preprocessor for producing filtered images from said sequence of images and a flow field processor for generating said motion flow field from said filtered images, wherein said image preprocessor comprises an image selector for selecting three images from said sequence of images, a first subtractor for subtracting a first image from a second image to create a first difference image, a second subtractor for subtracting said second image from a third image to create a second difference image, a first filter for filtering said first difference image to produce a first filtered image, and a second filter for filtering said second difference image to produce a second filtered image;

a flow field segmentor for extracting said motion flow field from said sequence of images; and

an alarm detector for analyzing said motion flow field to determine if said motion flow field meets an alarm criteria.

2. The apparatus of claim 1 wherein said first and second filters are shaped filters.

3. The apparatus of claim 1 wherein said flow field processor further comprises:

a first image shifter for selectively shifting a first filtered image;

second image shifter for selectively shifting a second filtered image;

a multiplier for multiplying said first and second shifted filtered images with one another;

an integrator, connected to said multiplier, for integrating a series of multiplied images;

a flow field estimator, connected to said integrator, for generating said motion flow field from said integrated images.

4. The apparatus of claim 3 wherein said flow field processor further comprises a confidence value generator for producing a confidence value for each flow field vector in said motion flow field.

5. The apparatus of claim 3 further comprising means, connected to said flow field segmentor, for adjusting the sensitivity of the flow field generator in response to analyzing the segmented flow field generated by the flow field segmentor.

6. The apparatus of claim 5 wherein said sensitivity adjusting means controls the image selector.

7. The apparatus of claim 5 wherein said sensitivity adjusting means controls the flow field estimator.

8. A security system comprising:

motion detector comprising;

a flow field generator for processing a sequence of images to produce a motion flow field;

a flow field segmentor, connected to said flow field generator, for extracting said motion flow field from said sequence of images; and

an alarm detector, connected to said flow field segmentor, for analyzing said motion flow field to determine if said motion flow field meets an alarm criteria;

a warning camera, connected to said motion detector for imaging a first area of a scene;

an alarm camera, connected to said motion detector, for imaging a second area of a scene; and

where said motion detector generates a warning signal when a motion flow field meets certain alarm criteria in said first area of said scene imaged by said warning camera and generates an alarm signal when a motion flow field meets certain alarm criteria in said second area of said scene imaged by said alarm camera.

9. The security system of claim 8 further comprising:

an identification camera for producing images of a third area of a scene; and

recording means, connected to said motion detector and to said identification camera, for recording images from said identification camera in response to an alarm signal generated when said images from said alarm camera are processed.

10. The security system of claim 8, wherein said flow field generator comprises an image preprocessor for producing filtered images and a flow field processor for generating said motion flow field from said filtered images.

11. The security system of claim 10 wherein said image preprocessor further comprises:

an image selector for selecting three images from said sequence of images;

a first subtractor for subtracting a first image from a second image to create a first difference image;

a second subtractor for subtracting said second image from a third image to create a second difference image;

a first shaped filter for filtering said first difference image to produce a first filtered image; and

a second shaped filter for filtering said second difference image to produce a second filtered image.

12. The security system of claim 11 wherein said flow field processor further comprises:

a first image shifter for selectively shifting a first filtered image;

a second image shifter for selectively shifting a second filtered image;

a multiplier for multiplying said first and second shifted filtered images with one another;

an integrator, connected to said multiplier, for integrating a series of multiplied images;

a flow field estimator, connected to said integrator, for generating said motion flow field from said integrated images.

13. A method for detecting motion within a scene that is represented by a sequence of images, comprising the steps of:

selecting three images from said sequence of images;

subtracting a first image from a second image to create a first difference image;

subtracting said second image from a third image to create a second difference image;

filtering said first difference image to produce a first filtered image; and

filtering said second difference image to produce a second filtered image;

generating a motion flow field from said first and second filtered images;

extracting said motion flow field from said sequence of images; and

analyzing said motion flow field to determine if said motion flow field meets an alarm criteria.

14. The method of claim 13 wherein said first filtered image is produced by shaped filtering said first difference image and said second filtered image is produced by shaped filtering said second difference image.

15. The method of claim 13 wherein said processing step further comprises the steps of:

selectively shifting a first filtered image;

selectively shifting a second filtered image;

multiplying said first and second shifted filtered images with one another;

integrating a series of multiplied images;

generating said motion flow field from said integrated images.

16. The method of claim 13 wherein said extracting step further comprises the step of producing a confidence value for each flow field vector in said motion flow field.

17. The method of claim 13 further comprising the step of adjusting the sensitivity used to generate said flow field in response to analyzing the extracted flow field.