BACKGROUND IMAGE AND MASK ESTIMATION FOR ACCURATE SHIFT-ESTIMATION FOR VIDEO OBJECT DETECTION IN PRESENCE OF MISALIGNMENT

• US 20110150282A1
• Filed: 12/15/2010
• Published: 06/23/2011
• Est. Priority Date: 12/18/2009
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Gupta, Amit Kumar
• Granted Patent: US 8,520,894 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/98 Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Disclosed herein are a method, system, and computer program product for aligning an input video frame from a video sequence with a background model associated with said video sequence. The background model includes a plurality of model blocks (820, 830), each one of the plurality of model blocks (820, 830) being associated with at least one mode (821, 822, 823, 831, 832, 833, 834), wherein each mode (821, 822, 823, 831, 832, 833, 834) includes a plurality of frequency domain attributes. The method selects, for each one of the plurality of model blocks, a reference mode dependent upon a mode of the corresponding model block, generates a background reference image (840) comprising the plurality of selected reference blocks (841, 842), and then aligns the input frame with the background model (810), based on the background reference image (810).

Claims

1. A computer-implemented method of aligning an input video frame from a video sequence with a background model, said background model including a plurality of model blocks, each one of the plurality of model blocks being associated with at least one mode, wherein each mode includes a plurality of frequency domain attributes, said method comprising the steps of:

selecting, for each one of the plurality of model blocks, a reference mode from the at least one mode associated with a corresponding model block;

generating a background reference image comprising the plurality of selected reference modes; and

aligning the input frame with the background model, based on the background reference image.

2. The method according to claim 1, comprising the further step of:

is for each one of said plurality of model blocks, selecting said reference mode from the at least one mode associated with the corresponding model block, based on the input image.

3. The method according to claim 1, comprising the further steps of:

generating a background image by applying an inverse transformation process to said background reference image;

determining a misalignment estimate based on a comparison of said input image and said background image; and

using said misalignment estimate to shift said input frame to align with said background model.

4. The method according to claim 1, comprising the further step of:

detecting video objects in said aligned input frame, based on said background model.

5. A computer readable storage medium having recorded thereon a computer program for aligning an input video frame from a video sequence with a background model, said background model including a plurality of model blocks, each one of the plurality of model blocks being associated with at least one mode, wherein each mode includes a plurality of frequency domain attributes, said computer program comprising code for performing the steps of:

selecting, for each one of the plurality of model blocks, a reference mode from the at least one mode associated with a corresponding model block;

generating a background reference image comprising the plurality of selected reference modes; and

aligning the input frame with the background model, based on the background reference image.

6. An apparatus for aligning an input video frame from a video sequence with a background model, said background model including a plurality of model blocks, each one of the plurality of model blocks being associated with at least one mode, wherein each mode includes a plurality of frequency domain attributes, said system comprising:

a storage device for storing a computer program; and

a processor for executing the program, said program comprising code for performing the method steps of;

selecting, for each one of the plurality of model blocks, a reference mode from the at least one mode associated with a corresponding model block;

generating a background reference image comprising the plurality of selected reference modes; and

aligning the input frame with the background model, based on the background reference image.

7. A computer-implemented method of generating a composite reference image to align an input image for video object detection, said method comprising the steps of:

receiving a background model, said background model comprising a plurality of model blocks, each one of the plurality of model blocks being associated with a plurality of modes, wherein each mode includes a plurality of frequency domain attributes;

for each one of said model blocks, selecting a reference mode from said plurality of modes associated with that block, based on the input image; and

generating a composite reference image comprising a plurality of reference blocks, wherein each reference block is associated with the reference mode of the corresponding model block.

8. The method according to claim 7, wherein at least one of the plurality of model blocks is masked in the composite reference image.

9. The method according to claim 7, comprising the further steps of:

aligning said input frame with said background model, based on said composite reference image.

10. The method according to claim 9, comprising the further step of detecting video objects in the aligned input frame, based on said background model.

11. A computer-implemented method of generating a composite reference image to correct misalignment in an input image, said method comprising the steps of:

receiving a background model, said background model comprising a plurality of model blocks, each one of the plurality of model blocks being associated with a plurality of modes, wherein each mode includes a plurality of attributes;

selecting for each one of the model blocks, based on the input image, a reference mode from the plurality of modes associated with the respective model block; and

generating a composite reference image comprising a plurality of reference blocks, each reference block being associated with the reference mode selected for the corresponding model block.