System and method for motion detection and the use thereof in video coding

• US 8,532,339 B2
• Filed: 08/08/2011
• Issued: 09/10/2013
• Est. Priority Date: 02/28/2007
• Status: Active Grant

• Current Assignee: IMEC International
• Original Assignee: IMEC International, Katholieke Universiteit Leuven
• Inventors: Lu, Jiangbo
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US13/205,546
• Publication Number:
  US 20120063646A1
• Time in Patent Office: 764 Days
• Field of Search: 382/107, 382/166, 382/232, 382/244
• US Class Current: 382/107
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

A system and method for motion detection and the use thereof in video coding are disclosed. In one aspect, a method of defining a region of motion within a video frame in a sequence of video frames comprises loading a current video frame and at least one reference video frame from the sequence, the reference video frame being different from the current video frame. The method further comprises applying filtering operations on the current and the reference video frame in order to obtain at least two scales of representation of the current and the reference video frame. The method further comprises determining for each of the scale representations a video-frame like representation of the structural changes between the current and the reference video frame. The method further comprises combining the video-frame like representations of different scales. The method further comprises determining one or more regions of motion from the combination.

Claims

1. A method of detecting a region of motion in a sequence of video frames, the method comprising:

receiving a first video frame and a reference video frame from a sequence of video frames, the reference video frame being different from the first video frame;

processing the first video frame and the reference video frame to obtain each of the first video frame and the reference video frame at a plurality of different scales;

determining, for each of the scales, a representation of at least one structural change between the first video frame and the reference video frame at the scale; and

determining one or more regions of motion based at least in part on the representations, the regions of motion being moving foreground objects.

2. The method of claim 1, wherein at least one of the first and reference video frames is pre-filtered to reduce noise.

3. The method of claim 1, wherein at least one of the scales is obtained by applying down-sampling operations.

4. The method of claim 1, wherein at least one of the scales is filtered to suppress first order illumination changes.

5. The method of claim 1, determining the representation comprises performing a subtraction operation between the first and the reference video frame at the scale.

6. The method of claim 1, wherein determining one or more regions of motion comprises performing a summing operation of the representations.

7. A video processing system adapted for carrying out the method of claim 1.

8. A non-transitory computer-readable medium having stored therein a computer program that, when executed, is configured to perform the method of claim 1.

9. The method of claim 1, wherein processing the first video frame and the reference video frame comprises filtering the first video frame and the reference video frame to obtain each of the first video frame and the reference video frame at a plurality of different scales.

10. The method of claim 1, further comprising extracting, for each of the scales, underlying structure of the processed first video frame and the reference video frame at the scale.

11. The method of claim 10, wherein the process of extracting underlying structure comprises, for each of the scales, applying a Laplacian operator to attain a Laplacian map of the first video frame and the reference video frame at the scale.

12. A method of coding a sequence of video frames, comprising:

performing a method of detecting a region of motion in a sequence of video frames, the method comprising;

receiving a first video frame and a reference video frame from the sequence, the reference video frame being different from the first video frame,

processing the first video frame and the reference video frame to obtain each of the first video frame and the reference video frame at a plurality of different scales,

determining, for each of the scales, a representation of at least one structural change between the first video frame and the reference video frame at the scale,

determining one or more regions of motion based at least in part on the representations, the regions of motion being moving foreground objects; and

performing motion estimation/compensation based encoding, wherein the motion estimation/motion compensation is performed only on the determined regions of motion.

13. A video encoder adapted for carrying out the method of claim 12.

14. A non-transitory computer-readable medium having stored therein a computer program that, when executed, is configured to perform the method of claim 12.

15. A device for detecting a region of motion in a sequence of video frames, the device comprising:

a loading module configured to receive a first video frame and a reference video frame from a sequence of video frames, the reference video frame being different from the first video frame;

a processing module configured to process the first video frame and the reference video frame to obtain each of the first video frame and the reference video frame at a plurality of different scales;

a first determining module configured to determine, for each of the scales, a representation of at least one structural change between the first video frame and the reference video frame at the scale; and

a second determining module configured to determine one or more regions of motion based at least in part on the representations, the regions of motion being moving foreground objects.

16. The device of claim 15, further comprising a pre-filtering module to pre-filter at least one of the first video frame and reference video frames to reduce noise.

17. The device of claim 15, wherein the process module is configured to apply down-sampling operations to obtain at least one of the scales.

18. The device of claim 15, further comprising a filtering module to filter at least one of the scales so as to suppress first order illumination changes.

19. The device of claim 15, wherein the first determining module is configured to determine the representation by performing a subtraction operation between the first video frame and the reference video frame at the scale.

20. The device of claim 15, wherein the second determining module is configured to determine one or more regions of motion by performing a summing operation of the representations.

21. The device of claim 15, wherein the processing module is additionally configured to filter the first video frame and the reference video frame to obtain each of the first video frame and the reference video frame at a plurality of different scales.

22. A device for detecting a region of motion in a sequence of video frames, the device comprising:

means for receiving a first video frame and a reference video frame from a sequence of video frames, the reference video frame being different from the first video frame;

means for processing the first video frame and the reference video frame to obtain each of the first video frame and the reference video frame at a plurality of different scales;

means for determining, for each of the scales, a representation of at least one structural change between the first video frame and the reference video frame at the scale; and

means for determining one or more regions of motion based at least in part on the representations, the regions of motion being moving foreground objects.