Directional motion vector filtering

• US 9,560,372 B2
• Filed: 12/27/2010
• Issued: 01/31/2017
• Est. Priority Date: 12/27/2010
• Status: Active Grant

• Current Assignee: STMicroelectronics International N.V. (STMicroelectronics NV)
• Original Assignee: STMicroelectronics Incorporated (STMicroelectronics NV)
• Inventors: Tsyrganovich, Anatoliy Vasilevich
• Primary Examiner(s): SOSANYA, OBAFEMI OLUDAYO
• Application Number:
  US12/979,107
• Publication Number:
  US 20120163459A1
• Time in Patent Office: 2,227 Days
• Field of Search: 375/240.16
• US Class Current: 1/1
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/55 Motion estimation with spatial constraints, e.g. at image or region borders
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

An appropriate motion vector to assign to a pixel in a digital video frame is performed by a comparison of motion vectors of particular surrounding pixels. Direction of at least one of color transition or color brightness transition in the digital video frame is detected to detect direction of object boundaries in the digital video frame. The particular surrounding pixels are selected and grouped (filtered) according to the detected object boundary direction at each pixel. A comparison of the motion vectors of the surrounding pixels then provides information on which group of pixels to assign a current pixel being processed based in part on how close the motion vectors of the surrounding groups match a group pixels to which the pixel being processed belongs.

Claims

1. A method, comprising:

receiving digital video data and motion vector information associated with the digital video data;

detecting direction of at least one of color transition or brightness transition in a digital video frame of the digital video data;

detecting direction of slope of an object boundary in a digital video frame of the digital video data based on the detected direction of the at least one of color transition or brightness transition;

filtering motion vectors of selected pixels associated with the object boundary according to the detected direction of the object boundary, the filtering of the motion vectors including;

selecting a block of pixels associated with a first detected direction,

partitioning the block of the selected pixels into a middle section, a first side section and a second side section in a manner corresponding to the first detected direction,

calculating an average middle section motion vector value based on motion vector values of pixels of the middle section,

calculating an average first side section motion vector value based on motion vector values of pixels of the first side section,

calculating an average second side section motion vector value based on motion vector values of pixels of the second side section,

comparing the average middle section motion vector value to the average first side section motion vector value and the average second side section motion vector value, and

performing the motion vector filtering based on the comparing; and

providing a filtered vector output resulting from the filtering of the motion vectors.

2. The method of claim 1 further comprising assigning a directional index to a block of pixels associated with the detected direction of the at least one of the color transition or the color brightness transition.

3. The method of claim 2 wherein the directional index corresponds to one of a predetermined number of directions.

4. The method of claim 1 further comprising detecting a plurality of directions of one or more object boundaries in the digital video frame.

5. The method of claim 1 wherein the detecting the direction of at least one of color transition or color brightness transition in the digital video frame comprises executing spatial motion vector filtering of motion vectors of the digital video frame.

6. The method of claim 1 wherein the performing motion vector filtering based on the comparing comprises:

determining, based on the comparing, whether a center pixel of the selected block of pixels is to be assigned to an object associated with the motion vector values of pixels of the first side section, to an object associated with the motion vector values of pixels of the second side section, to an object associated with both the motion vector values of pixels of the first side section and second side section, or to no object.

7. The method of claim 6 further comprising:

assigning the center pixel to a selected object according to the determining; and

outputting a motion vector filtered output according to assigning the center pixel to a selected object.

8. The method of claim 7 further comprising:

outputting a motion vector filtered output according to assigning the pixel to a selected object.

9. The method of claim 1 further comprising repeating the selecting, partitioning, all the calculating steps, the comparing, and the performing motion vector filtering according to a second, third and fourth detected direction.

10. The method of claim 1 wherein the direction detected corresponds to an omni-directional area of the digital video frame.

11. The method of claim 1 wherein the digital video data is received by a mobile device.

12. The method of claim 1 wherein the digital video data is in a compressed format.

13. The method of claim 1 wherein the digital video data is in an uncompressed format.

14. A device for providing a filtered vector output comprising:

a direction selector configured to assign an object boundary slope direction to boundaries of an object within a digital video frame;

a filter bank including vector filter structures, the filter bank coupled to the direction selector;

a motion vector analyzer coupled to the filter bank and configured to assign a pixel to the object represented in the digital video frame based on a comparison of averaged motion vectors of surrounding pixels using a filter structure from the filter bank, the surrounding pixels being partitioned into at least two sections, a first section and a second section, each of the two sections being associated with a first detected direction by the direction selector,

the motion vector analyzer calculating an average first section motion vector value based on motion vector values of pixels of the first section,

calculating an average second section motion vector value based on motion vector values of pixels of the second section,

the comparison being carried out by comparing the average first section motion vector value to the average second section motion vector value; and

a memory coupled to the filter bank and motion vector analyzer, the memory being configured to store motion vectors associated with the digital video frame.

15. The device of claim 14, wherein the direction selector is configured to assign an object boundary slope direction to boundaries of an object within the digital video frame by being configured to select the boundaries associated with a particular slope direction for processing and to assign a corresponding directional index to those boundaries.

16. The device of claim 14, wherein direction selector is configured to designate the pixel as omni-directional based on a particular direction not being detected to a certain degree of specificity for the pixel.

19. The device of claim 14, wherein the motion vector analyzer coupled to the filter bank is configured to:

select a block of pixels associated with a first detected direction selected by the direction selector;

partition the block of the selected pixels into a middle section, a first side section and a second side section in a manner corresponding to the first detected direction;

calculate an average middle section motion vector value based on motion vector values of pixels of the middle section;

calculate an average first side section motion vector value based on motion vector values of pixels of the first side section;

calculate an average second side section motion vector value based on motion vector values of pixels of the second side section;

determine whether the filter structure from the filter bank is an omni-directional motion vector filter due to a discernable object boundary not being detected by the direction selector; and

if the determination is the filter structure from the filter bank is not an omni-directional motion vector, compare the average middle section motion vector value to the average first side section motion vector value and the average second side section motion vector value.

17. A non-transitory computer readable storage medium, having computer executable instructions thereon, that when executed by a computer processor cause the following method to be performed:

receiving digital video data and motion vector information associated with the digital video data;

detecting direction of at least one of color transition or color brightness transition in a digital video frame of the digital video data;

detecting direction of slope of an object boundary in a digital video frame of the digital video data based on the detected direction of the at least one of color transition or color brightness transition;

filtering motion vectors of selected pixels associated with the object boundary according to the detected direction of the object boundary, wherein the filtering of the motion vectors of the selected pixels associated with the object boundary comprises;

selecting a block of pixels associated with a first detected direction;

partitioning the block of the selected pixels into a middle section, a first side section and a second side section in a manner corresponding to the first detected direction,

calculating an average middle section motion vector value based on motion vector values of pixels of the middle section,

calculating an average first side section motion vector value based on motion vector values of pixels of the first side section,

calculating an average second side section motion vector value based on motion vector values of pixels of the second side section,

comparing the average middle section motion vector value to the average first side section motion vector value and the average second side section motion vector value, and

performing the motion vector filtering based on the comparing; and

providing a filtered vector output resulting from the filtering of the motion vectors.

18. The non-transitory computer readable storage medium of claim 17 wherein the computer executable instructions thereon, when executed by a computer processor, further cause performing motion vector filtering based on the comparing to be performed.

20. The device of claim 18, wherein the comparison of the average middle section motion vector value to the average first side section motion vector value and the average second side section motion vector value is according to the assigned object boundary slope direction.