Adaptive interlace-to-progressive scan conversion algorithm

US 20020171759A1
Filed: 02/08/2002
Published: 11/21/2002
Est. Priority Date: 02/08/2001
Status: Active Grant

First Claim

Patent Images

1. An interlace-to-progressive scan conversion system, comprising:

a prefilter having a prefiltered signal as an output;

a motion estimator having the prefiltered signal as input and a motion-corrected signal as an output;

an adaptive filter having the prefiltered signal and the motion-corrected signal as inputs.

View all claims

12 Assignments

Timeline View

Assignment View

0 Petitions

Accused Products

Abstract

An interlace-to-progressive scan conversion system comprises: a spatial line averaging prefilter; a motion estimator; a three-stage adaptive recursive filter. The motion estimator comprises: a 3-D recursive search sub-component having a bilinear interpolator; a motion correction sub-component having an error-function including penalties related to the difference between a given candidate vector and a plurality of neighboring vectors; a block erosion sub-component. The motion estimator assumes that motion is constant between fields. The three-stage adaptive recursive filter comprises: a first stage that selects between using static pixels data and moving pixels data from a next field; a second stage that selects a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; a third stage that combines an intra-field interpolation with the more valid set of data selected by the second stage.

Citations

33 Claims

1. An interlace-to-progressive scan conversion system, comprising:
- a prefilter having a prefiltered signal as an output;
  
  a motion estimator having the prefiltered signal as input and a motion-corrected signal as an output;
  
  an adaptive filter having the prefiltered signal and the motion-corrected signal as inputs.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
- - 2. The interlace-to-progressive scan conversion system of claim 1, wherein the prefilter is a line averaging filter.
  - 3. The interlace-to-progressive scan conversion system of claim 2, wherein the prefilter is a spatial line averaging filter.
  - 4. The interlace-to-progressive scan conversion system of claim 1, wherein the motion estimator is adapted to perform a 3-D recursive search.
  - 5. The interlace-to-progressive scan conversion system of claim 1, wherein the motion estimator is adapted to perform motion vector correction.
  - 6. The interlace-to-progressive scan conversion system of claim 1, wherein the motion estimator is adapted to perform a block erosion process.
  - 7. The interlace-to-progressive scan conversion system of claim 1, wherein the adaptive filter comprises a median filter.
  - 8. The interlace-to-progressive scan conversion system of claim 1, wherein the adaptive filter comprises a line averaging filter.
  - 9. The interlace-to-progressive scan conversion system of claim 1, wherein the adaptive filter comprises an adaptive recursive filter.
  - 10. The interlace-to-progressive scan conversion system of claim 1, wherein the adaptive filter comprises a time recursive filter.
  - 11. The interlace-to-progressive scan conversion system of claim 1, wherein:
    - the adaptive filter comprises a three-stage adaptive recursive filter, wherein;
      
      a first stage comprises a function that selects between using static pixels data and moving pixels data from a next field;
      
      a second stage comprises a function that selects a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; and
      
      a third stage comprises a function that combines an intra-field interpolation with the more valid set of data selected by the second stage.
  - 12. The interlace-to-progressive scan conversion system of claim 11, wherein the prefilter comprises a spatial line average filter.
  - 13. The interlace-to-progressive scan conversion system of claim 11, wherein the motion estimator comprises a 3-D recursive search sub-component.
  - 14. The interlace-to-progressive scan conversion system of claim 11, wherein the motion estimator comprises a motion vector correction sub-component.
  - 15. The interlace-to-progressive scan conversion system of claim 11, wherein the motion estimator comprises a block erosion sub-component.
  - 17. The interlace-to-progressive scan conversion system of claim 16, wherein the 3-D recursive search sub-component resolves motion vectors to at least quarter-pixel accuracy.
  - 18. The interlace-to-progressive scan conversion system of claim 17, wherein the look-up table consists of:
  - 19. The interlace-to-progressive scan conversion system of claim 16, wherein the motion estimator includes a bilinear interpolator.
  - 20. The interlace-to-progressive scan conversion system of claim 19, wherein a value of a first estimator is set to a value of a second estimator if:
    - e({overscore (MV)}_a, {overscore (X)}−
      
      {overscore (SMV)}_a,t)>
      
      e({overscore (MV)}_b, {overscore (X)}−
      
      {overscore (SMV)}_b,t)+Thand wherein the value of the second estimator is set to the value of the first estimator if;
      
      e({overscore (MV)}_b,{overscore (X)}−
      
      {overscore (SMV)}_b,t)>
      
      e({overscore (MV)}_a,{overscore (X)}−
      
      {overscore (SMV)}_a,t)+Thwhere Th is a fixed threshold.
  - 21. The interlace-to-progressive scan conversion system of claim 16, wherein an error function of the motion estimator includes penalties related to a length of the difference vector between a given candidate vector and a plurality of neighboring vectors.
  - 22. The interlace-to-progressive scan conversion system of claim 21, wherein the error function is defined by:
  - 23. The interlace-to-progressive scan conversion system of claim 21, wherein the motion estimator assumes that a motion vector for an object between a previous field and a current field is the same as a motion vector for the object between the current field and a next field.
  - 24. The interlace-to-progressive scan conversion system of claim 23, wherein a motion vector error correction function is defined by:
  - 25. The interlace-to-progressive scan conversion system of claim 23, wherein a motion vector error correction function is defined by:
  - 26. The interlace-to-progressive scan conversion system of claim 16, wherein a cost function is defined by:
    - ∀
      
      F(x,y,t)ε
      
      B(x,y,t);
      
      D=|F(x,y,t)−
      
      F(x−
      
      MV_x,y−
      
      MV_y,t−
      
      1)|TD=TD+DDiff=D−
      
      EstErrEstErr=EstErr+(δ
      
      +Diff);
      
      Dev=Dev+δ
      
      (|Diff|−
      
      Dev)
  - 27. The interlace-to-progressive scan conversion system of claim 16, wherein the block erosion sub-component divides each block according to:
    - B(x,y,t)={(x,y)|X_x−
      
      X|2≦
      
      x≦
      
      X_x+X/2X_y−
      
      Y/2≦
      
      y≦
      
      X_y+Y/2}wherein a vector {overscore (MV)}(x,y,t) is assigned, into four sub-blocks B_i,j( x , y , t)
  - 28. The interlace-to-progressive scan conversion system of claim 16, wherein the first stage selection function is given by:
  - 29. The interlace-to-progressive scan conversion system of claim 16, wherein the third stage combining function is given by:

16. An interlace-to-progressive scan conversion system, comprising:
- a spatial line averaging prefilter having a prefiltered signal as an output;
  
  a motion estimator having the prefiltered signal as input and a motion-corrected signal as an output, the motion estimator comprising;
  
  a 3-D recursive search sub-component;
  
  a motion vector correction sub-component;
  
  a block erosion sub-component;
  
  a three-stage adaptive recursive filter, wherein;
  
  a first stage comprises a function that selects between using static pixels data and moving pixels data from a next field;
  
  a second stage comprises a function that selects a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; and
  
  a third stage comprises a function that combines an intra-field interpolation with the more valid set of data selected by the second stage.

30. An interlace-to-progressive scan conversion system, comprising:
- a spatial line averaging prefilter having a prefiltered signal as an output;
  
  a motion estimator having the prefiltered signal as input and a motion-corrected signal as an output, the motion estimator comprising;
  
  a 3-D recursive search sub-component having a bilinear interpolator;
  
  a motion vector correction sub-component having an error function, the error function including penalties related to a length of the difference vector between a given candidate vector and a plurality of neighboring vectors;
  
  a block erosion sub-component;
  
  wherein the motion estimator assumes that a motion vector for an object between a previous field and a current field is the same as a motion vector for the object between the current field and a next field a three-stage adaptive recursive filter having the prefiltered output and the motion-corrected output as inputs, the three stages comprising;
  
  a first stage that comprises a function that selects between using static pixels data and moving pixels data from a next field;
  
  a second stage that comprises a function that selects a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; and
  
  a third stage that comprises a function that combines an intra-field interpolation with the more valid set of data selected by the second stage.

31. An interlace-to-progressive scan conversion system, comprising:
- a spatial line averaging prefilter having a prefiltered signal as an output;
  
  a motion estimator having the prefiltered signal as input and a motion-corrected signal as an output, the motion estimator comprising;
  
  a 3-D recursive search sub-component;
  
  a motion vector correction sub-component; and
  
  a block erosion sub-component;
  
  wherein;
  
  the 3-D recursive search sub-component includes a bilinear interpolator defined by;
  
  $\begin{matrix} F (x, y, t) = (yf \cdot xf \cdot F (xi, yi, t)) + (yf \cdot (1 - xf) \cdot F (xi + 1, yi, t)) + \\ ((1 - yf) \cdot xf \cdot F (xi, yi + 1, t)) + ((1 - yf) \cdot (1 - xf) \cdot \\ F (xi + 1, yi + 1, t) \\ where; \\ yf = ⌊ y ⌋ xf = ⌊ x ⌋ \\ and; \\ yi = y - ⌊ y ⌋ xi = x - ⌊ x ⌋ \end{matrix}$ and wherein a value of a first estimator is set to a value of a second estimator if;
  
  e({overscore (MV)}_a,{overscore (X)}−
  
  {overscore (SMV)}_a,t)>
  
  e({overscore (MV)}_b,{overscore (X)}−
  
  {overscore (SMV)}_b,t)+Thand wherein the value of the second estimator is set to the value of the first estimator if;
  
  e({overscore (MV)}_b,{overscore (X)}−
  
  {overscore (SMV)}_b,t)>
  
  e({overscore (MV)}_a,{overscore (X)}−
  
  {overscore (SMV)}_a,t)+Thwhere Th is a fixed threshold;
  
  the 3-D recursive search sub-component has a look-up table consisting of;
  
  ${US}_{n} = {(\begin{matrix} 0 \\ 0 \end{matrix}), (\begin{matrix} 0 \\ 1 \end{matrix}), (\begin{matrix} 0 \\ - 1 \end{matrix}), (\begin{matrix} 1 \\ 0 \end{matrix}), (\begin{matrix} - 1 \\ 0 \end{matrix}), (\begin{matrix} 0 \\ 2 \end{matrix}), (\begin{matrix} 0 \\ - 2 \end{matrix}), (\begin{matrix} 3 \\ 0 \end{matrix}), (\begin{matrix} - 3 \\ 0 \end{matrix}), (\begin{matrix} 0 \\ \frac{1}{4} \end{matrix}), (\begin{matrix} 0 \\ - \frac{1}{4} \end{matrix}), (\begin{matrix} \frac{1}{4} \\ 0 \end{matrix}), (\begin{matrix} - \frac{1}{4} \\ 0 \end{matrix})}$ a motion vector correction sub-component having an motion vector error correction function defined by;
  
  $\overline{MV} (x, y, t) = {\begin{matrix} (\begin{matrix} 0 \\ 0 \end{matrix}), & (e_{m} (x, y, t) \geq e_{s} (x, y, t)) \\ \overline{MV} (x, y, t), & (e_{m} (x, y, t) < e_{s} (x, y, t)) \end{matrix} where; e_{m} (x, y, t) = \sum \langle F (C) - F (D) \rangle e_{s} (x, y, t) = \sum \langle F (A) - F (B) \rangle$ and where A, B, C, D, and X are blocks containing ends of candidate motion vectors, X being in the current field, A and C being in the previous field, and B and D being in the next field. a block erosion sub-component that divides each block according to;
  
  B(x,y,t)={(x,y)|X_x−
  
  X/2≦
  
  x≦
  
  X_x+X/2X_y−
  
  Y/2≦
  
  y≦
  
  X_y+Y/2}wherein a vector {overscore (MV)}(x,y,t) is assigned, into four sub-blocks B_i,j(x , y , t) $B_{i, j} (x, y, t) = {(x, y)  X_{x} - (1 - i) \cdot \frac{X}{4} \leq x \leq X_{x} + (1 + i) \cdot \frac{X}{4} ⋀ X_{y} (1 - j) \cdot \frac{Y}{4} \leq y \leq X_{y} + (1 + j) \cdot \frac{Y}{4}}$ and wherein the variables I and j take the values +1 and −
  
  1;
  
  wherein a vector MV_ij(x, y, t) is assigned to the pixels of each of the sub-blocks B_ij(x, y, t);
  
  ∀
  
  (x,y)ε
  
  B_i,j(x,y,t);
  
  {overscore (MV)}_i,j(x,y,t)=;
  
  {overscore (MV)}_i,j({overscore (X)},t)wherein;
  
  {overscore (MV)}_i,j({overscore (X)},t)=med[{overscore (MV)}(x+i·
  
  X,y,t),{overscore (MV)}({overscore (X)},t), {overscore (MV)}(x,y+j·
  
  Y,t)]wherein the median function is a median on the x and y vector components separately; and
  
  wherein a resulting vector is replaced by an original motion vector unless the resulting vector is equal to one of the three input vectors. a three-stage adaptive recursive filter having the prefiltered signal and motion-corrected signals as output, the three stages comprising;
  
  a first stage comprises a function that selects between using static pixels data and moving pixels data from a next field according to the function;
  
  $F_{n} (x, y, t) = {\begin{matrix} F (x + {MV}_{x} (x, y, t), y + {MV}_{y} (x, y, t), t + 1), & (D_{m} < D_{s}) \\ F (x, y, t + 1), & (D_{m} \geq D_{s}) \end{matrix} where; D_{s} = \sum_{k = - 2}^{2} C_{v} (k) \cdot \langle F (x, y + k, t) - F (x, y + k, t + 1) \rangle D_{m} = \sum_{k = - 2}^{2} C_{v} (k) \cdot \langle F (x, y + k, t) - F (x - {MV}_{x} (x, y, t), y - {MV}_{y} (x, y, t) + k, t + 1) \rangle$ a second stage comprises a function that selects a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; and
  
  a third stage comprises a function that combines an intra-field interpolation with the more valid set of data selected by the second stage according to the function;
  
  $F_{o} (x, y, t) = {\begin{matrix} F (x, y, t), & (y \mod 2 = t \mod 2) \\ (c_{i} \cdot F_{i} (x, y, t)) + (1 - c_{i}) (c_{p} \cdot F_{p} (x, y, t) + (1 - c_{p}) F_{n} (x, y, t)), & (otherwise) \end{matrix}$ wherein c_iand c_pare adaptive coefficients ranging from 0 to 1;
  
  F_nis given by;
  
  $F_{n} (x, y, t) = {\begin{matrix} F (x + {MV}_{x} (x, y, t), y + {MV}_{y} (x, y, t), t + 1), & (D_{m} < D_{s}) \\ F (x, y, t + 1), & (D_{m} \geq D_{s}) \end{matrix}$ wherein intra-field interpolation is given by;
  
  $F_{i} (x, y, t) = \frac{F (x, y - 1, t) + F (x, y + 1, t)}{2}$ and wherein backward data prediction is given by;
  
  F_p(x,y,t)=F(x−
  
  MV_x(x,y,t),y−
  
  MV_y(x,y,t),t−
  
  1)

32. A method for converting an interlaced image to a progressive scan image, the method comprsing:
- providing an input signal corresponding to an image;
  
  prefiltering the input signal with a spatial line averaging prefilter;
  
  estimating motion in the image by;
  
  performing a 3-D recursive search;
  
  performing a motion vector correction;
  
  performing a block erosion to reduce blockiness in the progressive scan image;
  
  filtering the signal in three stages;
  
  in the first stage selecting between using static pixels data and moving pixels data from a next field;
  
  in the second stage selecting a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; and
  
  in the third stage combining an intra-field interpolation with the more valid set of data selected by the second stage.

33. A method for converting an interlaced image to a progressive scan image, the method comprsing:
- providing an input signal corresponding to an image;
  
  prefiltering the input signal with a spatial line averaging prefilter;
  
  estimating motion in the image by;
  
  assuming that a motion vector for an object between a previous field and a current field is the same as a motion vector for the object between the current field and a next field;
  
  performing a 3-D recursive search;
  
  performing a motion vector correction in which the error function penalizes a candidate vector based on a length of a difference vector between the candidate vector and a plurality of neighboring vectors;
  
  performing a block erosion to reduce blockiness in the progressive scan image;
  
  filtering the signal in three stages;
  
  in the first stage selecting between using static pixels data and moving pixels data from a next field;
  
  in the second stage selecting a more valid set of data between motion compensated data from a previous field and the pixels selected by the first stage; and
  
  in the third stage combining an intra-field interpolation with the more valid set of data selected by the second stage.

Specification

Resources

Litigation Campaign Assessment

Current Assignee
MaxLinear Communications, LLC (f/k/a Entropic Communications, LLC) (Maxlinear Incorporated)
Original Assignee
Micronas Semiconductors Incorporated (TDK Corporation)
Inventors
Li, Wenhua, Handjojo, Benitius M.

Granted Patent

US 6,940,557 B2
Time in Patent Office

Days
Field of Search
US Class Current

348/452
CPC Class Codes

H04N 5/145   Movement estimation for vid...

H04N 7/012   Conversion between an inter...

H04N 7/014   involving the use of motion...

Adaptive interlace-to-progressive scan conversion algorithm

First Claim

12 Assignments

0 Petitions

Accused Products

Abstract

Citations

33 Claims

Specification

Solutions

Use Cases

Quick Links

Adaptive interlace-to-progressive scan conversion algorithm

First Claim

12 Assignments

Subscription Required

Subscription Required

0 Petitions

Subscription Required

Accused Products

Subscription Required

Abstract

Citations

33 Claims

Specification

Subscription Required

Solutions

Use Cases

Quick Links