System and process for image rescaling using adaptive interpolation kernel with sharpness and overshoot control

US 20100165204A1
Filed: 12/23/2009
Published: 07/01/2010
Est. Priority Date: 12/31/2008
Status: Active Grant

First Claim

Patent Images

1. A digital video rescaling system comprising:

an input module configured to receive an input image data;

an adaptive spline interpolation kernel coupled to the input module, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel based on the input image data;

a sharpness control module coupled to the adaptive spline interpolation kernel, wherein the sharpness control module is configured to generate sharpness control parameters to control the adaptive spline interpolation kernel; and

an overshoot control module coupled to the adaptive spline interpolation kernel, wherein the overshoot control module is configured to generate overshoot control parameters to control the adaptive spline interpolation kernel.

View all claims

1 Assignment

Timeline View

Assignment View

0 Petitions

Accused Products

Abstract

Provided are a digital video rescaling system, a method of rescaling video images, and a chip comprising a computer executable medium embedded therein computer executable instructions for rescaling video images.

29 Citations

View as Search Results

20 Claims

1. A digital video rescaling system comprising:
- an input module configured to receive an input image data;
  
  an adaptive spline interpolation kernel coupled to the input module, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel based on the input image data;
  
  a sharpness control module coupled to the adaptive spline interpolation kernel, wherein the sharpness control module is configured to generate sharpness control parameters to control the adaptive spline interpolation kernel; and
  
  an overshoot control module coupled to the adaptive spline interpolation kernel, wherein the overshoot control module is configured to generate overshoot control parameters to control the adaptive spline interpolation kernel.
- View Dependent Claims (2, 3, 4, 5, 6, 7)
- - 2. The digital video rescaling system of claim 1, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel y₁(x) based on the following equation:
    - $y_{1} (x) = \sum_{n = A, B, C, D} y_{n} * f_{n} (s, Kshp),$ wherein y_A, y_B, y_Cand y_Dare four support pixels from input image data, wherein s is the distance from interpolation position to the position of the support pixel y_b, wherein Kshp is a global sharpness control parameter generated by the sharpness control module, and wherein f_n(s,Kshp) (n=A, B, C, D) are four control synthesis functions which can be expressed by the following equations;
      
      f_A(s,Kshp)=Kshp*(−
      
      s³+2*s²−
      
      s)/2;
      
      f_B(s,Kshp)=(3*Kshp*s³−
      
      5*Kshp*s²+2*(Kshp−
      
      1)*s+2)/2;
      
      f_C(s,Kshp)=(−
      
      3*Kshp*s³+4*Kshp*s²+(2−
      
      (Kshp)*s)/2; and
      
      f_D(s,Kshp)=Kshp*(s³−
      
      s²)/2.
  - 3. The digital video rescaling system of claim 2, wherein the overshoot control module comprises:
    - a max-min calculation unit configured to receive the data of pixels y_Band y_C, and configured to calculate the maximum value and the minimum value of the support pixels y_Band y_c;
      
      a comparator coupled to the max-min calculation unit and adaptive spline interpolation kernel, wherein the comparator is configured to compare the maximum value and the minimum value of the support pixels y_Band y_cwith the interpolated pixel y₁(x) to generate an output y₂(x) according the following equation;
      
      $y_{2} (x) = {\begin{matrix} \max (y_{B}, y_{C}) & if (y_{1} (x) > \max (y_{B}, y_{C})) \\ \min (y_{B}, y_{C}) & if (y_{1} (x) < \min (y_{B}, y_{C})) \\ y_{1} (x) & else; \end{matrix}$ a subtractor coupled to the max-min calculation unit and the comparator, configured to subtract the outputs y₁(x) and y₂(x), thereby generating an output (y₁(x)−
      
      y₂(x)); and
      
      a multiplier coupled to the subtractor, wherein the multiplier is configured to multiply the output (y₁(x)−
      
      y₂(x)) by an overshoot control parameter Kovs to generate an output Kovs*(y₁(x)−
      
      y₂(x)); and
      
      whereby the final output y(x)=Kovs*(y₁(x)−
      
      y₂(x))+y₂(x) is based on the addition of y₂(x) to Kovs*(y₁(x)−
      
      y₂(x)).
  - 4. The digital video rescaling system of claim 1 further comprising a local sharpness control module to adapt the sharpness control module to the local image content of the input image data, wherein the local sharpness control module comprises:
    - a local gradient estimation module coupled to the input module, wherein the local gradient estimation module is configured to generate a local gradient magnitude based on the input image data;
      
      a noise level estimator coupled to the input module, wherein the noise level estimator is configured to generate a noise level based on the input image data; and
      
      a local sharpness control parameter function module configured to control the sharpness control module using a transfer function based on the local gradient magnitude and the noise level.
  - 5. The digital video rescaling system of claim 4, wherein the transfer function of the local sharpness control parameter function module can be expressed as:
    - Kshp(Gn)=AKshp(Gn)+GKshp, wherein GKshp is a fixed value that is defined by a global sharpness control parameter Kshp, AKshp(Gn) is a variable value that is defined by the control parameters max K, min K, K1, K2, K3, and Sigma, wherein Sigma is the noise level, wherein Gn is the local gradient magnitude; and
      
      wherein AKshp(Gn) can be expressed by a transfer function according to the following equation;
      
      $AKshp (G_{n}) = {\begin{matrix} \frac{\langle \min K \rangle * G_{n}}{Sigma} + \min K & if (G_{n} \leq Sigma) \\ \frac{\max K * (G_{n} - Sigma)}{K 1 - Sigma} & if (Sigma < G_{n} \leq K 1) \\ \max K & if (K 1 < G_{n} \leq K 2) \\ \frac{\max K * (K 3 - G_{n})}{K 3 - K 2} & if (K 2 < G_{n} \leq K 3) \\ 0 & else . \end{matrix}$
  - 6. The digital video rescaling system of claim 1 further comprising an anti-aliasing filter coupled between the input module and the adaptive spline interpolation kernel, wherein the anti-aliasing filter comprises a set of low pass filters that are configured to reduce aliasing artifacts during image downscaling.
  - 7. The digital video rescaling system of claim 6 further comprising a switch device coupled between the input module and the anti-aliasing filter, wherein the switch device is configured to activate the anti-aliasing filter during downscaling, and wherein the switch device is configured to deactivate the anti-aliasing filter during upscaling.

8. A method of rescaling video images including:
- receiving data corresponding to an input image;
  
  calculating interpolated pixel values based on the input image data by using an adaptive spline interpolation kernel;
  
  generating sharpness control parameters to control the adaptive spline interpolation kernel by using a sharpness control module; and
  
  generating overshoot control parameters to control the adaptive spline interpolation kernel by using an overshoot control module.
- View Dependent Claims (9, 10, 11, 12, 13)
- - 9. The method of claim 8, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel y₁(x) based on the following equation:
    - $y_{1} (x) = \sum_{n = A, B, C, D} y_{n} * f_{n} (s, Kshp),$ wherein y_A, y_B, y_Cand y_Dare four support pixels from input image data, wherein s is the distance from interpolation position to the position of the support pixel y_b, wherein Kshp is a global sharpness control parameter generated by the sharpness control module, and wherein f_n(s,Kshp) (n=A, B, C, D) are four control synthesis functions which can be expressed by the following equations;
      
      f_A(s,Kshp)=Kshp*(−
      
      s³+2*s²−
      
      s)/2;
      
      f_B(s,Kshp)=(3*Kshp*s³−
      
      5*Kshp*s²+2*(Kshp−
      
      1)*s+2)/2;
      
      f^C(s,Kshp)=(−
      
      3*Kshp*s³+4*Kshp*s²+(2−
      
      (Kshp)*s)/2; and
      
      f_D(s,Kshp)=Kshp*(s³−
      
      s²)/2.
  - 10. The method of claim 9, wherein the operation of generating overshoot control parameters to control the adaptive spline interpolation kernel comprises:
    - calculating a maximum value and a minimum value of the support pixels y_Band y_c;
      
      comparing the maximum value and the minimum value of the support pixels y_Band y_Cwith the interpolated pixel y₁(x) to generate an output y₂(x) according the following equation;
      
      $y_{2} (x) = {\begin{matrix} \max (y_{B}, y_{C}) & if (y_{1} (x) > \max (y_{B}, y_{C})) \\ \min (y_{B}, y_{C}) & if (y_{1} (x) < \min (y_{B}, y_{C})) \\ y_{1} (x) & else; \end{matrix}$ subtracting the outputs y₁(x) and y₂(x) to generate an output (y₁(x)−
      
      y₂(x));
      
      multiplying the output (y₁(x)−
      
      y₂(x)) by an overshoot control parameter Kovs to generate an output Kovs*(y₁(x)−
      
      y₂(x)); and
      
      generating the final output y(x)=Kovs*(y₁(x)−
      
      y₂(x))+y₂(x) by adding y₂(x) to the output Kovs*(y₁(x)−
      
      y₂(x)).
  - 11. The method of claim 8 further comprising:
    - generating a local gradient magnitude based on the input image data;
      
      generating a noise level based on the input image data; and
      
      controlling the sharpness control module by using a transfer function based on the local gradient magnitude and the noise level.
  - 12. The method of claim 11, wherein the transfer function can be expressed as:
    - Kshp(Gn)=AKshp(Gn)+GKshp, wherein GKshp is a fixed value that is defined by a global sharpness control parameter Kshp, AKshp(Gn) is a variable value that is defined by the control parameters max K, min K, K1, K2, K3, and Sigma, wherein Sigma is the noise level, wherein Gn is the local gradient magnitude, andwherein AKshp(Gn) can be expressed by a transfer function;
      
      $AKshp (G_{n}) = {\begin{matrix} \frac{\langle \min K \rangle * G_{n}}{Sigma} + \min K & if (G_{n} \leq Sigma) \\ \frac{\max K * (G_{n} - Sigma)}{K 1 - Sigma} & if (Sigma < G_{n} \leq K 1) \\ \max K & if (K 1 < G_{n} \leq K 2) \\ \frac{\max K * (K 3 - G_{n})}{K 3 - K 2} & if (K 2 < G_{n} \leq K 3) \\ 0 & else . \end{matrix}$
  - 13. The method of claim 8 further comprising:
    - selecting a set of filter coefficients from a low pass filter bank according to a downscaling factor; and
      
      applying the set of filter coefficients on the input image data to reduce anti-aliasing effects during downscaling.

14. A chip comprising a computer executable medium embedded therein computer executable instructions for rescaling video images, wherein the instructions comprise:
- receiving data corresponding to an input image;
  
  calculating interpolated pixel values based on the input image data by using an adaptive spline interpolation kernel;
  
  generating sharpness control parameters to control the adaptive spline interpolation kernel by using a sharpness control module; and
  
  generating overshoot control parameters to control the adaptive spline interpolation kernel by using an overshoot control module.
- View Dependent Claims (15, 16, 17, 18, 19, 20)
- - 15. The chip of claim 14, wherein the adaptive spline interpolation kernel is configured to generate an interpolated pixel y₁(x) based on the following equation:
    - $y_{1} (x) = \sum_{n = A, B, C, D} y_{n} * f_{n} (s, Kshp),$ wherein y_A, y_B, y_Cand y_D) are four support pixels from input image data, wherein s is the distance from interpolation position to the position of the support pixel y_b, wherein Kshp is a global sharpness control parameter generated by the sharpness control module, and wherein f_n(s,Kshp) (n=A, B, C, D) are four control synthesis functions which can be expressed by the following equations;
      
      f_A(s,Kshp)=Kshp*(−
      
      s³+2*s²−
      
      s)/2;
      
      f_B(s,Kshp)=(3*Kshp*s³−
      
      5*Kshp*s²+2*(Kshp−
      
      1)*s+2)/2;
      
      f_C(s,Kshp)=(−
      
      3*Kshp*s³+4*Kshp*s²+(2−
      
      (Kshp)*s)/2; and
      
      f_D(s,Kshp)=Kshp*(s³−
      
      s²)/2.
  - 16. The chip of claim 15, wherein the operation of generating overshoot control parameters to control the adaptive spline interpolation kernel comprises:
    - calculating a maximum value and a minimum value of the support pixels y_Band y_C;
      
      comparing the maximum value and the minimum value of the support pixels y_Band y_Cwith the interpolated pixel y₁(x) to generate an output y₂(x) according the following equation;
      
      $y_{2} (x) = {\begin{matrix} \max (y_{B}, y_{C}) & if (y_{1} (x) > \max (y_{B}, y_{C})) \\ \min (y_{B}, y_{C}) & if (y_{1} (x) < \min (y_{B}, y_{C})) \\ y_{1} (x) & else; \end{matrix}$ subtracting the outputs y₁(x) and y₂(x) to generate an output (y₁(x)−
      
      y₂(x));
      
      multiplying the output (y₁(x)−
      
      y₂(x)) by an overshoot control parameter Kovs to generate an output Kovs*(y₁(x)−
      
      y₂(x)); and
      
      generating the final output y(x)=Kovs*(y₁(x)−
      
      y₂(x))+y₂(x) by adding y₂(x) to the output Kovs*(y₁(x)−
      
      y₂(x)).
  - 17. The chip of claim 14, wherein the instructions further comprise:
    - generating a local gradient magnitude based on the input image data;
      
      generating a noise level based on the input image data; and
      
      controlling the sharpness control module by using a transfer function based on the local gradient magnitude and the noise level.
  - 18. The chip of claim 17, wherein the transfer function can be expressed as:
    - Kshp(Gn)=AKshp(Gn)+GKshp, wherein GKshp is a fixed value that is defined by a global sharpness control parameter Kshp, AKshp(Gn) is a variable value that is defined by the control parameters max K, min K, K1, K2, K3, and Sigma, wherein Sigma is the noise level, wherein Gn is the local gradient magnitude, andwherein AKshp(Gn) can be expressed by a transfer function;
      
      $AKshp (G_{n}) = {\begin{matrix} \frac{\langle \min K \rangle * G_{n}}{Sigma} + \min K & if (G_{n} \leq Sigma) \\ \frac{\max K * (G_{n} - Sigma)}{K 1 - Sigma} & if (Sigma < G_{n} \leq K 1) \\ \max K & if (K 1 < G_{n} \leq K 2) \\ \frac{\max K * (K 3 - G_{n})}{K 3 - K 2} & if (K 2 < G_{n} \leq K 3) \\ 0 & else . \end{matrix}$
  - 19. The chip of claim 14, wherein the instructions further comprise:
    - selecting a set of filter coefficients from a low pass filter bank according to a downscaling factor; and
      
      applying the set of filter coefficients on the input image data to reduce anti-aliasing effects during downscaling.
  - 20. The chip of claim 19, wherein the instructions further comprise:
    - deactivating the set of filter coefficients on the input image data during upscaling.

Specification

Resources

Litigation Campaign Assessment

Current Assignee
STMicroelectronics Asia Pacific Pte Limited (STMicroelectronics NV), STMicroelectronics SA (STMicroelectronics NV)
Original Assignee
STMicroelectronics Asia Pacific Pte Limited (STMicroelectronics NV), STMicroelectronics SA (STMicroelectronics NV)
Inventors
Lebowsky, Fritz, Huang, Yong, Hui, Lucas, Wang, Haiyun

Granted Patent

US 8,587,726 B2
Time in Patent Office

Days
Field of Search
US Class Current

348/581
CPC Class Codes

G06T 2200/12   involving antialiasing

G06T 2207/10016   Video; Image sequence

G06T 2207/20192   Edge enhancement; Edge pres...

G06T 3/4007   based on interpolation, e.g...

G06T 5/002   Denoising; Smoothing noise ...

G06T 5/003   Deblurring; Sharpening vibr...

G06T 5/20   using local operators

G06T 5/70   Denoising; Smoothing

G06T 5/73   Deblurring; Sharpening

System and process for image rescaling using adaptive interpolation kernel with sharpness and overshoot control

First Claim

1 Assignment

0 Petitions

Accused Products

Abstract

29 Citations

20 Claims

Specification

Solutions

Use Cases

Quick Links

System and process for image rescaling using adaptive interpolation kernel with sharpness and overshoot control

First Claim

1 Assignment

Subscription Required

Subscription Required

0 Petitions

Subscription Required

Accused Products

Subscription Required

Abstract

29 Citations

20 Claims

Specification

Subscription Required

Solutions

Use Cases

Quick Links