“Don't care” pixel interpolation

• US 7,043,079 B2
• Filed: 06/26/2002
• Issued: 05/09/2006
• Est. Priority Date: 04/25/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Malvar, Henrique S., Simard, Patrice Y., Rinker, James Russell
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US10/180,800
• Publication Number:
  US 20030202700A1
• Time in Patent Office: 1,413 Days
• Field of Search: 348/586, 348/587, 348/590, 348/571, 348/584, 348/598, 382/181, 382/190, 382/195, 382/199, 382/128, 382/162, 382/254, 382/266, 382/283, 382/284, 382/264, 358/448, 358/464
• US Class Current: 382/195
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

A system and method facilitating image smoothing is provided. The invention includes an image processor having an image receptor and an image smoother. The invention provides for the image smoother to alter the value of a don'"'"'t care pixel based, at least in part, upon a weighted average of care pixels.

Claims

1. An image processor, comprising:

an image receptor adapted to receive an image, the image comprising care pixels and don'"'"'t care pixels, identification of care pixels and don'"'"'t care pixels being based, at least in part, upon a binary mask; and

an image smoother adapted to smooth the image by altering a value of at least one don'"'"'t care pixel based, at least in part, on a weighted average of care pixels, the image smoother utilizing the equation;

$v=\frac{v_1<br><br>{\mathrm{e<br><br>}}^{-d_1}+v_2<br><br>{\mathrm{e<br><br>}}^{-d_2}+v_3<br><br>{\mathrm{e<br><br>}}^{-d_3}+v_4<br><br>{\mathrm{e<br><br>}}^{-d_4}}{{\mathrm{e<br><br>}}^{-d_1}+{\mathrm{e<br><br>}}^{-d_2}+{\mathrm{e<br><br>}}^{-d_3}+{\mathrm{e<br><br>}}^{-d_4}}$

where v is the smoothed value of the at least one don'"'"'t care pixel,

v₁ is the value of a first care pixel,

v₂ is the value of a second care pixel,

v₃ is the value of a third care pixel,

v₄ is the value of a fourth care pixel,

d₁ is the distance from the at least one don'"'"'t care pixel to the first care pixel,

d₂ is the distance from the at least one don'"'"'t care pixel to the second care pixel,

d₃ is the distance from the at least one don'"'"'t care pixel to the third care pixel,

d₄ is the distance from the at least one don'"'"'t care pixel to the fourth care pixel, for alteration of the at least one don'"'"'t care pixel.

2. The image processor of claim 1, further comprising an image separator adapted to separate the image from a document image input, the image being based on at least one of a background and a foreground, separation being based, at least in part, upon information received from the binary mask.

3. The image processor of claim 2, the binary mask being based, at least in part, upon a partition of the document image based, at least in part, upon minimization of pixel energy of at least one of the foreground and the background.

4. The image processor of claim 1, the image smoother providing an output having at least one altered don'"'"'t care pixel.

5. A photocopier employing the image processor of claim 1.

6. A document scanner employing the image processor of claim 1.

7. An optical character recognition system employing the image processor of claim 1.

8. A personal digital assistant employing the image processor of claim 1.

9. A fax machine employing the image processor of claim 1.

10. A digital camera employing the image processor of claim 1.

11. A digital video camera employing the image processor of claim 1.

12. A segmented layered image system employing the image processor of claim 1.

13. A method for smoothing an image, comprising:

determining a first distance to a first care pixel and a first value of the first care pixel;

determining a second distance to a second care pixel and a second value of the second care pixel;

calculating a smoothed value for a don'"'"'t care pixel based, at least in part, upon a weighted average of the first distance, the first value, the second distance and the second value, wherein calculation of the smoothed value of the don'"'"'t care pixel is based, at least in part, upon the equation;

$v=\frac{v_1<br><br>{\mathrm{e<br><br>}}^{-d_1}+v_2<br><br>{\mathrm{e<br><br>}}^{-d_2}}{{\mathrm{e<br><br>}}^{-d_1}+{\mathrm{e<br><br>}}^{-d_2}}$

where v is the smoothed value of the don'"'"'t care pixel,

v₁ is the first value,

v₂ is the second value,

d₁ is the first distance, and

d₂ is the second distance; and

altering the value of the don'"'"'t care pixel based, at least in part, upon the calculated smoothed value.

14. The method of claim 13, calculation of the smoothed value of the don'"'"'t care pixel being based, at least in part, upon a two-pass process whose results are weighted by the equation:

$v_{\mathrm{FINAL}}=\frac{v_{\mathrm{FIRST}<br><br><br><br>\mathrm{PASS}}<br><br>w<br><br>\left(d_{1<br><br>L},d_{2<br><br>T}\right)+v_{\mathrm{SECOND}<br><br><br><br>\mathrm{PASS}}<br><br>w<br><br>\left(d_{1<br><br>R},d_{2<br><br>B}\right)}{w<br><br>\left(d_{1<br><br>L},d_{2<br><br>T}\right)+w<br><br>\left(d_{1<br><br>R},d_{2<br><br>B}\right)}$

where v_FINAL is the smoothed value of the don'"'"'t care pixel,

v_{FIRST PASS} is the value from the first pass,

v_{SECOND PASS} is the value from the second pass,

d_1L, d_2T are the two distances from the first pass,

w(d_1L, d_2T) is a weight factor computed from the two distances from the first pass,

d_1R, d_2S are the two distances from the second pass, and

w(d_1R, d_2B) is a weight factor computed from the two distances from the second pass.

15. A method for smoothing an image, comprising:

determining a horizontal distance to a left care pixel and a value of the left care pixel;

determining a horizontal distance to a right care pixel and a value of the right care pixel;

determining a vertical distance to a top care pixel and a value of the top cure pixel;

determining a vertical distance to a bottom care pixel and a value of the bottom care pixel;

calculating a smoothed value for a don'"'"'t care pixel based, at least in part, upon at least one of the horizontal distance to the left care pixel, the value of the left care pixel, the horizontal distance to the right care pixel, the value of the right care pixel, the vertical distance to the top care pixel, the value of the top care pixel, the vertical distance to the bottom care pixel and the value of the bottom care pixel, wherein calculation of the smoothed value is based, at least in part, upon the equation;

$v=\frac{v_1<br><br>{\mathrm{e<br><br>}}^{-d_1}+v_2<br><br>{\mathrm{e<br><br>}}^{-d_2}+v_3<br><br>{\mathrm{e<br><br>}}^{-d_3}+v_4<br><br>{\mathrm{e<br><br>}}^{-d_4}}{{\mathrm{e<br><br>}}^{-d_1}+{\mathrm{e<br><br>}}^{-d_2}+{\mathrm{e<br><br>}}^{-d_3}+{\mathrm{e<br><br>}}^{-d_4}}$

where v is the smoothed value of the don'"'"'t care pixel,

v₁ is the value of the left care pixel,

v₂ is the value of the right care pixel,

v₃ is the value of the top care pixel,

v₄ is the value of the bottom care pixel,

d₁ is the horizontal distance to the left care pixel,

d₂ is the horizontal distance to the right care pixel,

d₃ is the vertical distance to the top care pixel,

d₄ is the vertical distance to the bottom care pixel; and

altering the don'"'"'t care pixel value based, at least in part, upon the calculated smoothed value.

16. A method for smoothing an image, comprising:

scanning the image from top to bottom and from left to right;

determining a horizontal distance to a left care pixel and a value of the left care pixel;

determining a vertical distance to a top care pixel and a value of the top care pixel;

calculating a first smoothed value for a don'"'"'t care pixel based, at least in part, upon at least one of the horizontal distance to the left care pixel, the value of the left care pixel, the vertical distance to the top care pixel, the value of the top care pixel; and

,

altering the don'"'"'t care pixel value based, at least in part, upon the calculated first smoothed value;

scanning the image from bottom to top and from right to left;

determining a horizontal distance to a right care pixel and a value of the right care pixel;

determining a vertical distance to a bottom care pixel and a value of the bottom care pixel;

calculating a second smoothed value for the don'"'"'t care pixel based, at least in part, upon at least one of the horizontal distance to the right care pixel, the value of the right care pixel, the vertical distance to the bottom care pixel and the value of the bottom care pixel,

wherein calculation of the first or second smoothed value of the don'"'"'t care pixel is base, at least in part, upon the equation;

$v=\frac{v_1<br><br>{\mathrm{e<br><br>}}^{-d_1}+v_2<br><br>{\mathrm{e<br><br>}}^{-d_2}}{{\mathrm{e<br><br>}}^{-d_1}+{\mathrm{e<br><br>}}^{-d_2}}$

where v is a smoothed value of the don'"'"'t care pixel, the smoothed value being one of the first smoothed value or the second smoothed value,

v₁ is a first value, the first value being the value of one of the left care pixel and the right care pixel,

v₂ is a second value, the second value the value of one of the top care pixel and the bottom care pixel,

d₁ is a first distance, being the horizontal distance to one of the left care pixel and the right care pixel, and

d₂ is a second distance, the second value being the vertical distance to one of the top care pixel and the bottom care pixel; and

altering the don'"'"'t care pixel value based, at least in part, upon the calculated first and second smoothed values.

17. The method of claim 16, altering of the don'"'"'t care pixel value being based, at least in part, upon the equation:

$v_{\mathrm{FINAL}}=\frac{v_{\mathrm{FIRST}<br><br><br><br>\mathrm{PASS}}<br><br>w<br><br>\left(d_{1<br><br>L},d_{2<br><br>T}\right)+v_{\mathrm{SECOND}<br><br><br><br>\mathrm{PASS}}<br><br>w<br><br>\left(d_{1<br><br>R},d_{2<br><br>B}\right)}{w<br><br>\left(d_{1<br><br>L},d_{2<br><br>T}\right)+w<br><br>\left(d_{1<br><br>R},d_{2<br><br>B}\right)}$

where v_FINAL is the smoothed value of the don'"'"'t care pixel,

v_{FIRST PASS} is the first smoothed value,

v_{SECOND PASS} is the second smoothed value,

d_1L is the horizontal distance to the left care pixel,

d_2T is the vertical distance to the top care pixel,

w(d_1L, d_2T) is a weight factor computed from the horizontal distance to the left care pixel and the vertical distance to the top care pixel,

d_1R is the horizontal distance to the right care pixel,

d_2B is the vertical distance to the bottom care pixel, and

w(d_1R, d_2B) is a weight factor computed from the two horizontal distance to the right care pixel and the vertical distance to the bottom care pixel.

18. A segmented layered image system, comprising:

a mask separator adapted to assign pixels of a document image into at least one of a foreground and a background, the assignment information stored in a binary mask; and

, an image processor comprising an image receptor adapted to receive an image comprising care pixels and don'"'"'t care pixels, identification of care pixels and don'"'"'t care pixels being based, at least in part, upon the binary mask, the image processor further comprising an image separator adapted to separate the image from the document image, the image being based on at least one of a background and a foreground, separation being based, at least in part, upon information in the binary mask, the image processor further comprising an image smoother adapted to smooth the image, the image smoother altering a value of a don'"'"'t care pixel based, at least in part, on a weighted average of care pixels, the image smoother utilizing the equation;

$v=\frac{v_1<br><br>e^{-d_1}+v_2<br><br>e^{-d_2}+v_3<br><br>e^{-d_3}+v_4<br><br>e^{-d_4}}{e^{-d_1}+e^{-d_2}+e^{-d_3}+e^{-d_4}}$

where v is the smoothed value of the don'"'"'t care pixel,

v₁ is the value of a first care pixel,

v₂ is the value of a second care pixel,

v₃ is the value of a third care pixel,

v₄ is the value of a fourth care pixel,

d₁ is the distance from the don'"'"'t care pixel to the first care pixel,

d₂ the distance from the don'"'"'t care pixel to the second care pixel,

d₃ is the distance from the don'"'"'t care pixel to the third care pixel,

d₄ is the distance from the don'"'"'t care pixel to the fourth care pixel, for alteration of the don'"'"'t care pixel.

19. A data packet adapted to be transmitted between two or more computer components that facilitates image smoothing, the data packet being stored on a computer readable medium, the data packet comprising:

computer-executable instructions for altering at least one altered don'"'"'t care pixel, the alteration being based, at least in part, on a weighted average of care pixels and on the equation;

$v=\frac{v_1<br><br>e^{-d_1}+v_2<br><br>e^{-d_2}+v_3<br><br>e^{-d_3}+v_4<br><br>e^{-d_4}}{e^{-d_1}+e^{-d_2}+e^{-d_3}+e^{-d_4}}$

where v is the smoothed value of the at least one don'"'"'t care pixel,

v₁ is the value of a first care pixel,

v₂ is the value of a second care pixel,

v₃ is the value of a third care pixel,

v₄ is the value of a fourth care pixel,

d₁ is the distance from the don'"'"'t care pixel to the first care pixel,

d₂ is the distance from the don'"'"'t care pixel to the second care pixel,

d₃ is the distance from the don'"'"'t care pixel to the third care pixel,

d₄ is the distance from the don'"'"'t care pixel to the fourth care pixel.

20. A computer readable medium storing computer executable components of a system for image smoothing, comprising:

an image receptor component adapted to receive an image comprising care pixels and don'"'"'t care pixels, identification of care pixels and don'"'"'t care pixels being based, at least in part, upon a binary mask; and

an image smoother component adapted to smooth the image, the image smoother altering a value of a don'"'"'t care pixel based, at least in part, on a weighted average of care pixels, the image smoother component utilizing the equation;

$v=\frac{v_1<br><br>e^{-d_1}+v_2<br><br>e^{-d_2}+v_3<br><br>e^{-d_3}+v_4<br><br>e^{-d_4}}{e^{-d_1}+e^{-d_2}+e^{-d_3}+e^{-d_4}}$

where v is the smoothed value of the don'"'"'t care pixel,

v₁ is the value of a first care pixel,

v₂ is the value of a second care pixel,

v₃ is the value of a third care pixel,

v₄ is the value of a fourth care pixel,

d₁ is the distance from the don'"'"'t care pixel to the first care pixel,

d₂ is the distance from the don'"'"'t care pixel to the second care pixel,

d₃ is the distance from the don'"'"'t care pixel to the third care pixel,

d₄ is the distance from the don'"'"'t care pixel to the fourth care pixel, for alteration of the don'"'"'t care pixel.

21. A computer readable medium storing computer executable instructions operable to perform a method for smoothing an image, comprising:

determining a first distance to a first care pixel and a first value of the first care pixel;

determining a second distance to a second care pixel and a second value of the second care pixel;

calculating a smoothed value for a don'"'"'t care pixel based, at least in part, upon a weighted average of the first distance, the first value, the second distance and the second value, wherein calculation of the smoothed value of the don'"'"'t care pixel is based, at least in part, upon the equation;

$v=\frac{v_1<br><br>e^{-d_1}+v_2<br><br>e^{-d_2}}{e^{-d_1}+e^{-d_2}}$

where v is the smoothed value of the don'"'"'t care pixel,

v₁ is the first value,

v₂ is the second value,

d₁ is the first distance, and

d₂ is the second distance; and

altering the value of the don'"'"'t care pixel based, at least in part, upon the calculated smoothed value.

22. An image processor, comprising:

means for determining a first distance to a first care pixel and a first value of the first care pixel;

means for determining a second distance to a second care pixel and a second value of the second care pixel;

means for calculating a smoothed value for a don'"'"'t care pixel based, at least in part, upon a weighted average of the first distance, the first value, the second distance and the second value, wherein calculation of the smoothed value of the don'"'"'t care pixel is based, at least in part, upon the equation;

$v=\frac{v_1<br><br>e^{-d_1}+v_2<br><br>e^{-d_2}}{e^{-d_1}+e^{-d_2}}$

where v is the smoothed value of the don'"'"'t care pixel,

v₁ is the first value,

v₂ is the second value,

d₁ is the first distance, and

d₂ the second distance; and

means for altering the value of the don'"'"'t care pixel based, at least in part, upon the calculated smoothed value.