Apparatus and methods for image restoration

• US 8,073,277 B2
• Filed: 06/21/2007
• Issued: 12/06/2011
• Est. Priority Date: 06/21/2007
• Status: Expired due to Fees

• Current Assignee: University of Southern Mississippi
• Original Assignee: University of Southern Mississippi
• Inventors: Huan, Xiaoli, Murali, Beddhu, Ali, Adel L.
• Primary Examiner(s): Bhatnagar, Anand
• Assistant Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US11/812,814
• Publication Number:
  US 20080317375A1
• Time in Patent Office: 1,629 Days
• Field of Search: None
• US Class Current: 382/254
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

The present invention relates to an apparatus and methods for performing image restoration. More specifically, the present invention relates to an apparatus and methods which are capable of quickly restoring various sizes of selected areas of an image. In one embodiment, a user may select or mark a target region in an image which is to be restored. A source window may then be generated and a pixel from within the target region may be selected for restoration. The intensity distribution of pixels surrounding the selected pixel may be compared with other pixels in the image and a pixel to be used for restoration of the selected pixel may be chosen based on the comparison of the intensity distributions. Once a pixel is chosen for restoration, the selected pixel and its surrounding pixels may be restored using the intensity distribution of the chosen pixel and its surrounding pixels. The method may continue in this manner until all pixels within the target region have been restored.

Claims

1. A method comprising the steps of:

receiving an image having at least one marked region, each marked region of the image having a plurality of pixels;

generating a source window, the source window enclosing at least one of the marked regions of the image;

selecting a pixel from within one of the at least one marked regions enclosed by the source window;

generating a fill window around the selected pixel;

generating at least one search window around each pixel outside the at least one marked region of the image and inside the source window, each at least one search window having approximately the same size as the fill window;

comparing the intensity distribution of at least a portion of the fill window with the intensity distribution of the corresponding portion of each at least one search window;

selecting at least one search window from among the generated at least one search window based on the comparison of the intensity distribution of the fill window with the intensity distribution of each at least one search window;

filling at least a portion of the pixels in the fill window with corresponding pixels in the selected at least one search window;

wherein the selected pixel is selected based on a fill order; and

wherein the fill order is determined by a time of arrival computation using the Eikonal equation initialized with the contour of the edges of the originally received at least one marked region and a prescribed speed function based on image intensity gradient.

2. The method of claim 1, wherein the step of filling includes filling the selected pixel with a corresponding pixel in the selected search window.

3. The method of claim 1, further comprising the step of removing the selected pixel from the at least one marked region, thereby shrinking the size of the at least one marked region.

4. The method of claim 1, wherein the at least one marked region is marked by a user.

5. The method of claim 1, wherein a single source window is generated enclosing all of the marked regions of the image.

6. The method of claim 1, wherein a number of generated source windows is different from a number of marked regions.

7. The method of claim 1, wherein the steps of selecting a pixel, generating a fill window, generating a search window, comparing the intensity, selecting a search window and filling are performed for each pixel in the at least one marked region of the image.

8. The method of claim 1, wherein the selected pixel is selected based on a fill order.

9. The method of claim 1, wherein for pixels having the same computed time of arrival, the fill order is further determined based on the number of known pixels in the fill window such that a pixel having a higher number of known pixels in the fill window is filled first.

10. The method of claim 1, wherein the size of the fill window and the search windows are based on a first parameter.

11. The method of claim 10, wherein the first parameter is a window size selected by a user.

12. The method of claim 1, wherein the size of the source window is based on a second parameter.

13. The method of claim 12, wherein the second parameter is an offset number of pixels selected by a user.

14. The method of claim 1, wherein the step of comparing the intensity distribution of the fill window with the intensity distribution of each search window is performed by calculating the sum of the squared differences of the intensities of the pixels in the fill window and the corresponding pixels in each search window.

15. The method of claim 14, wherein the step of selecting a search window includes the step of selecting the search window having the smallest sum of the squared differences of the intensities of the pixels in the fill window and the corresponding pixels in the search window.

16. The method of claim 1, wherein the step of comparing the intensity distribution of the fill window with the intensity distribution of each search window is performed by calculating the sum of the absolute differences of the intensities of the pixels in the fill window and the corresponding pixels in each search window.

17. The method of claim 16, wherein the step of selecting a search window includes the step of selecting the search window having the smallest sum of the absolute differences of the intensities of the pixels in the fill window and the corresponding pixels in the search window.

18. The method of claim 1, wherein the image is a digital image.

19. The method of claim 1, wherein the image is a frame of a video.

20. The method of claim 1, wherein the image is a photograph.

21. The method of claim 1, wherein the method is used for image restoration.

22. The method of claim 1, wherein the image is used for the removal of a portion of an image.

23. The method of claim 1, wherein the step of filling comprises averaging the intensity distribution of each pixel in the unmarked regions of the fill window with the intensity distribution of the corresponding regions in the selected search window.

24. The method of claim 1, wherein the step of filling comprises the step of replacing the intensity distribution in the unmarked regions of the fill window with the intensity distribution of the corresponding regions in the selected search window.

25. A computer program product, stored on a non-transitory computer-readable storage medium, comprising computer program code for instructing a computer to perform a method comprising the steps of:

receiving an image having at least one marked region, each marked region of the image having a plurality of pixels;

generating a source window, the source window enclosing at least one of the marked regions of the image;

selecting a pixel from within one of the at least one marked regions enclosed by the source window;

generating a fill window around the selected pixel;

generating at least one search window around each pixel outside the at least one marked region of the image and inside the source window, each at least one search window having approximately the same size as the fill window;

comparing the intensity distribution of at least a portion of the fill window with the intensity distribution of the corresponding portion of each at least one search window;

selecting at least one search window from among the generated at least one search window based on the comparison of the intensity distribution of the fill window with the intensity distribution of each at least one search window; and

filling at least a portion of the pixels in the fill window with corresponding pixels in the selected at least one search window,

wherein the selected pixel is selected based on a fill order; and

wherein the fill order is determined by a time of arrival computation using the Eikonal equation initialized with the contour of the edges of the originally received at least one marked region and a prescribed speed function based on image intensity gradient.