Generating replacement data values for an image region

• US 20030210828A1
• Filed: 06/17/2003
• Published: 11/13/2003
• Est. Priority Date: 11/16/2001
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Hamburg, Mark, Chien, Jen-Chan, Georgiev, Todor G.
• Granted Patent: US 6,985,263 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 10/30 Noise filtering

Abstract

Given an image with an unwanted feature, a collection of pixels (a defined region) entirely containing the unwanted feature is designated, and replacement data values for the defined region are generated, thereby healing the unwanted feature. A selected image region is healed by propagating values in a boundary region into replacement data values for the selected image region by iteratively applying a procedure or algorithm (e.g., iteratively applying one or more kernels), such that values in the boundary region are continuously put back into the iterative calculations. An example of such a function uses kernels to generate replacement data values having multiple orders of continuity at the boundary. Optionally, replacement data values for an image region to be healed are generated based on a difference between existing data values and texture data values, thereby introducing texture (e.g., pattern, noise) components to a resulting healed region in the image. Mutiresolution processing and tiling can also be used to enhance performance.

Claims

1. A machine-implemented method for generating new data for a modification region of an image, the method comprising:

determining boundary pixels bounding a modification region in an image, the boundary pixels having values and being outside the modification region; and

generating new pixel values for the modification region using the boundary pixel values such that a rate of a rate of change in pixel values is minimized from the boundary pixel values to new modification region pixel values.

2. The method of claim 1, wherein generating new pixel values comprises generating new pixel values for multiple channels to create similar overall gradations of color and illumination from the boundary pixel values to new modification region pixel values.

3. The method of claim 1, wherein the image has an original resolution, the method further comprising:

resizing the original-resolution image to a lower-resolution image;

generating lower-resolution new pixel values for a lower-resolution modification region using lower-resolution boundary pixel values such that a rate of a rate of change in pixel values is minimized from the lower-resolution boundary pixel values to lower-resolution new modification region pixel values, the lower-resolution modification region corresponding to the modification region of the original-resolution image, and the lower-resolution boundary pixels corresponding to the boundary pixels of the original-resolution image; and

resampling the lower-resolution new pixel values to fill the modification region of the original-resolution image before generating the new pixel values for the modification region of the original-resolution image.

4. The method of claim 1, wherein generating new pixel values comprises iteratively applying multiple kernels.

5. The method of claim 4, wherein each of the multiple kernels comprises a discretized Laplace operator.

6. The method of claim 4, wherein determining boundary pixels in an image comprises determining the boundary pixels based on the modification region and a kernel to be applied.

7. The method of claim 1, wherein if the image has been saved using a lossy compression technique, the modification region comprises one or more compression artifact regions between compression blocks.

8. The method of claim 7, wherein the lossy compression technique comprises a Joint Photographic Experts Group compression technique.

9. The method of claim 1, wherein the image comprises an image copy of an original image, the method further comprising writing the new pixel values into the original image.

10. A machine-readable medium embodying information indicative of instructions for causing one or more machines to perform operations comprising:

determining boundary pixels bounding a modification region in an image, the boundary pixels having values and being outside the modification region; and

generating new pixel values for the modification region using the boundary pixel values such that a rate of a rate of change in pixel values is minimized from the boundary pixel values to new modification region pixel values.

11. The machine-readable medium of claim 10, wherein generating new pixel values comprises generating new pixel values for multiple channels to create similar overall gradations of color and illumination from the boundary pixel values to new modification region pixel values.

12. The machine-readable medium of claim 10, wherein the image has an original resolution, and wherein the operations further comprise:

resizing the original-resolution image into a lower-resolution image;

generating lower-resolution new pixel values for a lower-resolution modification region using lower-resolution boundary pixel values such that a rate of a rate of change in pixel values is minimized from the lower-resolution boundary pixel values to lower-resolution new modification region pixel values, the lower-resolution modification region corresponding to the modification region of the original-resolution image, and the lower-resolution boundary pixels corresponding to the boundary pixels of the original-resolution image; and

resampling the lower-resolution new pixel values to fill the modification region of the original-resolution image before generating the new pixel values for the modification region of the original-resolution image.

13. The machine-readable medium of claim 10, wherein generating new pixel values comprises iteratively applying multiple kernels.

14. The machine-readable medium of claim 10, wherein if the image has been saved using a lossy compression technique, the modification region comprises one or more compression artifact regions between compression blocks.

15. The machine-readable medium of claim 14, wherein the lossy compression technique comprises a Joint Photographic Experts Group compression technique.

16. The machine-readable medium of claim 10, wherein the image comprises an image copy of an original image, and wherein the operations further comprise writing the new pixel values into the original image.

17. A machine-implemented method comprising:

subtracting texture pixel values from pixel values corresponding to at least a portion of an image;

generating new pixel values for a modification region of the image using the texture-subtracted pixel values;

adding texture pixel values to the new pixel values; and

writing the texture-added new pixel values to the image.

18. The method of claim 17, wherein texture pixel values comprise high frequency data.

19. The method of claim 17, wherein texture pixel values comprise a pattern.

20. The method of claim 17, wherein generating new pixel values comprises iteratively applying a kernel.

21. The method of claim 20, wherein generating new pixel values further comprises iteratively applying multiple kernels to create a continuous rate of change in pixel values from boundary pixel values to new modification region pixel values, the boundary pixel comprising pixels in the image portion that bound the modification region.

22. The method of claim 21, wherein generating new pixel values further comprises iteratively applying the multiple kernels at multiple resolutions of the image, starting with a lowest resolution version of the image.

23. The method of claim 22, wherein iteratively applying the multiple kernels at multiple resolutions of the image further comprises applying each kernel separately to the multiple resolutions of the image, starting with the smallest kernel.

24. The method of claim 23, further comprising breaking each resolution version of the image into one or more tiles before applying the multiple kernels, wherein the tiles overlap to provide padding when there are two or more tiles, the padding being large enough to accommodate a largest kernel from the multiple kernels.

25. The method of claim 24, further comprising:

converting to a higher precision value representation before applying the multiple kernels; and

blurring each resolution version of the image before applying the multiple kernels.

26. The method of claim 25, wherein generating new pixel values further comprises generating new pixel values for multiple channels to create similar overall gradations of color and illumination from the boundary pixel values to new modification region pixel values.

27. A machine-readable medium embodying information indicative of instructions for causing one or more machines to perform operations comprising:

subtracting texture pixel values from pixel values corresponding to a portion of an image;

generating new pixel values for a modification region of the image using the texture-subtracted pixel values;

adding texture pixel values to the new pixel values; and

writing the texture-added new pixel values to the image.

28. The machine-readable medium of claim 27, wherein texture pixel values comprise high frequency data.

29. The machine-readable medium of claim 27, wherein texture pixel values comprises a pattern.

30. The machine-readable medium of claim 27, wherein the generating operation comprises iteratively applying a kernel.

31. The machine-readable medium of claim 30, wherein the generating operation further comprises iteratively applying multiple kernels to create a continuous rate of change in pixel values from boundary pixel values to new modification region pixel values, the boundary pixel values comprising pixels in the image portion that bound the modification region.

32. The machine-readable medium of claim 31, wherein the generating operation further comprises iteratively applying the multiple kernels at multiple resolutions of the image, starting with a lowest resolution version of the image.

33. The machine-readable medium of claim 32, wherein iteratively applying the multiple kernels at multiple resolutions of the image further comprises applying each kernel separately to the multiple resolutions of the image, starting with the smallest kernel.

34. The machine-readable medium of claim 33, wherein the operations further comprise breaking each resolution version of the image into one or more tiles before applying the multiple kernels, wherein the tiles overlap to provide padding when there are two or more tiles, the padding being large enough to accommodate a largest kernel from the multiple kernels.

35. The machine-readable medium of claim 34, wherein the operations further comprise:

converting to a higher precision value representation before applying the multiple kernels; and

blurring each resolution version of the image before applying the multiple kernels.

36. The machine-readable medium of claim 35, wherein the generating operation further comprises generating new pixel values for multiple channels to create similar overall gradations of color and illumination from the boundary pixel values to new modification region pixel values.