Digital image manipulation

• US 9,396,569 B2
• Filed: 09/26/2013
• Issued: 07/19/2016
• Est. Priority Date: 02/15/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Mobile Imaging in Sweden AB
• Inventors: Lindskog, Alexander, Pettersson, Gustaf, Holmstedt, Ulf, Windmark, Johan, Niemi, Sami
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US14/037,708
• Publication Number:
  US 20140101590A1
• Time in Patent Office: 1,027 Days
• Field of Search: 382/100, 382/218, 382/254, 382/290, 345/619, 345/652, 345/654, 345/689, 348/222.1, 348/352
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

There is disclosed a method for seamlessly replacing areas in a digital image with corresponding data from temporally close digital images depicting substantially the same scene. The method uses localized image registration error minimization over a fixed preliminary boundary. A least cost closed path which constitutes a boundary for the area to be replaced is calculated using dynamic programming. The replacement area is blended such that image data information from one image is seamlessly replaced with image data information from another image.

Claims

1. A method comprising:

receiving a first image comprising a first non-stationary object in a first area obstructing a desired first view;

receiving a second image comprising a second non-stationary object in a second area obstructing a desired second view, wherein the first and second areas are different;

displaying the first image; and

with a processor, removing the first non-stationary object from the displayed first image by replacing, in the displayed first image, displayed image data information representing the first non-stationary object in the first area of the displayed first image with replacement image data representing the first area in the second image.

2. The method according to claim 1, further comprising:

processing the first image and the second image to determine the first image area and the second image area.

3. The method according to claim 1, further comprising:

enabling a user to control replacement of the first area.

4. The method according to claim 3, wherein the user control is via a touch-sensitive display.

5. The method according to claim 1, wherein the first area is identified to a user.

6. The method according to claim 1, wherein the first image and the second image have been captured in succession within a time interval with a small temporal difference between captures.

7. The method according to claim 1, wherein the first image is selected from a plurality of possible first images.

8. The method according to claim 7, wherein each one of said plurality of possible first images is associated with a quality measure, and wherein the source image is selected from the plurality of possible source image as the image having smallest quality measure.

9. The method according to claim 1, wherein the first image and the second image are captured in any order.

10. The method according to claim 1, wherein first image and the second image represent substantially the same scene.

11. The method according to claim 1, wherein areas of digital images are replaced seamlessly.

12. The method according to claim 1, further comprising:

identifying the first area in the first image, the identified area having a set of coordinates; and

using the set of coordinates for the identified area to, in response to said identification, identify the first area in the second image.

13. The method according to claim 12, wherein identifying said first area in the second image further comprises finding a compensation movement between at least a part of the first image and at least a part of the second, such that said compensation movement minimizes a difference between the source image and the target image.

14. The method according to claim 1, wherein the first image, the second image and the third image originate from a video sequence.

15. A non-transitory computer-readable storage medium encoded with instructions that, when performed by a processor, cause performance of the method of claim 1.

16. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least perform:

receiving a first image comprising a first non-stationary object in a first area obstructing a desired first view;

receiving a second image comprising a second non-stationary object in a second area obstructing a desired second view, wherein the first and second areas are different;

displaying the first image; and

removing the first non-stationary object from the displayed first image by replacing, in the displayed first image, displayed image data information representing the first non-stationary object in the first area of the displayed first image with replacement image data representing the first area in the second image.

17. The apparatus according to claim 16, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:

processing the first image and the second image to determine the first image area and the second image area.

18. The apparatus according to claim 16, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform:

enabling a user to control replacement of the first area.

19. The apparatus according to claim 18, wherein the apparatus further comprises a touch-sensitive display.

20. An apparatus according to claim 19 wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to perform identifying the first area to a user.

21. The apparatus according to claim 16, wherein the apparatus is configured as a mobile communications apparatus comprising a man-machine interface and a transceiver.