Method and apparatus for processing image based on motion of object

• US 9,582,856 B2
• Filed: 04/09/2015
• Issued: 02/28/2017
• Est. Priority Date: 04/14/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jung, Seung-Won, Choi, Ouk
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/682,752
• Publication Number:
  US 20150294178A1
• Time in Patent Office: 691 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

An apparatus and method for processing an image based on a motion of an object, the apparatus including a motion estimator configured to estimate a motion of an object included in a current image, an image determiner configured to determine a neighboring image neighboring the current image based on the motion of the object, and a pixel value determiner configured to determine a pixel value of a hole region neighboring the object based on the neighboring image is provided.

Claims

1. An image processing apparatus comprising:

a processor configured to execute computer-readable instructions to,

estimate a motion of an object included in a current image;

determine a neighboring image neighboring the current image based on the motion of the object; and

determine a pixel value of a hole region neighboring the object based on the neighboring image,

wherein if the hole region includes a pixel for which a pixel value is not determined based on the neighboring image, the processor searches images preceding or subsequent to the current image for an image in which all pixel values of the hole region are determined, and the processor is configured to execute the computer-readable instructions to determine the image in which all pixel values of the hole region are determined to be the neighboring image.

2. The apparatus of claim 1, wherein the processor is configured to execute the computer-readable instructions to determine one of the images preceding the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-in motion.

3. The apparatus of claim 1, wherein the processor is configured to execute the computer-readable instructions to determine one of the images subsequent to the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-out motion.

4. The apparatus of claim 1, wherein the processor is configured to execute the computer-readable instructions to determine one of the images preceding and subsequent to the current image to be the neighboring image based on a position of the hole region and a moving direction of the object.

5. The apparatus of claim 4, wherein the processor is configured to execute the computer-readable instructions to determine one of the images preceding the current image to be the neighboring image when the moving direction of the object is identical to a direction in which the hole region is located.

6. The apparatus of claim 4, wherein the processor is configured to execute the computer-readable instructions to determine one of the images subsequent to the current image to be the neighboring image if the moving direction of the object is opposite to a direction in which the hole region is located.

7. The apparatus of claim 1, wherein the processor is configured to execute the computer-readable instructions to divide the hole region into a first region and a second region.

8. The apparatus of claim 7, wherein the processor is configured to execute the computer-readable instructions to divide the hole region into the first region and the second region based on a size of the hole region and a motion vector of a neighboring region of the hole region.

9. The apparatus of claim 7, wherein the processor is configured to execute the computer-readable instructions to determine a pixel value of the second region based on a pixel value of the first region and a pixel value of a neighboring region of the hole region.

10. The apparatus of claim 1, wherein the processor is configured to execute the computer-readable instructions to search a neighboring region for a pixel corresponding to the hole region based on a motion vector of the hole region and determine the pixel value of the hole region based on a pixel value of the pixel corresponding to the hole region.

11. The apparatus of claim 10, wherein the processor is configured to execute the computer-readable instructions to predict the motion vector of the hole region, and set a search range for a pixel in the neighboring region based on an accuracy of the predicted motion vector.

12. The apparatus of claim 1, wherein the current image corresponds to at least one depth image based rendering (DIBR) multiview image, and the processor is configured to execute the computer-readable instructions to use a DIBR multiview image of one of the preceding images as the first neighboring image for the first view image and a DIBR-applied multiview image of one of the subsequent images as a second neighboring image for a second view image of the current image.

13. The apparatus of claim 1, wherein the current image corresponds to at least one depth image based rendering (DIBR)-multiview image, the processor is configured to execute the computer-readable instructions to apply DIBR to the neighboring image to generate a multiview image of the neighboring image and determine the pixel value of the hole region.

14. An image processing method comprising:

estimating a motion of an object included in a current image;

determining a neighboring image neighboring the current image based on the estimated motion of the object;

determining a pixel value of a hole region neighboring the object based on the neighboring image;

searching images preceding or subsequent to the current image for an image in which all pixel values of the hole region are determined, when the hole region includes a pixel for which a pixel values is not determined based on the neighboring image; and

determining the image in which all pixel values of the hole region are determined to be the neighboring image.

15. The method of claim 14, wherein the determining the neighboring image comprises determining one of the images preceding the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-in motion.

16. The method of claim 14, wherein the determining the neighboring image comprises determining one of the images subsequent to the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-out motion.

17. The method of claim 14, wherein the determining the neighboring image comprises determining one of the images preceding and subsequent to the current image to be the first neighboring image based on a position of the hole region and a moving direction of the object.

18. The method of claim 14, wherein the determining the pixel value comprises:

searching the neighboring image for a pixel corresponding to the hole region based on a motion vector of the hole region; and

determining the pixel value of the hole region based on the searching.

19. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the method of claim 14.