METHOD AND APPARATUS FOR PROCESSING IMAGE BASED ON MOTION OF OBJECT

• US 20150294178A1
• Filed: 04/09/2015
• Published: 10/15/2015
• Est. Priority Date: 04/14/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: JUNG, Seung-Won, CHOI, Ouk
• Granted Patent: US 9,582,856 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

An apparatus and method for processing an image based on a motion of an object, the apparatus including a motion estimator configured to estimate a motion of an object included in a current image, an image determiner configured to determine a neighboring image neighboring the current image based on the motion of the object, and a pixel value determiner configured to determine a pixel value of a hole region neighboring the object based on the neighboring image is provided.

Claims

1. An image processing apparatus comprising:

a motion estimator configured to estimate a motion of an object included in a current image;

an image determiner configured to determine a neighboring image neighboring the current image based on the motion of the object; and

a pixel value determiner configured to determine a pixel value of a hole region neighboring the object based on the neighboring image.

2. The apparatus of claim 1, wherein the image determiner is configured to determine an image preceding the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-in motion.

3. The apparatus of claim 1, wherein the image determiner is configured to determine an image subsequent to the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-out motion.

4. The apparatus of claim 1, wherein the image determiner is configured to determine one of an image preceding the current image and an image subsequent to the current image to be the neighboring image based on a position of the hole region and a moving direction of the object.

5. The apparatus of claim 4, wherein the image determiner is configured to determine the image preceding the current image to be the neighboring image when the moving direction of the object is identical to a direction in which the hole region is located.

6. The apparatus of claim 4, wherein the image determiner is configured to determine an image subsequent to the current image to be the neighboring image if the moving direction of the object is opposite to a direction in which the hole region is located.

7. The apparatus of claim 1, further comprising:

a hole region divider configured to divide the hole region into a first region and a second region.

8. The apparatus of claim 7, wherein the hole region divider is configured to divide the hole region into the first region and the second region based on a size of the hole region and a motion vector of a neighboring region of the hole region.

9. The apparatus of claim 7, wherein the pixel value determiner is configured to determine a pixel value of the second region based on a pixel value of the first region and a pixel value of a neighboring region of the hole region.

10. The apparatus of claim 1, wherein if the hole region includes a pixel for which a pixel value is not determined based on the neighboring image, the image determiner searches images preceding or subsequent to the current image for an image in which all pixel values of the hole region are determined, and the image determiner is configured to determine the found image to be the neighboring image.

11. The apparatus of claim 1, wherein the pixel value determiner is configured to search a neighboring region for a pixel corresponding to the hole region based on a motion vector of the hole region and determine the pixel value of the hole region based on a pixel value of the found pixel.

12. The apparatus of claim 11, wherein the pixel value determiner is configured to predict the motion vector of the hole region, and set a search range for a pixel in the neighboring region based on an accuracy of the predicted motion vector.

13. The apparatus of claim 1, wherein if the current image corresponds to at least one depth image based rendering (DIBR)-applied multiview image, the image determiner determines one of a DIBR-applied multiview image of a preceding image and a DIBR-applied multiview image of a subsequent image, to be the neighboring image.

14. The apparatus of claim 1, wherein if the current image corresponds to at least one DIBR-applied multiview image, the pixel value determiner applies DIBR to the neighboring image to generate a multiview image of the neighboring image and determines the pixel value of the hole region.

15. An image processing method comprising:

estimating a motion of an object included in a current image;

determining a neighboring image neighboring the current image based on the estimated motion of the object; and

determining a pixel value of a hole region neighboring the object based on the neighboring image.

16. The method of claim 15, wherein the determining the neighboring image comprises determining an image preceding the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-in motion.

17. The method of claim 15, wherein the determining the neighboring image comprises determining an image subsequent to the current image to be the neighboring image if the estimated motion of the object corresponds to a zoom-out motion.

18. The method of claim 15, wherein the determining the neighboring image comprises determining one of an image preceding the current image and an image subsequent to the current image to be the neighboring image based on a position of the hole region and a moving direction of the object.

19. The method of claim 15, wherein the determining the pixel value comprises:

searching the neighboring image for a pixel corresponding to the hole region based on a motion vector of the hole region; and

determining the pixel value of the hole region based on the searching.

20. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the method of claim 15.