Method and apparatus for estimating image motion using disparity information of a multi-view image

• US 9,798,919 B2
• Filed: 05/09/2013
• Issued: 10/24/2017
• Est. Priority Date: 07/10/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Seung-ho, Park, Se-hyeok
• Primary Examiner(s): Tran, Trang U
• Application Number:
  US13/890,791
• Publication Number:
  US 20140015936A1
• Time in Patent Office: 1,629 Days
• Field of Search: 348 46, 348 47, 348 44, 348 51, 348 50
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method and apparatus for processing a multi-view image is provided. The method includes: extracting disparity information between an image of a first point of view and an image of a second point of view; and estimating a motion between two sequential images of the first point of view or the second point of view using the extracted disparity information. The apparatus may include a processor which is configured to extract disparity information between an image of a first point of view and an image of a second point of view in the multi-view image and is further configured to estimate a motion using the extracted disparity information between two sequential images of the first point of view or the second point of view.

Claims

1. A method of processing a multi-view image, the method comprising:

extracting disparity information between an image of a first point of view and an image of a second point of view;

estimating a motion between two sequential images of the first point of view and a motion between two sequential images of the second point of view; and

generating intermediate image data for each point of view using the estimated motion,

wherein the estimating comprises acquiring a motion vector between the two sequential images of the first point of view and a motion vector between the two sequential images of the second point of view, and

compensating the motion vector between the two sequential images of the first point of view using the extracted disparity information and compensating the motion vector between the two sequential images of the second point of view using the extracted disparity information, and

wherein the motion vector for each point of view is compensated so that the intermediate image data for each point of view have a disparity which corresponds to the extracted disparity information.

2. The method of claim 1, wherein the estimating comprises:

acquiring a corresponding area in images of the first point of view and the second point of view using the extracted disparity information;

estimating a motion between two sequential images of each of the first point of view and the second point of view based on the corresponding area; and

compensating for motion information estimated at each of the first point of view and the second point of view using motion information estimated at the other point of view.

3. The method of claim 1, wherein the estimating comprises:

estimating a motion between two sequential images of the first point of view; and

estimating a motion between two sequential images of the second point of view using the extracted disparity information and motion information estimated at the first point of view.

4. The method of claim 1, wherein the extracting comprises extracting at the same time disparity information between two images of the first point of view and the second point of view.

5. The method of claim 4, wherein a time where at least one of the two sequential images is equal to or adjacent to a time that the disparity information is extracted.

6. The method of claim 1, wherein the estimating comprises:

estimating a motion between two sequential images of each of the first point of view and the second point of view; and

compensating motion information estimated at the second point of view using the extracted disparity information or motion information estimated at the first point of view.

7. The method of claim 1, wherein the estimating comprises:

estimating a motion between two sequential images of each of the first point of view and the second point of view; and

compensating motion information estimated at the first point of view using the extracted disparity information or motion information estimated at the second point of view.

8. An apparatus for processing a multi-view image, the apparatus comprising:

a memory for storing a multi-view image; and

a processor which extracts disparity information between an image of a first point of view and an image of a second point of view in the multi-view image, estimates a motion between two sequential images of the first point of view and a motion between two sequential images of the second point of view, and generates intermediate image data for each point of view using the estimated motion,

wherein the estimating comprises acquiring a motion vector between the two sequential images of the first point of view and a motion vector between the two sequential images of the second point of view, and

compensating the motion vector between the two sequential images of the first point of view using the extracted disparity information and compensating the motion vector between the two sequential images of the second point of view using the extracted disparity information, and

wherein the motion vector for each point of view is compensated so that the intermediate image data for each point of view have a disparity which corresponds to the extracted disparity information.

9. The apparatus of claim 8, wherein the processor acquires a corresponding area in images of the first point of view and the second point of view using the extracted disparity information, estimates a motion between two sequential images of each of the first point of view and the second point of view based on the corresponding area, and compensates for motion information estimated at each of the first point of view and the second point of view using motion information estimated at the other point of view.

10. The apparatus of claim 8, wherein the processor estimates a motion between two sequential images of the first point of view or the second point of view and compensates estimated motion information between the two sequential images of the first point of view or the second point of view using the extracted disparity information.

11. The apparatus of claim 8, wherein the processor estimates a motion between two sequential images of the first point of view and estimates a motion between two sequential images of the second point of view using the extracted disparity information and the motion information estimated at the first point of view.

12. The apparatus of claim 8, wherein the processor extracts, at the same time, disparity information between two images of the first point of view and the second point of view.

13. The apparatus of claim 12, wherein a time of at least one of the two sequential images is equal to or adjacent to a time the disparity information is extracted.

14. The apparatus of claim 8, wherein the processor estimates a motion between two sequential images of each of the first point of view and the second point of view and compensates motion information estimated at the second point of view using the extracted disparity information or motion information estimated at the first point of view.

15. The apparatus of claim 8, wherein the processor estimates a motion between two sequential images of each of the first point of view and the second point of view and compensates motion information estimated at the first point of view using the extracted disparity information or motion information estimated at the second point of view.

16. A non-transitory computer-readable recording medium storing a computer-readable program which executes a method of processing a video signal, wherein the program, when executed by a processor of a computer causes the computer to execute a method comprising:

extracting disparity information between an image of a first point of view and an image of a second point of view;

estimating a motion between two sequential images of the first point of view and a motion between two sequential images of the second point of view; and

generating intermediate data for each point of view using the estimated motion,

wherein the estimating comprises acquiring a motion vector between the two sequential images of the first point of view and a motion vector between the two sequential images of the second point of view, and

compensating the motion vector between the two sequential images of the first point of view using the extracted disparity information and compensating the motion vector between the two sequential images of the second point of view using the extracted disparity information, and

wherein the motion vector for each anoint of view is compensated so that the intermediate image data for each point of view have a disparity which corresponds to the extracted disparity information.

17. An apparatus for processing a multi-view image, the apparatus comprising:

at least two cameras each photographing two objects which are spaced apart from each other and spaced apart from the cameras; and

a processor which is configured to extract disparity information between an image of a first point of view taken by a first camera and an image of a second point of view taken by a second camera in the multi-view image and is further configured to estimate a motion between two sequential images of the first point of view and a motion between two sequential images of the second point of view,

wherein the estimating comprises acquiring a motion vector between the two sequential images of the first point of view and a motion vector between the two sequential images of the second point of view, and

compensating the motion vector between the two sequential images of the first point of view using the extracted disparity information and compensating the motion vector between the two sequential images of the second point of view using the extracted disparity information.

18. An apparatus for processing a multi-view image, the apparatus comprising:

a first motion estimation module which estimates a motion between two sequential images of a first point of view;

a second motion estimation module which estimates a motion between two sequential images of a second point of view;

a disparity extraction module which extracts disparity information between an image of the first point of view and an image of the second point of view; and

a motion estimation compensation module which compensates the estimated motion between the two sequential images of the first point of view and the estimated motion between the two sequential images of the second point of view,

wherein the first motion estimation module acquires a motion vector between the two sequential images of the first point of view and the second motion estimation module acquires a motion vector between the two sequential images of the second point of view, and

wherein the motion estimation compensation module compensates the motion vector between the two sequential images of the first point of view using the extracted disparity information and compensates the motion vector between the two sequential images of the second point of view using the extracted disparity information.