Semantics-based motion estimation for multi-view video coding

• US 20050031035A1
• Filed: 03/31/2004
• Published: 02/10/2005
• Est. Priority Date: 08/07/2003
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Puri, Rohit, Tabatabai, Ali J., Vedula, Sundar
• Granted Patent: US 7,778,328 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A motion estimation method and apparatus for video coding of a multi-view sequence is described. In one embodiment, a motion estimation method includes identifying one or more pixels in a first frame of a multi-view video sequence, and constraining a search range associated with a second frame of the multi-view video sequence based on an indication of a desired correlation between efficient coding and semantic accuracy. The semantic accuracy relies on use of geometric configurations of cameras capturing the multi-view video sequence. The method further includes searching the second frame within the constrained search range for a match of the pixels identified in the first frame.

Claims

1. A motion estimation method comprising:

identifying one or more pixels in a first frame of a multi-view video sequence;

constraining a search range associated with a second frame of the multi-view video sequence based upon an indication of a desired correlation between efficient coding and semantic accuracy, the semantic accuracy relying on use of geometric configurations of cameras capturing the multi-view video sequence; and

searching the second frame within the constrained search range for a match of the one or more pixels identified in the first frame.

2. The method of claim 1 wherein the search range is constrained with respect to a position in the second frame of a epipolar line corresponding to the one or more pixels in the first frame, the position of the corresponding epipolar line depending on the geometric configurations of the cameras.

3. The method of claim 1 wherein the one or more pixels in the first frame represent a block.

4. The method of claim 2 further comprising:

computing the epipolar line in the second frame.

5. The method of claim 4 wherein the epipolar line is computed using a fundamental matrix.

6. The method of claim 2 wherein constraining the search range comprises:

finding a position of an initial seed on the epipolar line; and

determining parameters of a window covering the initial seed and the epipolar line based on the desired correlation between efficient coding and semantic accuracy.

7. The method of claim 6 wherein the position of the initial seed is found using a disparity vector.

8. The method of claim 1 further comprising:

receiving the indication of the desired correlation between efficient coding and semantic accuracy from a user.

9. The method of claim 8 further comprising:

communicating to a user a user interface facilitating user input of the desired correlation between efficient coding and semantic accuracy.

10. The method of claim 9 wherein the user interface provides a slider to enable the user to specify the desired correlation between efficient coding and semantic accuracy.

11. The method of claim 9 wherein the user interface allows the user to modify a previously specified correlation between efficient coding and semantic accuracy at any time.

12. A computer readable medium that provides instructions, which when executed on a processor cause the processor to perform a method comprising:

identifying one or more pixels in a first frame of a multi-view video sequence;

constraining a search range associated with a second frame of the multi-view video sequence based upon an indication of a desired correlation between efficient coding and semantic accuracy, the semantic accuracy relying on use of geometric configurations of cameras capturing the multi-view video sequence; and

searching the second frame within the constrained search range for a match of the one or more pixels identified in the first frame.

13. The computer readable medium of claim 12 wherein the search range is constrained with respect to a position in the second frame of a epipolar line corresponding to the one or more pixels in the first frame, the position of the corresponding epipolar line depending on the geometric configurations of the cameras.

14. The computer readable medium of claim 12 wherein the one or more pixels in the first frame represent a block.

15. The computer readable medium of claim 13 wherein the method further comprises:

computing the epipolar line in the second frame.

16. The computer readable medium of claim 15 wherein the epipolar line is computed using a fundamental matrix.

17. The computer readable medium of claim 13 wherein constraining the search range comprises:

finding a position of an initial seed on the epipolar line; and

determining parameters of a window covering the initial seed and the epipolar line based on the desired correlation between efficient coding and semantic accuracy.

18. The computer readable medium of claim 17 wherein the position of the initial seed is found using a disparity vector.

19. The computer readable medium of claim 12 wherein the method further comprises:

communicating to a user a user interface facilitating user input of the desired correlation between efficient coding and semantic accuracy.

20. A computerized system comprising:

a memory; and

at least one processor coupled to the memory, the at least one processor executing a set of instructions which cause the at least one processor to

identify one or more pixels in a first frame of a multi-view video sequence,

constrain a search range associated with a second frame of the multi-view video sequence based upon an indication of a desired correlation between efficient coding and semantic accuracy, the semantic accuracy relying on use of geometric configurations of cameras capturing the multi-view video sequence, and

search the second frame within the constrained search range for a match of the one or more pixels identified in the first frame.

21. The system of claim 20 wherein the search range is constrained with respect to a position in the second frame of a epipolar line corresponding to the one or more pixels in the first frame, the position of the corresponding epipolar line depending on the geometric configurations of the cameras.

22. The system of claim 20 wherein the one or more pixels in the first frame represent a block.

23. The system of claim 21 wherein the processor is to constrain the search range by finding a position of an initial seed on the epipolar line, and determining parameters of a window covering the initial seed and the epipolar line based on the desired correlation between efficient coding and semantic accuracy.

24. The system of claim 23 wherein the processor is to find the position of the initial seed using a disparity vector.

25. The system of claim 20 wherein the processor is further to communicate to a user a user interface facilitating user input of the desired correlation between efficient coding and semantic accuracy.

26. A motion estimation apparatus comprising:

a block identifier to identify one or more pixels in a first frame of a multi-view video sequence;

a search range determinator to constrain a search range associated with a second frame of the multi-view video sequence based upon an indication of a desired correlation between efficient coding and semantic accuracy, the semantic accuracy relying on use of geometric configurations of cameras capturing the multi-view video sequence; and

a searcher to search the second image within the constrained search range for a match of the one or more pixels identified in the first frame.

27. The apparatus of claim 26 wherein the search range is constrained with respect to a position in the second frame of a epipolar line that corresponds to the one or more pixels in the first frame, the position of the corresponding epipolar line depending on the geometric configurations of the cameras.

28. The apparatus of claim 26 wherein the one or more pixels in the first frame represent a block.

29. The apparatus of claim 27 wherein the search range determinator is further to compute the epipolar line in the second frame.

30. The apparatus of claim 27 wherein the search range determinator is to constrain the search range by finding a position of an initial seed on the epipolar line, and determining parameters of a window covering the initial seed and the epipolar line based on the desired correlation between efficient coding and semantic accuracy.

31. The apparatus of claim 26 wherein the search range determinator is further to communicate to a user a user interface facilitating user input of the desired correlation between efficient coding and semantic accuracy.