Motion estimation for compressing multiple view images

• US 7,463,778 B2
• Filed: 01/30/2004
• Issued: 12/09/2008
• Est. Priority Date: 01/30/2004
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Damera-Venkata, Niranjan
• Primary Examiner(s): Ahmed; Samir A.
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US10/768,239
• Publication Number:
  US 20050169543A1
• Time in Patent Office: 1,775 Days
• Field of Search: 382/100, 382/106, 382/236, 382/238, 382/243, 382/154, 382/284, 382/294, 382/107, 382/103, 382/232, 382/250, 382/251, 348/42, 348/402, 348/407, 348/408, 348/400, 348/412, 348/400.1, 348/402.1, 348/403.1, 348/407.1, 348/408.1, 348/412.1, 348/669, 345/419, 345/427, 345/473, 386/68, 375/240.16
• US Class Current: 382/236
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Systems and methods of estimating motion for compressing multiple view images are described. In accordance with a machine-implemented method of encoding a target image of a scene captured at a first image plane, a transformation is computed. The transformation maps at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a references image of the scene captured at a second image plane different from the first image plane. At least one point in the target image off the scene plane and at least one corresponding point in the reference image are identified. A motion between the target image and the reference image is estimated based on the computed transformation and the identified corresponding off-scene-plane points. The target image is encoded based at least in part on the estimated motion.

Claims

1. A machine-implemented method of encoding a target image of a scene captured at a first image plane, comprising:

computing a transformation (H) mapping at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a reference image of the scene captured at a second image plane different from the first image plane;

ascertaining an epipole (e_REF) in the reference image from at least one point in the target image off the scene plane and at least one corresponding point in the reference image;

determining respective values of a scalar parameter Θ

that map blocks of points in the target image to respective matching blocks of points in the reference image in accordance with a motion model corresponding to {right arrow over (b)}_REF=(H·

{right arrow over (b)}_TARGET)+Θ

·

{right arrow over (e)}_REF, wherein {right arrow over (b)}_TARGET is a vector that represents a point within a respective one of the blocks of the target image and {right arrow over (b)}_REF is a vector that represents a point within a respective one of the blocks of the reference image; and

compressing the target image, wherein the compressing comprises encoding ones of the blocks of the target image in terms of the reference image and the respective vectors {right arrow over (b)}_REF that are specified by respective ones of the determined values of the scalar parameter Θ

.

2. The method of claim 1, further comprising identifying the at least three scene plane points in the target image and the at least three corresponding scene plane points in the reference image.

3. The method of claim 1, wherein the motion model defines single-parameter search spaces each relating points in the reference image to respective points in the target image.

4. The method of claim 3, wherein the ascertaining comprises computing the epipole in the reference image based on the computed transformation and the identified corresponding off-scene-plane points.

5. The method of claim 4, wherein the motion model parameterizes an epipolar line extending through the computed epipole in the reference image.

6. The method of claim 5, wherein the motion model defines a respective single-parameter search space for each of the blocks of points in the target image.

7. The method of claim 6, wherein the epipolar line in the reference image extends through the epipole and a point corresponding to a mapping of a given point in the target image to a corresponding point in the reference image based on the computed transformation.

8. The method of claim 3, wherein the determining comprises dividing the target image into blocks of points and computing for each of the divided blocks a respective one of the vectors {right arrow over (b)}_REF representing motion between a the divided block and a matching one of the reference image blocks.

9. The method of claim 8, wherein each of the vectors {right arrow over (b)}_REF describes a one-to-one mapping between a respective one of the blocks of points in the target image and a matching one of the blocks of points in the reference image.

10. The method of claim 1, wherein the compressing comprises representing points of the target image in terms of the vectors {right arrow over (b)}_REF and motion compensation difference data representing intensity adjustments to points of the reference image for reconstructing corresponding points of the target image.

11. An apparatus for encoding a target image of a scene captured at a first image plane, comprising an encoder operable to perform operations comprising:

computing a transformation (H) mapping at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a reference image of the scene captured at a second image plane different from the first image plane;

ascertaining an epipole (e_REF) in the reference image from at least one point in the target image off the scene plane and at least one corresponding point in the reference image;

determining respective values of a scalar parameter Θ

that map blocks of points in the target image to respective matching blocks of points in the reference image in accordance with a motion model corresponding to {right arrow over (b)}_REF =(H·

{right arrow over (b)}_TARGET)+Θ

·

{right arrow over (e)}_REF, wherein {right arrow over (b)}_TARGET is a vector that represents a point within a respective one of the blocks of the target image and {right arrow over (b)}_REF is a vector that represents a point within a respective one of the blocks of the reference image and;

compressing the target image, wherein the compressing comprises encoding ones of the blocks of the target image in terms of the reference image and the respective vectors {right arrow over (b)}_REF that are specified by respective ones of the determined values of the scalar parameter Θ

.

12. The apparatus of claim 11, wherein the encoder is further operable to identify the at least three scene plane points in the target image and the at least three corresponding scene plane points in the reference image.

13. The apparatus of claim 11, wherein the motion model defines single-parameter search spaces each relating points in the reference image to respective points in the target image.

14. The apparatus of claim 13, wherein in the ascertaining the encoder is operable to compute the epipole in the reference image based on the computed transformation and the identified corresponding off-scene-plane points.

15. The apparatus of claim 14, wherein the motion model parameterizes an epipolar line extending through the computed epipole in the reference image.

16. The apparatus of claim 15, wherein the motion model defines a respective single-parameter search space for each of the blocks of points in the target image.

17. The apparatus of claim 16, wherein the epipolar line in the reference image extends through the epipole and a point corresponding to a mapping of a given point in the target image to a corresponding point in the reference image based on the computed transformation.

18. The apparatus of claim 13, wherein in the determining the encoder is operable to perform operations comprising dividing the target image into blocks of points and computing for each of the divided blocks a respective one of the vectors {right arrow over (b)}_REF representing motion between the divided block and a matching one of the reference image blocks.

19. The apparatus of claim 18, wherein each of the vectors {right arrow over (b)}_REF describes a one-to-one mapping between a respective one of the blocks of points in the target image and a matching one of the blocks of points in the reference image.

20. The apparatus of claim 11, wherein in the compressing the encoder is operable to perform operations comprising representing points of the target image in terms of the vectors {right arrow over (b)}_REF and motion compensation difference data representing intensity adjustments to points of the reference image for reconstructing corresponding points of the target image.

21. A computer-readable medium storing machine-readable instructions for causing a machine to perform operations comprising:

computing a transformation (H) mapping at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a reference image of the scene captured at a second image plane different from the first image plane;

ascertaining an epipole (e_REF) in the reference image from at least one point in the target image off the scene plane and at least one corresponding point in the reference image;

determining respective values of a scalar parameter Θ

that map blocks of points in the target image to respective matching blocks of points in the reference image in accordance with a motion model corresponding to {right arrow over (b)}_REF =(H·

{right arrow over (b)}_TARGET) +Θ

·

{right arrow over (e)}_REF, wherein {right arrow over (b)}_TARGET is a vector that represents a point within a respective one of the blocks of the target image and {right arrow over (b)}_REF is a vector that represents a point within a respective one of the blocks of the reference image; and

compressing the target image, wherein the compressing comprises encoding ones of the blocks of the target image in terms of the reference image and the respective vectors {right arrow over (b)}_REF that are specified by respective ones of the determined values of the scalar parameter Θ

.

22. The computer-readable medium of claim 21, wherein the machine-readable instructions further cause the machine to identify the at least three scene plane points in the target image and the at least three corresponding scene plane points in the reference image.

23. The computer-readable medium of claim 21, wherein the motion model defines single-parameter search spaces each relating points in the reference image to respective points in the target image.

24. The computer-readable medium of claim 23, wherein in the ascertaining the machine-readable instructions cause the machine to perform operations comprising computing the epipole in the reference image based on the computed transformation and the identified corresponding off-scene-plane points.

25. The computer-readable medium of claim 24, wherein the motion model parameterizes an epipolar line extending through the computed epipole in the reference image.

26. The computer-readable medium of claim 25, wherein the motion model defines a respective single-parameter search space for each block of points in the target image.

27. The machine-readable medium of claim 26, wherein the epipolar line in the reference image extends through the epipole and a point corresponding to a mapping of a given point in the target image to a corresponding point in the reference image based on the computed transformation.

28. The computer-readable medium of claim 23, wherein in the determining the machine-readable instructions cause the machine to perform operations comprising dividing the target image into blocks of points and computing for each of the divided blocks a respective one of the vectors {right arrow over (b)}_REF representing motion between the divided block and a matching one of the reference image blocks.

29. The computer readable medium of claim 28, wherein each of the motion vectors {right arrow over (b)}_REF describes a one-to-one mapping between a respective one of the blocks of points in the target image and a matching one of the blocks of points in the reference image.

30. The computer-readable medium of claim 21, wherein in the compressing the machine-readable instructions cause the machine to perform operations comprising representing points of the target image in terms of the vectors {right arrow over (b)}_REF and motion compensation difference data representing intensity adjustments to points of the reference image for reconstructing corresponding points of the target image.