Motion estimation for compressing multiple view images

• US 20050169543A1
• Filed: 01/30/2004
• Published: 08/04/2005
• Est. Priority Date: 01/30/2004
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Damera-Venkata, Niranjan
• Granted Patent: US 7,463,778 B2
• Time in Patent Office: Days
• US Class Current: 382/236
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Systems and methods of estimating motion for compressing multiple view images are described. In accordance with a machine-implemented method of encoding a target image of a scene captured at a first image plane, a transformation is computed. The transformation maps at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a references image of the scene captured at a second image plane different from the first image plane. At least one point in the target image off the scene plane and at least one corresponding point in the reference image are identified. A motion between the target image and the reference image is estimated based on the computed transformation and the identified corresponding off-scene-plane points. The target image is encoded based at least in part on the estimated motion.

Claims

1. A machine-implemented method of encoding a target image of a scene captured at a first image plane, comprising:

computing a transformation mapping at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a references image of the scene captured at a second image plane different from the first image plane;

identifying at least one point in the target image off the scene plane and at least one corresponding point in the reference image;

estimating a motion between the target image and the reference image based on the computed transformation and the identified corresponding off-scene-plane points; and

encoding the target image based at least in part on the estimated motion.

2. The method of claim 1, further comprising identifying the at least three scene plane points in the target image and the at least three corresponding scene plane points in the reference image.

3. The method of claim 1, wherein estimating the motion comprises defining single-parameter search spaces each relating points in the reference image to respective points in the target image.

4. The method of claim 3, wherein defining the single-parameter search space comprises computing an epipole in the reference image based on the computed transformation and the identified corresponding off-scene-plane points.

5. The method of claim 4, wherein defining a respective single-parameter search space comprises parameterizing an epipolar line extending through the computed epipole in the reference image.

6. The method of claim 5, wherein a respective single-parameter search space is defined for each block of points in the target image.

7. The method of claim 6, wherein a respective single-parameter search space is defined by a parameterized epipolar line in the reference image extending through the epipole and a point corresponding to a mapping of a given point in the target image to a corresponding point in the reference image based on the computed transformation.

8. The method of claim 3, wherein estimating the motion comprises dividing the target image into blocks of points and computing for each block a respective motion vector representing motion between a target image block and a reference image block.

9. The method of claim 8, wherein each motion vector describes a one-to-one mapping between a block of points in the target image and a block of points in the reference image.

10. The method of claim 1, wherein encoding the target image comprises representing points of the target image in terms of the estimated motion and motion compensation difference data representing intensity adjustments to points of the reference image for reconstructing corresponding points of the target image.

11. An apparatus for encoding a target image of a scene captured at a first image plane, comprising an encoder operable to:

compute a transformation mapping at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a references image of the scene captured at a second image plane different from the first image plane;

identify at least one point in the target image off the scene plane and at least one corresponding point in the reference image;

estimate a motion between the target image and the reference image based on the computed transformation and the identified corresponding off-scene-plane points; and

encode the target image based at least in part on the estimated motion.

12. The apparatus of claim 11, wherein the encoder is further operable to identify the at least three scene plane points in the target image and the at least three corresponding scene plane points in the reference image.

13. The apparatus of claim 11, wherein the encoder is operable to estimate the motion by defining single-parameter search spaces each relating points in the reference image to respective points in the target image.

14. The apparatus of claim 13, wherein the encoder is operable to define the single-parameter search space by computing an epipole in the reference image based on the computed transformation and the identified corresponding off-scene-plane points.

15. The apparatus of claim 14, wherein the encoder is operable to define a respective single-parameter search space by parameterizing an epipolar line extending through the computed epipole in the reference image.

16. The apparatus of claim 15, wherein the encoder defines a respective single-parameter search space for each block of points in the target image.

17. The apparatus of claim 16, wherein a respective single-parameter search space is defined by a parameterized epipolar line in the reference image extending through the epipole and a point corresponding to a mapping of a given point in the target image to a corresponding point in the reference image based on the computed transformation.

18. The apparatus of claim 13, wherein the encoder is operable to estimate the motion by dividing the target image into blocks of points and computing for each block a respective motion vector representing motion between a target image block and a reference image block.

19. The apparatus of claim 18, wherein each motion vector describes a one-to-one mapping between a block of points in the target image and a block of points in the reference image.

20. The apparatus of claim 11, wherein the encoder is operable to encode the target image by representing points of the target image in terms of the estimated motion and motion compensation difference data representing intensity adjustments to points of the reference image for reconstructing corresponding points of the target image.

21. A machine-readable medium storing machine-readable instructions for causing a machine to:

compute a transformation mapping at least three noncollinear points substantially coplanar on a scene plane in the target image to corresponding points in a references image of the scene captured at a second image plane different from the first image plane;

identify at least one point in the target image off the scene plane and at least one corresponding point in the reference image;

estimate a motion between the target image and the reference image based on the computed transformation and the identified corresponding off-scene-plane points; and

encode the target image based at least in part on the estimated motion.

23. The machine-readable medium of claim 21, wherein the machine-readable instructions cause the machine to estimate the motion by defining single-parameter search spaces each relating points in the reference image to respective points in the target image.

24. The machine-readable medium of claim 23, wherein the machine-readable instructions cause the machine to define the single-parameter search space by computing an epipole in the reference image based on the computed transformation and the identified corresponding off-scene-plane points.

25. The machine-readable medium of claim 24, wherein the machine-readable instructions cause the machine to define a respective single-parameter search space by parameterizing an epipolar line extending through the computed epipole in the reference image.

26. The machine-readable medium of claim 25, wherein the machine-readable instructions cause the machine to define a respective single-parameter search space for each block of points in the target image.

27. The machine-readable medium of claim 26, wherein a respective single-parameter search space is defined by a parameterized epipolar line in the reference image extending through the epipole and a point corresponding to a mapping of a given point in the target image to a corresponding point in the reference image based on the computed transformation.

28. The machine-readable medium of claim 23, wherein the machine-readable instructions cause the machine to estimate the motion by dividing the target image into blocks of points and computing for each block a respective motion vector representing motion between a target image block and a reference image block.

29. The machine-readable medium of claim 28, wherein each motion vector describes a one-to-one mapping between a block of points in the target image and a block of points in the reference image.

30. The machine-readable medium of claim 21, wherein the machine-readable instructions cause the machine to encode the target image by representing points of the target image in terms of the estimated motion and motion compensation difference data representing intensity adjustments to points of the reference image for reconstructing corresponding points of the target image.

22. The machine-readable medium of claim 22, wherein the machine-readable instructions further cause the machine to identify the at least three scene plane points in the target image and the at least three corresponding scene plane points in the reference image.