System and method for computing rectifying homographies for stereo vision processing of three dimensional objects

• US 6,606,404 B1
• Filed: 06/19/1999
• Issued: 08/12/2003
• Est. Priority Date: 06/19/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Loop, Charles T.
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US09/336,222
• Time in Patent Office: 1,515 Days
• Field of Search: 382/100, 382/154, 382/250, 382/276, 382/280, 382/284, 382/294, 348/42, 348/47, 345/419, 345/426, 345/427
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The present invention is embodied in a system and method for computing rectifying homographies for stereo vision processing of three dimensional (3D) objects. In general, the system and method of the present invention computes 2D projective transforms or homographies derived from specialized projective and affine transforms or components. The affine transform is comprised of a similarity transform and an optional shearing transform for image rectification. During image rectification of the present invention, optimization techniques are used for reducing the distortion.

Claims

1. A method for computing rectifying homographies of images of an object comprising:

computing a special projective transform for each image that minimizes a well defined projective distortion criteria;

computing a similarity transform that satisfies predefined constraints for rectification; and

combining the special projective transform with the similarity transform into a rectifying homography.

2. The method of claim 1, further comprising obtaining a fundamental matrix between the images.

3. The method of claim 1, wherein the similarity transform is an affine component.

4. The method of claim 1, further comprising computing a shearing transform that further reduces distortion introduced by the rectification.

5. The method of claim 4, further comprising combining the similarity transform with the shearing transform to define an affine component and combining the affine component with the special projective transform into the rectifying homography.

6. The method of claim 2, wherein the fundamental matrix embodies epipolar geometrical relationships between the images.

7. The method of claim 6, further comprising sending epipoles of the epipolar geometrical relationships to infinity so that epipolar lines become parallel by respective special projective transforms.

8. The method of claim 7, further comprising capturing image distortion due to projective transformation by respective special projective transforms.

9. The method of claim 1, wherein respective special projective transforms are computed to characterize projective components of the rectifying homography.

10. The method of claim 6, further comprising mapping the epipoles to points with coordinates of a predetermined coordinate axis equal to zero.

11. A rectification system for processing two-dimensional images of a three-dimensional object comprising:

a homographical processor that computes a special projective transform of the images comprised of isolated projective characteristics and a similarity transform that satisfies predefined constraints for rectification; and

a geometrical processor that combines the special projective and similarity transforms into a rectifying homography.

12. The rectification system of claim 11, wherein the homographical processor obtains a fundamental matrix between the images.

13. The rectification system of claim 12, wherein the fundamental matrix embodies epipolar geometrical relationships between the images.

14. The rectification system of claim 13, wherein the similarity transform is computed by rotating and translating the images such that the epipolar lines are horizontally aligned so that it satisfies predefined constraints for rectification.

15. The rectification system of claim 11, wherein the similarity transform is an affine component.

16. The rectification system of claim 11, wherein the homographical processor further computes a shearing transform that reduces distortion introduced by the rectification.

17. The rectification system of claim 16, wherein the geometrical processor combines the similarity transform with the shearing transform to define an affine component and further combines the affine component with the special projective transform into the rectifying homography.

18. The rectification system of claim 16, wherein the shearing transform is computed by scaling and translating the images in a horizontal direction while preserving original image resolution to reduce distortion introduced by the special projective transform.

19. The rectification system of claim 13, wherein the geometrical processor further sends epipoles of the epipolar geometrical relationships to infinity so that epipolar lines become parallel by respective special projective transforms.

20. The rectification system of claim 18, wherein the homographical processor assigns weights to pixel elements defining the images and minimizes variations of the assigned weights over all pixels such that the transformation is near affine.

21. The rectification system of claim 20, wherein respective special projective transforms are computed to characterize projective components of the rectifying homography.

22. The rectification system of claim 20, wherein the homographical processor maps the epipoles to points with coordinates of a predetermined coordinate axis equal to zero.

23. A computer-readable storage device encoded with a set of instructions for rectifying images, the set of instructions comprising:

isolating projective characteristics of the images defined by a special projective transform;

obtaining at least one an affine component defined by a similarity transform that satisfies predefined constraints for rectification; and

combining the transforms into a rectifying homography.

24. The set of instructions of claim 23, wherein the special projective transform is comprised of components that minimize a predefined projective distortion criteria.

25. The set of instructions of claim 23, further comprising obtaining a fundamental matrix between the images.

26. The set of instructions of claim 25, wherein the fundamental matrix embodies epipolar geometrical relationships between the images.

27. The set of instructions of claim 26, further comprising sending epipoles of the epipolar geometrical relationships to infinity so that epipolar lines become parallel by respective special projective transforms.

28. The set of instructions of claim 27, further comprising capturing image distortion due to projective transformation by respective special projective transforms.

29. The set of instructions of claim 28, further comprising assigning weights to pixel elements defining the images and minimizing variations of the assigned weights over predetermined pixels to make the transformation affine by respective special projective transforms.

30. The set of instructions of claim 23, wherein respective special projective transforms are obtained so that they characterize projective components of the rectifying homography.

31. The set of instructions of claim 27, further comprising mapping the epipoles to points with coordinates of a predetermined coordinate axis equal to zero.

32. The set of instructions of claim 25, wherein the similarity transform is obtained by rotating and translating the images such that the epipolar lines are horizontally aligned so that it satisfies predefined constraints for rectification.

33. The set of instructions of claim 23, wherein the similarity transform is an affine component.

34. The set of instructions of claim 23, further comprising obtaining a shearing transform that reduces distortion introduced by the rectification.

35. The set of instructions of claim 34, further comprising combining the similarity transform with the shearing transform to define a multi-part affine component and further combining the multi-part affine component with the special projective transform into the rectifying homography.

36. The set of instructions of claim 34, wherein the shearing transform is obtained by scaling and translating the images in a horizontal direction while preserving original image resolution to reduce distortion introduced by the special projective transform.