System and method for rectifying images of three dimensional objects

• US 6,608,923 B1
• Filed: 06/19/1999
• Issued: 08/19/2003
• Est. Priority Date: 06/19/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Zhengyou, Loop, Charles T.
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US09/336,549
• Time in Patent Office: 1,522 Days
• Field of Search: 382/100, 382/154, 382/250, 382/276, 382/280, 382/284, 382/294, 348/42, 348/47, 345/419, 345/426, 345/427
• US Class Current: 382/154
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19

Abstract

The present invention is embodied in a system and method for rectifying two dimensional images of three dimensional (3D) objects for stereo vision processing. In general, the system and method of the present invention computes 2D projective transforms or homographies derived from specialized projective and affine transforms or components. The affine transform is comprised of a first transform and an optional second transform for image rectification. During image rectification of the present invention, optimization techniques are used for reducing the distortion.

Claims

1. A method for processing images of an object comprising:

computing a first homography transform of the images, the first homography transform comprised of isolated projective characteristics;

computing a second homography transform of the images, the second homography transform comprised of at least one an affine component; and

combining the homographies into a rectifying homography.

2. The method of claim 1, wherein the first homography is comprised of components that minimize a predefined projective distortion criteria.

3. The method of claim 1, further comprising obtaining a fundamental matrix between the images.

4. The method of claim 3, wherein the fundamental matrix embodies epipolar geometrical relationships between the images.

5. The method of claim 4, further comprising sending epipoles of the epipolar geometrical relationships to infinity so that epipolar lines become parallel by respective first homography transforms.

6. The method of claim 5, further comprising capturing image distortion due to projective transformation by respective first homography transforms.

7. The method of claim 6, further comprising assigning weights to pixel elements defining the images and minimizing variations of the assigned weights over all pixels to make the transformation affine by respective first homography transforms.

8. The method of claim 1, wherein respective first homography transforms are computed to characterize projective components of the rectifying homography.

9. The method of claim 5, further comprising mapping the epipoles to points with coordinates of a predetermined coordinate axis equal to zero.

10. A rectification system for processing two-dimensional images of a three-dimensional object comprising:

a homographical processor that computes a first homography transform of the images comprised of isolated projective characteristics and a second homography of the images comprised of at least one an affine component; and

a geometrical processor that combines the first and second homographies into a rectifying homography.

11. The rectification system of claim 10, wherein the first homography transform is comprised of components that minimize a predefined projective distortion criteria.

12. The rectification system of claim 10, wherein the homographical processor obtains a fundamental matrix between the images.

13. The rectification system of claim 12, wherein the fundamental matrix embodies epipolar geometrical relationships between the images.

14. The rectification system of claim 13, wherein the homographical processor sends epipoles of the epipolar geometrical relationships to infinity so that epipolar lines become parallel by respective first homography transforms.

15. The rectification system of claim 14, wherein the homographical processor captures image distortion due to projective transformation.

16. The rectification system of claim 14, wherein all of the image distortion due to projective transformation is captured by the homographical processor.

17. The rectification system of claim 15, wherein the homographical processor assigns weights to pixel elements defining the images and minimizes variations of the assigned weights over all pixels such that the transformation is near affine.

18. The rectification system of claim 10, wherein respective first homography transforms are computed to characterize projective components of the rectifying homography.

19. The rectification system of claim 14, wherein the homographical processor maps the epipoles to points with coordinates of a predetermined coordinate axis equal to zero.

20. A computer-readable storage device encoded with a set of instructions for rectifying images, the set of instructions comprising:

isolating projective characteristics of the images defined by a first homography transform;

obtaining at least one an affine component defined by a second homography transform of the images; and

combining the homographies into a rectifying homography.

21. The set of instructions of claim 20, wherein the first homography is comprised of components that minimize a predefined projective distortion criteria.

22. The set of instructions of claim 20, further comprising obtaining a fundamental matrix between the images.

23. The set of instructions of claim 22, wherein the fundamental matrix embodies epipolar geometrical relationships between the images.

24. The set of instructions of claim 23, further comprising sending epipoles of the epipolar geometrical relationships to infinity so that epipolar lines become parallel by respective first homography transforms.

25. The set of instructions of claim 24, further comprising capturing image distortion due to projective transformation by respective first homography transforms.

26. The set of instructions of claim 25, further comprising assigning weights to pixel elements defining the images and minimizing variations of the assigned weights over predetermined pixels to make the transformation affine by respective first homography transforms.

27. The set of instructions of claim 20, wherein respective first homography transforms are obtained so that they characterize projective components of the rectifying homography.

28. The set of instructions of claim 24, further comprising mapping the epipoles to points with coordinates of a predetermined coordinate axis equal to zero.