Method and apparatus for determining camera pose

• US 7,613,323 B2
• Filed: 06/22/2005
• Issued: 11/03/2009
• Est. Priority Date: 06/22/2004
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Nister, David, Bergen, James
• Primary Examiner(s): Bella; Matthew C
• Assistant Examiner(s): Rush; Eric
• Application Number:
  US11/159,967
• Publication Number:
  US 20060013437A1
• Time in Patent Office: 1,595 Days
• Field of Search: 382/100, 382/103, 382/106, 382/291
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method and apparatus for determining camera pose characterized by six degrees of freedom (e.g., for use in computer vision systems) is disclosed. In one embodiment an image captured by the camera is received, and at least two constraints on the potential pose are enforced in accordance with known relations of the image to the camera, such that the potential pose is constrained to two remaining degrees of freedom. At least one potential pose is then determined in accordance with the remaining two degrees of freedom.

Claims

1. A computer implemented method for determining at least one potential pose of an image capturing device, said at least one potential pose having six degrees of freedom, the method comprising:

using a processor to perform the steps of;

receiving an image captured by said image capturing device, said image including images of at least;

a first know world point, a second known world point and a third known world point; and

imposing constraints on said at least one potential pose in accordance with known relations of said image to said image capturing device, said imposing comprising;

enforcing constraints given by a first ray, a second ray and a third ray, where said first ray, said second ray and said third ray each project from said image capturing device to, respectively, said first known world point, said second known world point and said third known world point,

wherein said enforcing constraints given by said first ray and said second ray constrains said at least one potential pose to two remaining degrees of freedom; and

wherein a first of said two remaining degrees of freedom constrains a position of said third known world point relative to said image capturing device within an area defined by a circle and a second of said two remaining degrees of freedom constrains a position of said third ray relative to said image capturing device within an area defined by a ruled quartic surface, said circle and said ruled quartic surface having a plurality of intersection points; and

determining said at least one potential pose in accordance with said constraints,

wherein said determining comprises the steps of;

lining up said first ray, said second ray, and said third ray in their canonical positions;

lining up said first known world point, said second known world point, and said third known world point in their canonical positions;

computing coefficients of two plane equations representing said third ray;

computing coefficients of an octic polynomial whose roots correspond to said plurality of intersection points of said circle with said ruled quartic surface;

extracting the roots of said octic polynomial; and

backsubstituting with each root of the octic polynomial to get solutions for the transformation between the coordinate system of said first known world point, said second known world point and said third known world point and the coordinate system of said first ray, said second ray and said third ray.

2. The method of claim 1, wherein said remaining two degrees of freedom comprise:

a first degree of freedom represent a transformation of said third known world point; and

a second degree of freedom representing at least one possible position of a third ray projecting from said image capturing device to said third known world point.

3. The method of claim 1, wherein each of said plurality of intersection points identifies one of said at least one potential pose of said image capturing device with respect to said first known world point, said second known world point and said third known world point.

4. The method of claim 3, therein each of said plurality of intersection points is determined by said extracting the roots of said octic polynomial.

5. The method of claim 1, wherein said image capturing device conforms to a non-central projection image formation model.

6. The method of claim 5, wherein said image capturing device comprises at least one of:

a multi camera rig, a catadioptric camera, a dioptric camera or a multi-lens camera.

7. The method of claim 5, wherein said image capturing device is a stereo camera head.

8. The method of claim 5, wherein sad image capturing device comprises a plurality of cameras positioned at different locations, relative orientations of said plurality of cameras being known.

9. The method of claim 5, wherein said image capturing device comprises a camera mounted to a vehicle for capturing a plurality of images as said vehicle moves.

10. A computer-readable memory having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of a method for determining at least one potential pose of an image capturing device, said at least one potential pose having six degrees of freedom, the method comprising:

receiving an image captured by said image capturing device, said in age including images of at least;

a first known world point, a second known world point and a third known work point; and

enforcing constraints on said at least one potential pose in accordance with known relations of said image to said image capturing device, said enforcing comprising;

enforcing constraints given by a first ray, a second ray and a third ray, where said first ray, said second ray and said third ray each project from said image capturing device to, respectively, said first known world point, said second known world point and said third known world point, wherein said enforcing constraints given by said first ray and said second ray constrains said at least one potential pose to two remaining degrees of freedom; and

wherein a first of said two remaining degrees of freedom constrains a position of said third known world point relative to said image capturing device within an area defined by a circle and a second of said two remaining degrees of freedom constrains a position of said third ray relative to said image capturing device within an area defined by a ruled quartic surface, said circle and said ruled quartic surface having a plurality of intersection points; and

determining at least one potential pose in accordance with said constraints,

wherein said determining comprises the steps of;

lining up said first ray, said second ray, and said third ray in their canonical positions;

lining up said first known world point, said second known world point, and said third known world point in their canonical positions;

computing coefficients of two plane equations resenting said third ray;

computing coefficients of an octic polynomial whose roots correspond to said plurality of intersection points of said circle with said ruled quartic surface;

extracting the roots of said octic polynomial; and

backsubstituting with each root of the octic polynomial to get solutions for the transformation between the coordinate system of said first known world point, said second known world point and said third known world point and the coordinate system of said first ray, said second ray and said third ray.

11. The computer-readable memory of claim 10, wherein said remaining two degrees of freedom comprise:

a first degree of freedom representing a transformation of said third known world point; and

a second degree of freedom representing at least one possible position of a third ray projecting from said image capturing device to said third known world point.

12. The computer-readable memory of claim 10, wherein each of said plurality of intersection points identities one of said at least one potential pose of said image capturing device with respect to said first known world point, said second known world point and said third known world point.

13. The computer-readable memory of claim 12, wherein each of said plurality of intersection points is determined extracting die roots of said octic polynomial.

14. The computer-readable memory of claim 10, wherein said image capturing device conforms to a non-central projection image formation model.

15. The computer-readable memory of claim 14, wherein said image capturing device comprises at least one of:

a multi-camera rig, a catadioptric camera, a dioptric camera or a multi-lens camera.

16. The computer-readable memory of claim 15, wherein said image capturing device is a stereo camera head.

17. The computer-readable memory of claim 14, wherein said image capturing device comprises a plurality of cameras positioned at different locations, relative orientations of said plurality of cameras as being known.

18. The computer-readable memory of claim 14, wherein said image capturing device comprises a camera mounted to a vehicle for capturing a plurality of images as said vehicle moves.

19. An apparatus for determining at least one potential pose of an image capturing device, said potential pose having six degrees of freedom, the apparatus comprising:

means for receiving an image captured by said image capturing device, said image including images of at least;

a first known world point, a second known world point and a third known world point; and

means for enforcing constraints on said at least one potential pose in accordance with known relations of said image to said image capturing device. said means for enforcing comprising;

means for enforcing constraints given by a first ray, a second ray an a third ray, where said first ray, said second ray and said third ray each project from said image capturing, device to, respectively, said first known world point, said second known world point and said third known world point, wherein said means for enforcing constraints given said first ray and said second ray constrains said at least one potential pose to two remaining degrees of freedom; and

wherein a first of said two remaining degrees of freedom constrains a position of said third known world point relative to said image capturing device within an area defined by a circle and a second of said two remaining degrees of freedom constrains a position of said third ray relative to said image capturing device within an area define by a ruled quartic surface, said circle and said ruled quartic surface having a plurality of intersection points; and

means for determining at least one potential pose in accordance with said constraints wherein said means for determining comprises;

means for lining up said first ray said second ray and said third ray in their canonical positions;

means for lining up said first known world point, said second known world point, and said third known world point in their canonical positions;

means tor computing coefficients of two plane equations representing said third ray;

means for computing coefficients of an octic polynomial whose roots correspond to said plurality of intersection points of said circle with said ruled quartic surface;

means for extracting the roots of said octic polynomial; and

means for backsubstituting with each root of the octic polynomial to get solutions for the transformation between the coordinate system of said fist known world point, said second known world point and said third known world point and the coordinate system of said first ray, said second ray and said third ray.