Dynamic visual registration of a 3-D object with a graphical model
First Claim
1. A method for finding the relative position and orientation of an imaging device in relation to an imaged object, wherein said imaging device produces a video signal representing a two-dimensional projection of said imaged object onto an image frame, the method comprising the steps of:
- recognizing a plurality of features from the video signal;
associating with each recognized feature a two-dimensional position in the image frame;
computing a disparity between the positions of the features and corresponding positions of features in a calculated projection of a stored, three-dimensional object model, said calculated projection calculated according to a virtual camera model; and
varying parameters of the virtual camera model, consistent with a projective geometric model, to reduce said disparity;
wherein said disparity is reduced by computing and applying a transformation matrix including translation sub-matrix and rotation sub-matrix, and wherein said sub-matrices are calculated from translation and rotation sub-vectors of a vector obtained by multiplying a constant with a product of an error function and a pseudo-inverse of an interaction matrix L;
wherein said interaction matrix L represents two dimensional coordinates xp, yp and a depth Z for a plurality of pre-defined points in a calculated three dimensional object model, and said two dimensional coordinates xp, yp are calculated projected positions of said pre-defined points onto a hypothetical plane of projection which corresponds to an image plane of said virtual camera model.
13 Assignments
0 Petitions
Accused Products
Abstract
The invention displays computer graphics in combination with imagery of real objects, while maintaining apparent alignment notwithstanding any changes of viewpoint of an imaging device relative to the real object. A computer executed control loop recognizes features in the image and finds a corresponding position and orientation of a CAD model by projecting the CAD representation onto a “virtual camera” and “moving” the virtual camera to track the relative motion of the real imaging device, according to an efficient “visual servoing” algorithm. In an alternate embodiment of the invention, computing tasks are divided between an “image processing host” and one or more “display hosts” which communicate over a channel. Bandwidth is conserved by performing image registration locally at the display host(s) using the “visual servoing” algorithm.
-
Citations
12 Claims
-
1. A method for finding the relative position and orientation of an imaging device in relation to an imaged object, wherein said imaging device produces a video signal representing a two-dimensional projection of said imaged object onto an image frame, the method comprising the steps of:
-
recognizing a plurality of features from the video signal;
associating with each recognized feature a two-dimensional position in the image frame;
computing a disparity between the positions of the features and corresponding positions of features in a calculated projection of a stored, three-dimensional object model, said calculated projection calculated according to a virtual camera model; and
varying parameters of the virtual camera model, consistent with a projective geometric model, to reduce said disparity;
wherein said disparity is reduced by computing and applying a transformation matrix including translation sub-matrix and rotation sub-matrix, and wherein said sub-matrices are calculated from translation and rotation sub-vectors of a vector obtained by multiplying a constant with a product of an error function and a pseudo-inverse of an interaction matrix L;
wherein said interaction matrix L represents two dimensional coordinates xp, yp and a depth Z for a plurality of pre-defined points in a calculated three dimensional object model, and said two dimensional coordinates xp, yp are calculated projected positions of said pre-defined points onto a hypothetical plane of projection which corresponds to an image plane of said virtual camera model. - View Dependent Claims (2, 3, 4)
reiteratively repeating said previously recited steps in a loop, to further reduce said disparity.
-
-
5. A method of registering a two-dimensional object image of a three-dimensional object with a stored, three-dimensional model having a predetermined spatial relationship to the three-dimensional object, comprising the steps of:
-
identifying the positions of a plurality of features in said object image;
rendering a virtual image by projecting at least portions of said three-dimensional model onto a virtual image plane;
calculating an error function which measures a difference between positions of said plurality of features in said object image and corresponding positions of previously associated features in said virtual image;
finding a position and/or orientation of said virtual image plane which reduces said error function, wherein said position and/or orientation are found by computing and applying a transformation matrix including a translation sub-matrix and a rotation sub-matrix, and wherein said sub-matrices are calculated from translation and rotation sub-vectors of a vector obtained by multiplying a constant with a product of an error function and a pseudo-inverse of an interaction matrix L;
wherein said interaction matrix L represents two dimensional coordinates xp, yp and a depth Z for a plurality of pre-defined points in a calculated three dimensional object model, and said two dimensional coordinates xp, yp are calculated projected positions of said pre-defined points onto the virtual image plane. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12)
-
Specification