Methods and systems for mapping a virtual model of an object to the object
First Claim
1. A method of mapping a model of an object, the model being a virtual model positioned in a virtual 3-D coordinate system in virtual space, substantially to the position of the object in a real 3-D coordinate system in real space, comprising:
- a) computer processing means accessing information indicative of the virtual model;
b) the computer processing means displaying on video display means a virtual image that is a view of at least part of the virtual model, the view being as if from a virtual camera fixed in the virtual coordinate system; and
also displaying on the display means real video images of the real space captured by a real video camera moveable in the real coordinate system;
wherein the real video images of the object at a distance from the camera in the real coordinate system are shown on the display means as being substantially the same size as the virtual image of the virtual model when the virtual model is at that same distance from the virtual camera in the virtual coordinate system;
c) the computer processing means receiving an input indicative of the camera having been moved in the real coordinate system into a position in which the display means shows the virtual image of the virtual model in virtual space to be substantially coincident with the real video images of the object in real space;
d) the computer processing means communicating with sensing means to sense the position of the camera in the real coordinate system;
e) the computer processing means accessing model position information indicative of the position of the virtual model relative to the virtual camera in the virtual coordinate system;
f) the computer processing means responding to the input to ascertain the position of the object in the real coordinate system from the sensed position of the camera sense in step (d) and the model position information of step (e); and
then mapping the position of the virtual model in the virtual coordinate system substantially to the position of the object in the real coordinate system.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems, apparati and methods for mapping a virtual model of a real object, such as a body part, to the real object are presented. Such virtual model can be generated, for example, from an imaging scan of the object, for example, using MRI, CT, etc. A camera with a probe fixed thereto can be moved relative to the object until a video image of the object captured by the camera appears to coincide on a video screen with the virtual model which is shown fixed on that screen. The position of the camera in a real coordinate system can be sensed, and the position in a virtual coordinate system of the virtual model relative to a virtual camera, by which the view of the virtual model on the screen is notionally captured, can be predetermined and known. From this, the position of the virtual model relative to the object can be mapped and a transform generated to position the object in the virtual coordinate system to approximately coincide with the virtual model. After completion of such an initial registration process, a second, refined, registration process can be initiated. Such refined registration process can include acquiring a large number of real points on the surface of the object. Such points can, for example, then be processed using an iterative closest point measure to generate a second, more accurate transform between the object and its virtual model. Further, the refined registration processing can be iterated and more and more accurate transforms generated until a termination condition is met and a final transform generated. Using the final transform generated by this process the virtual model can be positioned in the real coordinate system to substantially exactly coincide with the object.
-
Citations
58 Claims
-
1. A method of mapping a model of an object, the model being a virtual model positioned in a virtual 3-D coordinate system in virtual space, substantially to the position of the object in a real 3-D coordinate system in real space, comprising:
-
a) computer processing means accessing information indicative of the virtual model;
b) the computer processing means displaying on video display means a virtual image that is a view of at least part of the virtual model, the view being as if from a virtual camera fixed in the virtual coordinate system; and
also displaying on the display means real video images of the real space captured by a real video camera moveable in the real coordinate system;
wherein the real video images of the object at a distance from the camera in the real coordinate system are shown on the display means as being substantially the same size as the virtual image of the virtual model when the virtual model is at that same distance from the virtual camera in the virtual coordinate system;
c) the computer processing means receiving an input indicative of the camera having been moved in the real coordinate system into a position in which the display means shows the virtual image of the virtual model in virtual space to be substantially coincident with the real video images of the object in real space;
d) the computer processing means communicating with sensing means to sense the position of the camera in the real coordinate system;
e) the computer processing means accessing model position information indicative of the position of the virtual model relative to the virtual camera in the virtual coordinate system;
f) the computer processing means responding to the input to ascertain the position of the object in the real coordinate system from the sensed position of the camera sense in step (d) and the model position information of step (e); and
then mapping the position of the virtual model in the virtual coordinate system substantially to the position of the object in the real coordinate system. - View Dependent Claims (2, 3, 4, 13, 14, 16, 17, 30, 32, 34, 36)
-
- 5. A method according to an preceding claim and including the step of positioning the virtual model relative to the virtual camera in the virtual coordinate system so as to be a predefined distance from the virtual camera.
-
15. Mapping apparatus for mapping a model of an object, the model being a virtual model positioned in a virtual 3-D coordinate system in virtual space, substantially to the position of the object in a real 3-D coordinate system in real space;
- wherein the apparatus includes computer processing means, a video camera and video display means;
the apparatus arranged such that;
the video display means is operable to display real video images captured by the camera of the real space, the camera being moveable within the real coordinate system; and
the computer processing means is operable to display also on the video display means a virtual image that is a view of at least part of the virtual model, the view being as if from a virtual camera fixed in the virtual coordinate system,wherein the apparatus further includes sensing means to sense the position of the video camera in the real coordinate system and to communicate camera position information indicative of this to the computer processing means, and the computer processing means is arranged to access model position information indicative of the position of the virtual model relative to the virtual camera in the virtual coordinate system and to ascertain from the camera position information and the model position information the position of the object in the real coordinate system, and wherein the computer processing means is arranged to respond to an input indicative of the camera having been moved in the real coordinate system into a position in which the video display means shows the virtual image of the virtual model in virtual space to be substantially coincident with a real video image of the object in real space by mapping the position of the virtual model in the virtual coordinate system substantially to the position of the object in the real coordinate system. - View Dependent Claims (18, 19, 20, 21, 22, 23, 26)
- wherein the apparatus includes computer processing means, a video camera and video display means;
-
24. A method of more closely aligning a model of an object, the model being a virtual model positioned in a 3-D coordinate system in space, with the object in the coordinate system, the virtual model and the object having already been substantially aligned, the method including the steps of:
-
a) computer processing means receiving an input indicating that a real data collection procedure should begin;
b) the computer processing means communicating with sensing means to ascertain the position of a probe in the coordinate system, and thereby the position of a point on the surface of the object when the probe is in contact with that surface;
c) the computer processing means responding to the input to record automatically and at intervals respective real data indicative of each of a plurality of positions of the probe in the coordinate system, and hence indicative of each of a plurality of points on the surface of the object when the probe is in contact with that surface;
d) the computer processing means calculating a transform that substantially maps the virtual model to the real data. e) the computer processing means applying the transform to more closely align the virtual model with the object in the coordinate system. - View Dependent Claims (25, 27, 28, 29, 31, 33, 35, 37)
-
-
38. A method of registering a virtual model of a real object with the real object, comprising:
-
performing an initial registration between the virtual model and the real object; and
subsequently performing a refined registration between the virtual model and the real object, wherein the initial registration includes visually aligning an image of the virtual model of the object displayed on a display with a real-time image of the real object displayed on the display by causing one of the images to translate and or rotate relative to the other one, and wherein the refined registration includes acquiring the locations of a defined number of points on a surface of the real object, using those points and a set of respective corresponding points in the virtual model to find an overall best fit between said real points and said respective corresponding virtual points, and generating a transformation of the virtual model to the real object based upon said best fit. - View Dependent Claims (39, 40, 41, 42, 43, 44, 45)
-
-
46. A computer program product comprising a computer usable medium having computer readable program code means embodied therein, the computer readable program code means in said computer program product comprising means for causing a computer to:
-
perform an initial registration between the virtual model and the real object; and
subsequently perform a refined registration between the virtual model and the real object, wherein the initial registration includes visually aligning an image of the virtual model of the object displayed on a display with a real-time image of the real object displayed on the display by causing one of the images to translate and or rotate relative to the other one, and wherein the refined registration includes acquiring the locations of a defined number of points on a surface of the real object, using those points and a set of respective corresponding points in the virtual model to find an overall best fit between said real points and said respective corresponding virtual points, and generating a transformation of the virtual model to the real object based upon said best fit. - View Dependent Claims (47, 48, 49, 50, 51, 52, 53, 54, 57, 58)
-
-
55. A system for registering a virtual model of a real object with the real object, comprising:
-
at least one computer;
a memory arranged to store a virtual model of a real object;
a display;
a probe with an integrated camera; and
a tracking system, wherein, in operation, real images of the real object acquired by the camera and a virtual image of the virtual model are displayed on the display in a combined image, and wherein a user performs a first registration by aligning a real image with the virtual image, and a refined registration by moving the probe over the surface of the real object to acquire the locations of a set of points, and wherein the computer associates the set of real points with corresponding respective closest points in the virtual model, and uses the real points and the corresponding respective closest points to find an overall best fit between said real points and said corresponding respective virtual points, and generates a transformation of the virtual model to the real object based upon said best fit. - View Dependent Claims (56)
-
Specification