METHOD AND SYSTEM FOR REGISTRATION VERIFICATION

• US 20190108645A1
• Filed: 04/19/2018
• Published: 04/11/2019
• Est. Priority Date: 04/21/2016
• Status: Active Grant

• Current Assignee: Elbit Systems Limited
• Original Assignee: Elbit Systems Limited
• Inventors: BEN-YISHAI, Rani
• Granted Patent: US 10,482,614 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2063 Acoustic tracking systems, e.g. using ultrasound
• CPC Class Codes: A61B 2034/2068 using pointers, e.g. pointers having reference marks for determining coordinates of body points
• CPC Class Codes: A61B 2034/2074 Interface software
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/373 using light, e.g. by using optical scanners
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 2090/3945 Active visible markers, e.g. light emitting diodes
• CPC Class Codes: A61B 2090/395 with marking agent for marking skin or other tissue
• CPC Class Codes: A61B 2090/3966 Radiopaque markers visible in an X-ray image
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/337 : involving reference images or patches
• CPC Class Codes: G06T 7/75 : involving models

Abstract

Method for verifying registration of a model of an internal-body-part with the internal-body-part in a reference coordinate system. The internal-body-part is at least partially unseen directly by a user. The method includes the procedures of continuously determining a position and orientation of a head-mounted-display in the reference coordinate system, determining a display location of at least one virtual marker, and displaying the virtual marker according to the display location. The display location of the virtual marker is determined according to the expected position of a respective at least one reference point relative to the head-mounted-display. The reference point is directly visible to the user. The relative position between the reference point and the internal-body-part is substantially constant. The position of the reference point in the reference coordinate system is predetermined. When the model and the internal-body-part are effectively registered, the reference point and corresponding virtual marker appears visually in alignment.

Claims

1. A method for verifying registration of a model of an internal body part with the internal body part in a reference coordinate system, said internal body part being at least partially unseen directly by a user, the method comprising the procedures of:

continuously determining a position and orientation of a head mounted display in said reference coordinate system;

determining a display location of at least one virtual marker according to the expected position of a respective at least one reference point relative to said head mounted display, said at least one reference point being directly visible to said user, the relative position between said at least one reference point and said internal body part being substantially constant, the position of said at least one reference point in said reference coordinate system is predetermined;

displaying said at least one virtual marker according to said display location,

wherein, when said model and said internal body part are effectively registered, said at least one reference point and said corresponding at least one virtual marker appears visually in alignment.

2. The method according to claim 1, wherein, when said model and said internal body part effectively fall out of registration, said at least one reference point and corresponding virtual marker appears visually out of alignment.

3. The method according to claim 1, further including the procedures of registering said model with said internal body part in said reference coordinate system;

and

determining the position of said at least one reference point in said reference coordinate system;

4. The method according to claim 3, wherein said position of said at least one reference point is determined relative to said body part by marking said at least one reference point on said model, when said at least one reference point is visible on said model and directly visible by said user.

5. The method according to claim 3, wherein said position of said at least one reference point is determined relative to said body part by determining the position of a tip of tool in said reference coordinate system when said tip touches each of said at least one reference point.

6. The method according to claim 3, wherein said position of said at least one reference point is determined relative to said body part by determining the position and orientation of said head mounted display while said user designates each one of said at least one reference point from at least two view angles respective of at least two positions of said user.

7. The method according to claim 1, wherein said at least one reference point is one of:

a marking on the skin;

an adhesive marker attached to the skin;

a part of a fixture rigidly attached to said body part; and

an anatomical feature.

8. The method according to claim 1, wherein said model is at least one of:

a one dimensional model;

a two dimensional image;

a three dimensional segmentation based on one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data;

a volume rendering visualization of one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data;

9. The method according to claim 8, wherein said two dimensional image is one of a pre-acquired two dimensional image and an intra-operative two dimensional imager.

10. A system for verifying registration of a model of a body part with the internal body part in a reference coordinate system, said body part being at least partially unseen directly by a user, the system comprising:

a head mounted display to be donned by said user;

a head mounted display tracking module for continuously determining information relating to the position and orientation of said head mounted display in said reference coordinate system; and

a processor, coupled with said head mounted display tracking module, determining the position and orientation of said head mounted display in said reference coordinate system and determining the display location of at least one virtual marker according to the position of a respective at least one reference point relative to said head mounted display, said reference point being visible to said user, the relative position between said at least one reference point and said body part being substantially constant, the position of said at least one reference point in said reference coordinate system is predetermined, said processor rendering said at least one virtual marker to be displayed on said head mounted display,

wherein, when said model and said body part are effectively registered, said at least one reference point and corresponding virtual marker are displayed on said head mounted display visually in alignment.

11. The system according to claim 10, wherein, when said model and said body part effectively fall out of registration, said at least one reference point and corresponding virtual marker shall appear on said head mounted display visually out of alignment.

12. The system according to claim 10, wherein said processor registers said model with said reference coordinate system, and

wherein said processor determines the position of said at least one reference point.

13. The method according to claim 12, wherein said position of said at least one reference point is determined relative to said body part by marking said at least one reference point on said model, when said at least one reference point is visible on said model and directly visible by said user.

14. The system according to claim 12, wherein said processor determines said position of said of said at least one reference point relative to said body part by determining the position and orientation of said head mounted display while said user designates each one of said at least one reference point from at least two view angles respective of at least two positions of said user.

15. The system according to claim 10, wherein said at least one reference point is one of:

a marking on the skin;

an adhesive marker attached to the skin;

a part of a fixture rigidly attached to said body part; and

an anatomical feature.

16. The system according to claim 10, wherein said tracking module includes a head mounted display tracking unit and a reference tracking unit,

wherein, said head mounted display tracking unit being fixedly attached to said head mounted display, and

wherein said reference tracking unit is fixedly placed relative to said internal body part.

17. The system according to claim 16, wherein said head mounted display tracking unit includes an optical detector and at least one head mounted display light emitter,

wherein said reference tracking unit includes a reference optical detector and at least one reference light emitter,

wherein the total number of light emitters is at least three, and

wherein said reference optical detector acquires at least one image of said at least one head mounted display light emitter and said head mounted display optical detector acquires at least one image of said at least one reference light emitter.

18. The system according to claim 16, wherein said head mounted display tracking unit includes an optical detector, and

wherein said reference tracking unit includes at least three reference light emitters.

19. The system according to claim 16, wherein said head mounted display tracking unit includes at least three light emitters, and

wherein said reference tracking unit includes an optical detector.

20. The system according to claim 16, wherein said head mounted display tracking unit and said reference tracking unit each includes at least three light emitters, and

wherein said tracking module further includes at least one optical detector placed near the surgical field, such that a clear line-of-sight exists between said at least one optical detector and said surgical field.

21. The system according to claim 16, wherein said tracking module further includes a tool tracking unit for determining the position and orientation of a tool.

22. The system according to claim 21, wherein said processor determines said position of said of said at least one reference point relative to said body part by determining the position of a tip of tool in said reference coordinate system when said tip touches each of said at least one reference point.

23. The system according to claim 10, wherein said model is at least one of:

a one dimensional model;

a two dimensional image;

is a three dimensional segmentation based on one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data;

a volume rendering of one of a pre-acquired three dimensional image data and an intra-operative three-dimensional image data.

24. The system according to claim 10, wherein said two dimensional image is one of a pre-acquired two dimensional image and an intra-operative two dimensional imager.

25. The system according to claim 10, further including a camera coupled with said processor, said camera acquiring an image from the point of view of the user, said image including representations of said at least one reference point and virtual markers, said camera providing the acquired image to said processor, said processor detecting the alignment or misalignment between said at least one reference point and corresponding virtual marker.

26. A method for verifying registration of a model of an internal body part with the internal body part in a reference coordinate system, said internal body part being at least partially unseen directly by a user, the method comprising the procedures of:

acquiring an image of at least one reference point from a known position and orientation in said reference coordinate system;

identifying the at least one reference point in said acquired image and determining the image location of said at least one reference point, said at least one reference point being directly visible to said user, the relative position between said at least one reference point and said internal body part being substantially constant, the position of said at least one reference point in said reference coordinate system is predetermined;

determining the expected image location of at least one reference point according to the position of said at least one reference point and said position and orientation from which said image was acquired in the reference coordinate system;

determining the congruence between said predicted and determined image location of said at least one reference point,

wherein when said model and said internal body part are registered, said predicted and determined image locations are congruent with each other.

27. The method according to claim 26, wherein, when said model and said internal body part fall out of registration, said predicted and determined image locations are not congruent with each other.

28. The method according to claim 26, further including the procedures of registering said model with said internal body part in said reference coordinate system;

and

determining the position of at least one reference point in said reference coordinate system;

29. The method according to claim 26, wherein said image is one of an interrogation image and a tracking image.