System for and method of visualizing an interior of body

• US 9,248,000 B2
• Filed: 08/14/2009
• Issued: 02/02/2016
• Est. Priority Date: 08/15/2008
• Status: Active Grant

• Current Assignee: Stryker European Operations Holdings, LLC (Stryker Corporation)
• Original Assignee: Stryker European Holdings I, LLC (Stryker Corporation)
• Inventors: Sarvestani, Amir, Schoepp, Hans, Welte, Klaus
• Primary Examiner(s): Chea, Philip
• Assistant Examiner(s): Nguyen, Van Kim T
• Application Number:
  US12/583,148
• Publication Number:
  US 20100039506A1
• Time in Patent Office: 2,363 Days
• Field of Search: 348/65, 348/73, 348/75, 348/76, 348/77
• US Class Current: 1/1
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 2090/3983 Reference marker arrangements for use with image guided surgery
• CPC Class Codes: A61B 2090/3995 Multi-modality markers
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

A navigation system includes a display monitor, a CPU, and a camera, wherein the camera is mounted to a back side of the display monitor to form a monitor unit. One or more reference units are placed on a body while acquiring an image data set, and are tracked during a surgical operation by the monitor unit to register and correlate a position of a visual image of an exterior surface of the body with the image data set including information concerning internal structures of the body. The image data set is displayed on the display monitor superimposed in registration over the visible image of the exterior of the body with an angle of view and aperture in accordance with the actual position of the camera, whereby the display monitor displays the internal structures corresponding to the line-of-sight of the camera and the observer.

Claims

1. A system for visualizing an interior of a body from an exterior position, comprising:

a navigation system having a monitor and a camera, wherein the camera is rigidly fixed to the monitor and serves as a global reference system, wherein the monitor is movably positionable between an operator and the body;

an optical reference unit adapted to be attached to the body for tracking movements of the body;

an electronic processing device in communication with the navigation system; and

an image data set comprising features of the interior of the body, the image data set being accessible to the processing device;

wherein the processing device receives an optical image of an exterior surface of the body obtained with the camera, registers the image data set with respect to the global reference system of the camera by matching a spatial position of the optical reference unit in the image data set with a spatial position of the optical reference unit in the optical image without tracking either the monitor, or the camera, and causes the monitor to display the image data set superimposed and in registration with the optical image in a position and orientation that represents an angle of viewing and an aperture angle of the camera relative to the body.

2. The system of claim 1, wherein the monitor is aligned with a focal axis of the camera.

3. The system of claim 2, wherein the monitor is on a first side of the navigation system and the monitor comprises a back side, the camera is rigidly mounted to the back side of the monitor.

4. The system of claim 3, further comprising a movable arm, wherein the navigation system is carried by the movable arm, and wherein the movable arm is connected to a support structure.

5. The system of claim 1, further comprising surgical plan information, wherein the processing device continually tracks movement of the body relative to the camera, and wherein the processing device causes the monitor to display the surgical plan information superimposed with the optical image in a position and orientation in registration with the optical image.

6. The system of claim 5, wherein the surgical plan information includes at least one of a target location inside the body and a trajectory path for a tool and an incision location on the body.

7. The system of claim 1, wherein the optical reference unit includes an optical pattern of alternating light and dark contrast areas that are detectable by the processing device in the optical image and a radiopaque marker that is visible in the image data set, wherein the radiopaque marker is in an identifiable and known location with respect to the optical pattern.

8. The system of claim 7, wherein the optical pattern includes a distinguishing feature, wherein the distinguishing feature causes the optical pattern to be unique from each possible viewing angle.

9. The system of claim 8, wherein the optical reference unit comprises at least three radiopaque markers.

10. The system of claim 1, wherein the optical reference unit includes an optical pattern that provides functionality as a tracking device and a sufficient number of markers that are detectable in the image data set to provide functionality as a registration device, wherein the reference unit enables automatic registration of the image data set to the optical image by the processing device.

11. A method of visualizing an interior of a body from an exterior viewing position, the method comprising the steps of:

obtaining an internal image data set of an internal portion of the body, wherein the internal image data set includes a reference location on an exterior surface of the body;

accessing the internal image data set with a navigation system comprising a visual display screen, a camera rigidly fixed to the visual display screen in alignment with the display screen, and a digital electronic processing unit in data communication with the camera and the display screen, wherein the camera serves as a global reference system and the visual display screen is movably positionable between an operator and the body;

obtaining a visual image of an exterior of the body with the camera, wherein the reference location is visible in the visual image and oriented to the global reference system;

identifying the reference location in the visual image;

registering with the digital electronic processing unit the internal image data set to the visual image in relation to the global reference system of the camera by matching a spatial position of the reference location in the internal image data set with a spatial position of the reference location in the visual image without tracking either the monitor or the camera;

displaying the internal image data set superimposed and in registration with the visual image on the visual display screen in a position and orientation that represents an angle of view and an aperture angle of the camera and the visual display screen relative to the body;

tracking movements of the body with respect to the camera using the reference location in the visual image; and

updating the display of the internal image data set with the digital electronic processing unit to correspond with the position of the body with respect to the camera based on the tracked movements of the body.

12. The method of claim 11, wherein the step of registering comprises at least one of the step of performing a point by point correlation of one or more reference points between the internal image data set and the visual image, the step of performing a surface to surface correlation between the internal image data set and the visual image, and the step of digitizing points on the body with a pointing device in the visual image and correlating the digitized points with one or more corresponding reference locations in the internal image data set.

13. The method of claim 11, wherein the step of obtaining an image data set comprises the step of attaching a reference unit comprising the reference location to the body, wherein the reference unit comprises an optical pattern and at least one radiopaque marker, wherein the radiopaque marker is visible in the internal image data set and the optical pattern is visible in the visual image, wherein the radiopaque marker is in a known unique relation to the optical pattern, and wherein the step of registering comprises correlating positions of the radiopaque marker in the internal image data set with the optical pattern in the visual image.

14. The method of claim 13, wherein the step of updating comprises the steps of tracking movements of the reference unit relative to the camera using at most one reference unit.

15. The method of claim 11, further comprising the step of displaying virtual information relevant to a planned location of interest in a surgical procedure on the internal image data set along with the image data set on the visual display.

16. The method of claim 15, wherein the step of displaying virtual information further comprises displaying a planned trajectory of an instrument to be inserted into the body in registration with the internal image data set and the visual image.

17. The method of claim 11, wherein the image data set comprises image data regarding one or more internal structures inside the body, and wherein the step of displaying the internal image data set includes the step of selecting one or more of the internal structures to display.

18. The method of claim 15, further comprising the step of tracking an instrument in a field of view of the camera with known geometry that is equipped with at least a second reference unit.

19. The method of claim 18, further comprising the step of superimposing a representation of a feature of the instrument on the monitor in registration with the internal image data and the virtual information.

20. The method of claim 11, wherein the step of registering the internal image data set to the visual image further comprises the steps of:

affixing at least one reference unit to the exterior surface of the body at the reference location, wherein the reference unit has one or more features that create a unique image signal in the image data set and a unique representation in the visual image;

acquiring the image data set that includes the unique image signal of the reference unit;

identifying the position of the reference unit with respect to a coordinate system of the internal image data set;

acquiring the visual image of the exterior surface of the body that includes the unique representation of the reference unit;

identifying the position of the reference unit in the visual image; and

matching the spatial position of the reference unit in the internal image data set with the spatial position of the reference unit in the visual image to automatically register the internal image data set with the visual image of the body.

21. The method of claim 20, further comprising the step of affixing at least three of the reference units to the body, determining the orientation and size of the reference units based on a different unique optical pattern disposed on each of the reference units, and registering the orientation of the reference units with a coordinate system of the body.

22. The method of claim 20, further comprising the step of utilizing a navigation system that includes a monitor, a camera fixed to the monitor, and an electronic processor, wherein the navigation system performs the step of matching the spatial position.

23. The method of claim 20, wherein the step of identifying the position of the reference unit in the visual image includes the step of identifying the center of the reference unit with the electronic processor, and the step of matching includes the step of matching the spatial position of the reference unit in the internal image data set with the center of the reference unit in the visual image with the electronic processor.

24. The method of claim 20, wherein the features of the reference unit include one or more reference markers that are detectable in the image data set to create the unique image signal in the image data set and an optical pattern of alternating light and dark contrast areas that are detectable in the visual image to create the unique representation.

25. The method of claim 24, wherein the reference unit includes three or more radiopaque reference markers, and wherein a single reference unit is affixed to the body to register the image data set with the visual image and to track movement of the body.