System and method of surgical imagining with anatomical overlay for navigation of surgical devices

• US 7,831,294 B2
• Filed: 10/07/2004
• Issued: 11/09/2010
• Est. Priority Date: 10/07/2004
• Status: Active Grant

• Current Assignee: Stereotaxis, Inc.
• Original Assignee: Stereotaxis, Inc.
• Inventors: Viswanathan, Raju R.
• Primary Examiner(s): Casler; Brian
• Assistant Examiner(s): LAMPRECHT, JOEL
• Application Number:
  US10/962,174
• Publication Number:
  US 20060079745A1
• Time in Patent Office: 2,224 Days
• Field of Search: 600424-429, 600/407, 600/300, 378/20, 378/21, 378/29, 378/143, 378/144
• US Class Current: 600/425
• CPC Class Codes: A61B 2017/00243 cardiac
• CPC Class Codes: A61B 2017/00703 correcting for movement of heart, e.g. ECG-triggered
• CPC Class Codes: A61B 2034/101 Computer-aided simulation of surgical operations
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/547 involving tracking of position of the device or parts of the device
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for

Abstract

A system and method are provided for control of a navigation system for deploying a medical device within a subject, and for enhancement of a display image of anatomical features for viewing the projected location and movement of medical devices, and projected locations of a variety of anatomical features and other spatial markers in the operating region. The display of the X-ray imaging system information is augmented in a manner such that a physician can more easily become oriented in three dimensions with the use of a single-plane X-ray display. The projection of points and geometrical shapes within the subject body onto a known imaging plane can be obtained using associated imaging parameters and projective geometry.

Claims

1. A medical imaging system for imaging an operating region in a subject, the system comprising an imaging beam source, an imaging plate, and a display for displaying an image generated by the imaging beam on the imaging plate together with an overlay of at least one object in the operating region, wherein the system is configured to generate an overlay from the position of the object, the position of the imaging beam source, and the position and orientation of the imaging plate, wherein the location of where an overlay of the projected object is to appear on the display is determined in part based on the location of the imaging beam source and the location of the imaging plate, such that for any object in three-dimensional space a corresponding projected object may be suitably overlayed on the display.

2. The medical imaging system according to claim 1 wherein the object is at least one point in the operating region.

3. The medical imaging system according to claim 1 wherein the object is a loop comprising a plurality of points in the operating region.

4. The medical imaging system according to claim 1 wherein the object is a medical device in the operating region.

5. The medical imaging system according to claim 1 wherein the position of the medical device in the operating region is determined by a localization system.

6. The medical imaging system according to claim 1 wherein the position of the medical device is calculated using a mathematical model of the medical device.

7. The medical imaging system according to claim 1 wherein the imaging beam source and the imaging plate are movable about the operating region, and wherein the display updates both the displayed image and the overlay as the positions of the imaging beam and imaging plane change.

8. A method of providing medical images of an operating region in a subject'"'"'s body, the method comprising:

generating an image of the operating region using an imaging beam source and an imaging plate; and

displaying the generated image together with an overlay of at least one object in the operating region, the overlay generated from the position of the object, the position of the imaging beam source, and the position and orientation of the imaging plate, wherein the location of where an overlay of the projected object is to appear on the display is determined in part based on the location of the imaging beam source and the location of the imaging plate, such that for any object in three-dimensional space a corresponding projected object may be suitably overlayed on the display.

9. The medical imaging system according to claim 8 wherein the object is at least one point in the operating region.

10. The medical imaging system according to claim 8 wherein the object is a loop comprising a plurality of points in the operating region.

11. The medical imaging system according to claim 8 wherein the object is a medical device in the operating region.

12. The medical imaging system according to claim 8 wherein the position of the medical device in the operating region is determined by a localization system.

13. The medical imaging system according to claim 8 wherein the position of the medical device is calculated using a mathematical model of the medical device.

14. The method according to claim 8 wherein the imaging beam source and the imaging plate are movable about the operating region, and further comprising updating the both the displayed image and the overlay as the positions of the imaging beam and imaging plane change.

15. A medical imaging and display system for displaying medical images of an operating region in a subject, the system comprising:

a localization system for determining the location of an object in the operating region;

an imaging system having an imaging beam source, an imaging plate, and a display for displaying an image of the operating region generated by the imaging beam source and the imaging plate, together with an overlay of at least one object in the operating region, wherein the system is configured to generate an overlay from the position of the object, the position of the imaging beam source, and the position and orientation of the imaging plate, wherein the location of where an overlay of the projected object is to appear on the display is determined in part based on the location of the imaging beam source and the location of the imaging plate, such that for any object in three-dimensional space a corresponding projected object may be suitably overlayed on the display.

16. The medical imaging and display system according to claim 15 wherein the imaging system communicates the position of the imaging beam source and the imaging plate to the localization system, and the localization system calculates the position of the object on the display.

17. The medical imaging and display system according to claim 15 wherein the localization system communicates the position of the object in the operating region to the imaging system, and the imaging system calculates the position of the object on the display.

18. The medical imaging system according to claim 15 wherein the object is at least one point in the operating region.

19. The medical imaging system according to claim 15 wherein the object is a loop comprising a plurality of points in the operating region.

20. The medical imaging system according to claim 15 wherein the object is a medical device in the operating region.

21. The medical imaging system according to claim 15 wherein the position of the medical device in the operating region is determined by a localization system.

22. The medical imaging system according to claim 15 wherein the position of the medical device is calculated using a mathematical model of the medical device.

23. A method of displaying medical images of an operating region in a subject, the method comprising the steps of:

generating an image of the operating region using an imaging beam source and an imaging plate;

displaying the generated image;

determining the location relative to the generated image of an object in the operating region based upon the position of the object in the operating region, the location of the imaging beam source, and the location of the imaging beam plate;

overlaying a representation of the object on the display to position an overlay of at least one object in the operating region;

wherein the location of where an overlay of the projected object is to appear on the display is determined in part based on the location of the imaging beam source and the location of the imaging plate, such that for any object in three-dimensional space a corresponding projected object may be suitably overlayed on the display.

29. The system of claim 23, wherein the current location and orientation of the medical device is continuously updated and graphically overlaid onto the anatomical image shown on the live imaging display.

32. The system of claim 29, wherein the current location and orientation of the medical device is graphically displayed together with other geometrical information in the live imaging display.

33. The system of claim 29, wherein the geometrical information transferred to and displayed on the live imaging display is obtained from a three dimensional pre-operative image dataset.

34. The system of claim 29, wherein the current location and orientation of the medical device is graphically displayed in the live imaging display together with data from a three dimensional intra-operative surface image rendering.

24. A system for navigating a medical device in a body, having a display means for displaying an image of an operating region in the body, the navigation system comprising:

a means of remotely actuating the medical device for navigational purposes;

a connection to an anatomical imaging system, wherein image information can be transferred to the navigation system for viewing on the display means;

a means for transfer of imaging parameters including the location of an imaging beam source and an imaging plate associated with the imaging system to the navigation system;

a means of transformation of three dimensional geometrical information corresponding to a location in space, based on the imaging parameters including the location of an imaging beam source and an imaging plate; and

a means of transferring the suitably transformed geometrical information to the imaging system for display on its live imaging monitor, wherein the transformation of the three-dimensional information corresponding to a location in space is determined in part based on the location of the imaging beam source and the location of the imaging plate, such that for any three-dimensional location in space a corresponding projected location within an image being displayed may be determined.

25. The system of claim 24, where the imaging parameters include positions of mechanisms of an X-ray C-arm system.

26. The system of claim 24, where the geometrical information is the real-time location and orientation of a localized medical device.

27. The system of claim 24, further comprising a graphic overlay of at least one target location that is suitably projected onto the anatomical image shown on the live imaging display.

28. The system of claim 24, further comprising a graphic overlay of at least one reference location that is suitably projected onto the anatomical image shown on the live imaging display.

30. The system of claim 24, further comprising a mathematical model of a predicted configuration of the medical device that is suitably projected onto the live imaging display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a desired change in control variables.

31. The system of claim 24, further comprising a mathematical model of a predicted configuration of the medical device that is suitably projected onto the live imaging display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a change in control variables.

35. The system of claim 24, further comprising data input from a source of anatomical surface normal information, wherein the surface normal information is transformed to the frame of reference of the displayed live image, and is graphically overlaid onto the anatomical image shown on the live imaging display.

36. The system of claim 24, wherein the surface normal information is transformed using an orthographic projection matrix incorporating a source to image distance variable.

37. The method of claim 24, where the geometrical information comprises anatomical landmarks.

38. The method of claim 24, where the geometrical information comprises anatomical curvilinear features.

39. The method of claim 24, where the geometrical information comprises anatomical surface portions.

40. The method of claim 24, where the geometrical information comprises reconstructed vascular path data.

41. An improved system for remotely navigating a medical device in a body, having a display device for displaying an image of the medical device in the operating region, the improvement comprising:

a means of remote actuation of the medical device;

a means of controlling the remote actuation of the medical device from the navigation system;

transfer of real-time location data to the navigation system from a localization system for determining the position of the medical device;

a connection to an X-ray imaging system, wherein imaging parameters including the location of an imaging beam source and an imaging plate can be transferred to the navigation system from the imaging system, and said navigation system is configured to transform said real-time location data for the position of the medical device into a projection point, determined in part based on the location of the imaging beam source and the location of the imaging plate; and

wherein geometrical information relating to said projection point can be transferred from the navigation system to the imaging system so that for any real-time location of the medical device a corresponding projection point may be suitably located on the live imaging display device, such that a continuously updated current location and orientation of the medical device as determined by the localization system are suitably projected and graphically overlaid onto the live X-ray image.

42. The system of claim 41, wherein a graphic overlay of at least one target location is suitably projected and overlaid onto the live X-ray image display.

43. The system of claim 41, wherein a graphic overlay of at least one reference location is suitably projected and overlaid onto the live X-ray image display.

44. The system of claim 41, wherein at least one user-defined device marker is projected and graphically overlaid onto the live X-ray image display.

45. The system of claim 41, further comprising a mathematical model of the configuration of the medical device that is suitably projected onto the live X-ray image display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a desired change in control variables.

46. The system of claim 41, further comprising a mathematical model of the configuration of the medical device that is suitably projected onto the live X-ray image display, wherein the mathematical model provides a visual reference of a predicted location and orientation of the medical device that corresponds to a change in control variables.

47. The system of claim 41, wherein the current location and orientation of the medical device is graphically displayed on the live X-ray image display together with data obtained from a three dimensional pre-operative image dataset.

48. The system of claim 41, wherein the current location and orientation of the medical device is graphically displayed on the live X-ray image display together with data obtained from a three dimensional intra-operative surface image dataset.

49. The system of claim 41, further comprising data input from a source of anatomical surface normal information, wherein the surface normal information is graphically overlaid onto the live X-ray image display.

50. The system of claim 49, wherein the surface normal information is transformed using an orthographic projection matrix incorporating a source to image distance variable.

51. The method of claim 41, where the geometrical information comprises anatomical landmarks.

52. The method of claim 41, where the geometrical information comprises anatomical curvilinear features.

53. A system for remotely navigating a medical device in a body with a remote navigation system having a display for displaying an image of operating region, the system comprising:

a means of controlling the remote actuation of the medical device from the navigation system;

a connection to an X-ray imaging system, wherein X-ray imaging parameters including the location of a X-ray beam source and an X-ray plate associated with the X-ray imaging system can be transferred to the navigation system from the imaging system;

a means of transforming three dimensional geometrical information relating to a known point into a projection point for appropriate projection and overlay, wherein the transformation of the three-dimensional geometrical information is determined in part using the X-ray imaging parameters of the location of the imaging beam source and the location of the imaging plate; and

wherein the system is configured for displaying the transformed geometrical information on the live X-ray imaging display.

54. The system of claim 53, where the geometrical information comprises anatomical landmarks.

55. The system of claim 53, where the geometrical information comprises reference positions.

56. The system of claim 53, where the geometrical information comprises target locations.

57. The system of claim 53, where the geometrical information comprises predicted device configurations.

58. The system of claim 53, where the geometrical information comprises real-time device location and orientation data.

59. The system of claim 53, where the geometrical information comprises anatomical curvilinear features.

60. The system of claim 53, where the geometrical information comprises anatomical surface portions.

61. The system of claim 53, where the geometrical information comprises identified device markers.

62. The system of claim 53, where the geometrical information comprises surface normal data.

63. The system of claim 53, where the geometrical information comprises geometrical data obtained from three dimensional image data.

64. The system of claim 53, where the geometrical information comprises reconstructed vascular path data.

71. The system of claim 53 wherein the location of where the projected object is to appear as an overlay on the display is determined based on the distance h between the location of the imaging plate and the object in three-dimensional space, and the distance d between the imaging plate and the imaging beam source in three-dimensional space.

65. A method for overlaying geometrical information onto an X-ray imaging display for use in remotely navigating a medical device in a body, the method comprising the steps of:

connecting a navigation system to an X-ray imaging system, wherein X-ray imaging parameters including the location of a X-ray beam source and an X-ray plate associated with the X-ray imaging system can be transferred to the navigation system from the imaging system;

transforming three dimensional geometrical information relating to a known point into a projection point for appropriate projection and overlay, wherein the transformation of the three-dimensional geometrical information is determined in part using the X-ray imaging parameters of the location of the imaging beam source and the location of the imaging plate;

displaying the transformed geometrical information relating to a protection point on the live X-ray imaging display; and

controlling the remote actuation of the medical device from the navigation system using the geometrical information overlaid on the X-ray imaging display.

66. The method of claim 65, further comprising the step of providing predicted device configurations for use in controlling the actuation of the medical device.

67. The method of claim 65, wherein the geometrical information comprises anatomical landmarks.

68. The method of claim 65, wherein the geometrical information comprises reference positions.

69. The method of claim 65, wherein the geometrical information comprises target locations.

70. The method of claim 65, wherein the geometrical information comprises real-time device location and orientation data.

72. The method of claim 65 wherein the transforming of three dimensional geometrical information relating to a known point into a projection point for overlay on a display is determined based on the distance h between the location of the imaging plate and the object in three-dimensional space, and the distance d between the imaging plate and the imaging beam source in three-dimensional space.