Method and apparatus for medical image display for surgical tool planning and navigation in clinical environments

• US 6,614,453 B1
• Filed: 05/05/2000
• Issued: 09/02/2003
• Est. Priority Date: 05/05/2000
• Status: Expired due to Term

• Current Assignee: Marconi Medical Systems, Inc. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Xu, Yansun, Suri, Jasjit S., Quddus, Ruhul
• Primary Examiner(s): Bayerl, Raymond J.
• Application Number:
  US09/565,253
• Time in Patent Office: 1,215 Days
• Field of Search: 345/764, 345/424, 345/419, 345/427, 345/619, 345/643, 345/771, 345/773, 345/848, 345/849, 345/852, 345/961, 600/407, 600/410-411, 600/416, 600/425, 600/427, 709/315-316, 709/313, 709/330-332
• US Class Current: 715/764
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: Y10S 715/961 Operator interface with visual structure or function dictated by intended use

Abstract

A medical imaging display system includes a memory (40) for storing first image data representative of a region of interest. The memory (40) stores image data generated by medical imaging devices such as magnetic resonance devices (20), computed tomography devices (22), nuclear imaging devices (26,28,30), and ultrasound devices. Typically, image data from these devices is obtained some time prior to a surgical event and users may access this data in planning for the surgical event. A processor (42), in data communication with the memory, is organized under a component object modeling architecture. The processor (42) is connected to a user interface (10) for providing user requests to the processor. Thus, in response to user action via the user interface (10), the processor (42) determines an object (54) adapted to act on the request, selects a handle (60) for the determined object and, employs the object via the handle to act on the request. Additionally, the image guided surgical system also includes a source (48) of substantially real time image data generated in the surgical theater, such as spectroscopy devices, which can also be manipulated via software objects to display desired portions of the region of interest. Any of these images can be viewed on a display (46) in a planning environment, in the surgical suite, or even by a consultant, geographically remote from the surgical site.

Claims

1. A method of displaying medical images, the method including:

responsive to receipt of a request for manipulation of image data representative of a region of interest, determining a component object within a component object model architecture, which component object is configured to at least partially accomplish the request;

selecting a handle for an interface to the determined component object; and

processing the request via the handle.

2. The method as set forth in claim 1, further including displaying the processed image data.

3. The method as set forth in claim 1, further including:

receiving first image data from at least one medical imaging device selected from the set of a magnetic resonance device, a computed tomography device, a nuclear imaging device and an ultrasound device; and

storing the first image data in a memory.

4. The method as set forth in claim 3, wherein the method further includes planning a surgical event by performing tasks including manipulating a user interface to selectively control image data displayed.

5. The method as set forth in claim 3, further including:

receiving second image data; and

storing the second image data in a memory.

6. The method as set forth in claim 5, wherein the method further includes performing a surgical event by performing tasks including manipulating a user interface to selectively control display of first image data and second image data.

7. The method as set forth in claim 1, wherein the determining a process step includes:

referencing a registry to detect registration of a process adapted to accomplish the request; and

noting the handle associated with the process.

8. The method as set forth in claim 1, wherein the component object is determined from a plurality of component objects within the component object model architecture, the processing of the request including:

selecting a second handle for an interface to a second component object; and

formatting a second request directed to the second component object using the second handle to perform at least a portion of the processing.

9. A diagnostic imaging system comprising:

an image memory system for storing a plurality of volume digital image representations reconstructed from diagnostic data generated by a plurality diagnostic imagers;

a user interface through which a user selects among image display formats;

a component object modeling (COM) architecture layer which includes image processing components for retrieving portions of the digital image representations and processing the retrieved portions into the selected display format.

10. The diagnostic imaging system as set forth in claim 9 wherein:

the user interface includes a component ID registry table which correlates user selected display formats with IDs of selected components for retrieving and processing the portions of the image representations.

11. The diagnostic imaging system as set forth in claim 10 wherein:

at least some of the components identify other components which they access and cooperate with to process the portion of the image representations.

12. An image guided surgical system for use by a surgeon in performing an image-assisted surgical procedure on a surgical area of a subject, the surgical system including:

a real time surgical imaging system operated by the surgeon to acquire real-time images of the surgical area;

a database of diagnostic images of the surgical area acquired prior to the surgical procedure, the diagnostic images including images in a format different from that of the real-time images;

a component object model architecture software layer for performing image processing, the component object model architecture software layer containing a plurality of component objects, each component object being configured to perform a selected image processing task and having at least one interface accessible by a handle; and

a graphical user interface that receives the real-time images and performs image display processing including at least registering a real-time image with at least one diagnostic image, the graphical user interface delegating at least some of the image display processing to one or more selected component objects of the component object model architecture software layer by interfacing with the selected component objects via handles of the selected component objects.

13. The surgical system as set forth in claim 12, wherein the database of diagnostic images includes diagnostic images acquired using at least two different imaging modalities.

14. The surgical system as set forth in claim 12, wherein the database of diagnostic images includes diagnostic images acquired using at least two different imaging modalities selected from a group consisting of:

magnetic resonance (MR) imaging, computed tomography (CT) imaging, x-ray imaging, fluoroscopic imaging, positron emission tomography (PET) imaging, single photon emission computed tomography (SPECT) imaging, nuclear imaging, and ultrasound imaging.

15. The surgical system as set forth in claim 12, wherein the surgical probe includes an endoscope.

16. The surgical system as set forth in claim 12, wherein the component objects of the component object model architecture software layer include a component object selected from a group consisting of:

a 3D rendering rotation component object, an image registration component object, an image overlay component object, an image view selection component object, and an image view display arrangement component object.

17. The surgical system as set forth in claim 12, wherein the component objects of the component object model architecture software layer include:

an aggregation component object defined by aggregation of at least two other component objects.

18. The surgical system as set forth in claim 12, wherein the component objects of the component object model architecture software layer include:

a containment component object that contains at least one other component object.

19. A image-guided surgical method including:

acquiring a real-time surgical image during a surgical procedure;

selecting a display format that includes displaying the real-time surgical image and at least one stored diagnostic image;

interfacing with a component object modeling architecture software layer using handles to selected component objects of said software layer to compute display data; and

generating a display based on the display data.

20. The surgical method set forth in claim 19, wherein the interfacing includes:

aggregating a plurality of component objects to define an aggregate component object; and

interfacing with the aggregate component object using a handle of the aggregate component object.

21. The surgical method set forth in claim 19, further including:

allocating at least some of the computing of the display data to a component object contained by one of the selected component objects.

22. An image guided surgical system for use in performing an image-assisted surgical procedure, the surgical system including:

a real time imager that provides real-time images in a first format of a surgical area;

a database of diagnostic images of the surgical area acquired prior to the surgical procedure using one of at least two selectable imaging modalities, the diagnostic images including images in a second format different from the first format of the real-time images;

a component object model architecture software layer for performing image processing, the component object model architecture software layer including a plurality of component objects, each component object being configured to perform a selected image processing task and having at least one interface accessible by a handle, the component object model architecture supporting at least component aggregation and component containment; and

a graphical user interface that receives the real-time images and performs image display processing including at least combining a real-time image with at least one diagnostic image, the graphical user interface delegating at least some of the image display processing to one or more selected component objects of the component object model architecture software layer by interfacing with the selected component objects via handles of the selected component objects.

23. The image guided surgical system as set forth in claim 22, wherein the one or more selected component objects include:

an image processing component configured to process diagnostic images from any of the at least two selectable imaging modalities, the image processing component object including a first imaging modality component object configured to process images from a first of the at least two selectable imaging modalities and a second imaging modality component object configured to process images from a second of the at least two selectable imaging modalities, the first and second imaging modality component objects combining to contribute to the image processing component object by one of aggregation and containment.