System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities

• US 7,728,868 B2
• Filed: 08/02/2007
• Issued: 06/01/2010
• Est. Priority Date: 08/02/2006
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: Keller, Kurtis P., State, Andrei, Green, Caroline K., Razzaque, Sharif A., Ackerman, Jeremy D.
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US11/833,134
• Publication Number:
  US 20080030578A1
• Time in Patent Office: 1,034 Days
• Field of Search: None
• US Class Current: 348/77
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: G06T 3/16 Spatio-temporal transformations, e.g. video cubism
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method of providing composite real-time dynamic imagery of a medical procedure site from multiple modalities which continuously and immediately depicts the current state and condition of the medical procedure site synchronously with respect to each modality and without undue latency is disclosed. The composite real-time dynamic imagery may be provided by spatially registering multiple real-time dynamic video streams from the multiple modalities to each other. Spatially registering the multiple real-time dynamic video streams to each other may provide a continuous and immediate depiction of the medical procedure site with an unobstructed and detailed view of a region of interest at the medical procedure site at multiple depths. As such, a surgeon, or other medical practitioner, may view a single, accurate, and current composite real-time dynamic imagery of a region of interest at the medical procedure site as he/she performs a medical procedure, and thereby, may properly and effectively implement the medical procedure.

Claims

1. A system for producing medical image guidance, comprising:

a first device with which a first set of dynamic, real-time visual medical imaging data is related, wherein the first device'"'"'s first spatial state is trackable using a spatial state tracking device;

a second device with which a second set of dynamic, real-time medical imaging data is related, wherein the second device'"'"'s second spatial state is trackable using the spatial state tracking device, wherein the second device is separate from and distinct from the first device; and

an image guidance controller configured to;

receive data related to the first spatial state of the first device;

receive data related to the second spatial state of the second device;

receive the first set of dynamic, real-time visual medical imaging data;

receive the second set of dynamic, real-time medical imaging data;

determine a relative spatial state of the first device and the second device based on the first device'"'"'s first spatial state and the second device'"'"'s second spatial state;

produce combined dynamic, real-time image guidance data based on the first set of dynamic, real-time visual medical imaging data;

the second set of dynamic, real-time medical imaging data; and

the relative spatial state of the first device and the second device;

wherein the first spatial state, the first set of dynamic, real-time visual medical imaging data, the second spatial state, and the second set of dynamic, real-time medical imaging data are each updated continually and the image guidance controller is configured to produce dynamic, real-time image guidance data continually.

2. The system of claim 1, wherein the dynamic, real-time image guidance data comprises a two-dimensional representation of the first set of dynamic, real-time visual medical imaging data and the second set of dynamic, real-time medical imaging data, combined based on the relative registration of one to the other in three-dimensional space based on the first and second spatial states.

3. The system of claim 1, wherein the continual updating and continual producing comprise updating and producing at least ten times per second.

4. The system of claim 1, wherein first set of dynamic, real-time visual medical imaging data comprises the ultrasound data captured by an ultrasound wand.

5. The system of claim 1, wherein second set of imaging data comprises a three-dimensional model related to the second device.

6. A system for producing medical image guidance, comprising:

a first device with which a first set of dynamic, real-time visual medical imaging data is related, wherein the first device'"'"'s first spatial state is tracked;

a second device with which a second set of dynamic, real-time medical imaging data is related, wherein the second device'"'"'s second spatial state is tracked, wherein the first device is separate from and distinct from the second device; and

an image guidance controller configured to;

determine the first device'"'"'s first spatial state;

determine the second device'"'"'s second spatial state;

determine the relative spatial states of the first device and the second device based on the first spatial state of the first device and the second spatial state of the second device; and

produce dynamic, real-time image guidance data based on;

the first set of dynamic, real-time visual medical imaging data;

the second set of dynamic, real-time medical imaging data; and

the relative spatial states of the first device and the second device,

wherein the first spatial state, the first set of dynamic, real-time visual medical imaging data, the second spatial state, and the second set of dynamic, real-time medical imaging data are each updated continually and the image guidance controller is configured to produce dynamic, real-time image guidance data continually.

7. The system of claim 6, wherein the continual updating and continual producing comprise updating and producing at least ten times per second.

8. The system of claim 6, wherein first set of dynamic, real-time visual medical imaging data comprises the ultrasound data captured by an ultrasound wand.

9. The system of claim 6, wherein second set of imaging data comprises a three-dimensional model related to the second device.

10. The system of claim 6, wherein determining the first spatial state comprises receiving updates from spatial measurement equipment that is tracking the first device.

11. The system of claim 10, wherein the spatial measurement equipment comprises a tracker selected from the group consisting of an optical tracker and a magnetic tracker.

12. The system of claim 6, wherein the dynamic, real-time image guidance data comprises a two-dimensional representation of the first set of dynamic, real-time visual medical imaging data and the second set of dynamic, real-time medical imaging data, combined based on the relative registration of one to the other in three-dimensional space based on the first and second spatial states.

13. The system of claim 6, wherein first device comprises an ultrasound wand.

14. The system of claim 6, wherein second device comprises an ablation needle.

15. A system for producing medical image guidance, comprising:

an input module configured to receive as input spatial states and data related to two or more tracked devices, wherein the two or more tracked devices comprise at least a first tracked device and a second tracked device the first tracked device is separate and distinct from the second tracked device, the input comprises at least a first spatial state of the first tracked device, a first set of dynamic, real-time visual medical imaging data related to the first tracked device, a second spatial state of the second tracked device, and a second set of dynamic, real-time medical imaging data related to the second tracked device, wherein the input module is configured to receive continually-updated input for the two or more tracked devices and the first spatial state, second spatial state, first set of dynamic, real-time visual medical imaging data, and second set of dynamic, real-time medical imaging data are continually updated;

an image guidance controller configured to process the input for the two or more tracked devices in order to produce combined dynamic, real-time displayable data, the processing comprising at least spatially registering the input for the first tracked device and the input for the second tracked device based on the first spatial state and the second spatial state and composing the first set of dynamic, real-time visual medical imaging data and the second set of dynamic, real-time medical imaging data based on the spatial registration; and

an output module configured to transmit the combined dynamic, real-time displayable data to a display unit.

16. The system of claim 15, wherein the image guidance controller is configured to produce combined dynamic, real-time displayable data for each received, continually-updated input for the two or more tracked devices and the output module is configured to transmit the combined dynamic, real-time displayable data for each continually-updated input for the two or more tracked devices.

17. The system of claim 15, wherein the continually-updated input is updated at least ten times per second.

18. The system of claim 15, wherein the first set of dynamic, real-time visual medical imaging data comprises the ultrasound data captured by an ultrasound wand.

19. The system of claim 15, wherein second set of dynamic, real-time medical imaging data related comprises a three-dimensional model related to the second device.

20. The system of claim 15, wherein the first spatial state information is sent from spatial measurement equipment that is tracking the first device.

21. The system of claim 15, wherein the dynamic, real-time displayable data comprises a two-dimensional representation of the first set of dynamic, real-time visual medical imaging data and the second set of dynamic, real-time medical imaging data, combined based on the relative registration of one to the other in three-dimensional space based on the first and second spatial states.

22. The system of claim 15, wherein the first device comprises an ultrasound wand.

23. The system of claim 15, wherein the second device comprises an ablation needle.

24. A method for producing medical image guidance, comprising:

receiving at an imaging system electronic tracking data comprising a first spatial state of a first tracked device;

receiving at the imaging system electronic tracking data comprising a second spatial state of a second tracked device, wherein the second tracked device is separate and distinct from the first tracked device;

receiving at the imaging system a first set of dynamic, real-time visual medical imaging data related to the first tracked device;

receiving at the imaging system a second set of dynamic, real-time medical imaging data related to the second tracked device;

combining the first set of dynamic, real-time visual medical imaging data and the second set of dynamic, real-time medical imaging data into combined dynamic, real-time displayable data at the imaging system based on the first spatial state and second spatial state; and

causing the display of the combined dynamic, real-time displayable data.

25. The method of claim 24, wherein the receiving electronic tracking data comprises continually receiving the tracking data.

26. The method of claim 24, wherein the image guidance wherein the dynamic, real-time displayable data comprises a two-dimensional representation of the first set of dynamic, real-time visual medical imaging data and the second set of dynamic, real-time medical imaging data, combined based on the relative registration of one to the other in three-dimensional space based on the first and second spatial states.

27. The method of claim 24, wherein the first set of dynamic, real-time visual medical imaging data comprises ultrasound data captured by an ultrasound wand.

28. The method of claim 24, wherein second set of dynamic, real-time medical imaging data related comprises a three-dimensional model related to the second device.

29. The method of claim 24, wherein the first device comprises an ultrasound wand and the second device comprises an ablation needle.