System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities

• US 8,350,902 B2
• Filed: 12/21/2011
• Issued: 01/08/2013
• Est. Priority Date: 08/02/2006
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: Razzaque, Sharif A., Keller, Kurtis P., State, Andrei, Green, Caroline K., Ackerman, Jeremy D.
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US13/333,878
• Publication Number:
  US 20120108955A1
• Time in Patent Office: 384 Days
• Field of Search: None
• US Class Current: 348/77
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: G06T 3/16 Spatio-temporal transformations, e.g. video cubism
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method of providing composite real-time dynamic imagery of a medical procedure site from multiple modalities which continuously and immediately depicts the current state and condition of the medical procedure site synchronously with respect to each modality and without undue latency is disclosed. The composite real-time dynamic imagery may be provided by spatially registering multiple real-time dynamic video streams from the multiple modalities to each other. Spatially registering the multiple real-time dynamic video streams to each other may provide a continuous and immediate depiction of the medical procedure site with an unobstructed and detailed view of a region of interest at the medical procedure site at multiple depths. A user may thereby view a single, accurate, and current composite real-time dynamic imagery of a region of interest at the medical procedure site as the user performs a medical procedure.

Claims

1. A system for producing medical image guidance data, comprising:

a first device configured to produce a first set of real-time visual medical imaging data, the first device configured to continually update the first set of real-time medical imaging data;

a first spatial state tracking device configured to track the first device'"'"'s spatial state, the first spatial state tracking device configured to continually update data related to the first spatial state of the first device;

a second device associated with a second set of real-time medical imaging data, the second device movable relative to the first device, the second device configured to continually update the second set of real-time medical imaging data;

a second spatial state tracking device configured to track the second device'"'"'s spatial state, the second spatial state tracking device configured to continually update data related to the second spatial state of the second device; and

an image guidance controller configured to;

receive the data related to the first spatial state of the first device;

receive the data related to the second spatial state of the second device;

receive the first set of real-time visual medical imaging data;

receive the second set of real-time visual medical imaging data;

determine a relative spatial state of the first device and the second device based on the data related to the first spatial state of the first device and the data related to the second spatial state of the second device;

produce combined image guidance data based on the first set of real-time visual medical imaging data, the second set of real-time medical imaging data, and the relative spatial state of the first device and the second device,

wherein the image guidance controller is configured to produce image guidance data continually.

2. The system of claim 1, wherein the first spatial tracking device and the second spatial tracking device are the same spatial tracking device.

3. The system of claim 1, wherein the image guidance data comprises a two-dimensional representation of the first set of real-time visual medical imaging data and the second set of real-time medical imaging data, combined based on a relative registration of the first set of real-time visual medical imaging data to the second set of real-time medical imaging data in three-dimensional space, the relative registration based on the data related to the first spatial state and the data related to the second spatial state.

4. The system of claim 1, wherein continually updating comprises updating at least ten times per second.

5. The system of claim 1, wherein the first set of real-time visual medical imaging data comprises an ultrasound data captured by an ultrasound wand.

6. The system of claim 1, wherein the second set of real-time medical imaging data comprises a three-dimensional model of at least a portion of the second device.

7. A system for producing medical image guidance data, comprising:

a first device with which a first set of real-time visual medical imaging data is related, the first device configured to be tracked, the first device configured to continually update the first set of real-time visual medical imaging data;

a second device with which a second set of real-time medical imaging data is related, the second device configured to be tracked, the second device configured to continually update the second set of real-time medical imaging data; and

an image guidance controller configured to;

determine the first device'"'"'s first spatial state;

determine the second device'"'"'s second spatial state;

determine the relative spatial states of the first device and the second device based on the first spatial state of the first device and the second spatial state of the second device; and

produce image guidance data based on;

the first set of real-time visual medical imaging data;

the second set of real-time medical imaging data; and

the relative spatial states of the first device and the second device,

wherein the image guidance controller is configured to produce image guidance data continually.

8. The system of claim 7, wherein continually updating comprises updating at least ten times per second.

9. The system of claim 7, wherein the first set of real-time visual medical imaging data comprises an ultrasound data captured by an ultrasound wand.

10. The system of claim 7, wherein the second set of real-time imaging data comprises a three-dimensional model of at least a portion of the second device.

11. The system of claim 7, wherein to be configured to determine the first spatial state comprises being configured to receive updates from spatial measurement equipment that is tracking the first device.

12. The system of claim 11, wherein the spatial measurement equipment comprises a tracker selected from the group consisting of an optical tracker and a magnetic tracker.

13. The system of claim 7, wherein the image guidance data comprises a two-dimensional representation of the first set of real-time visual medical imaging data and the second set of real-time medical imaging data, combined based on a relative registration of one to the other in three-dimensional space based on the first and second spatial states.

14. The system of claim 7, wherein first device comprises an ultrasound wand.

15. The system of claim 7, wherein second device comprises an ablation needle.

16. A system for producing medical image guidance data, comprising:

an input module configured to receive at least;

a continually updated first spatial state data of a first tracked device;

a continually updated second spatial state data of a second tracked device;

a continually updated first set of visual medical imaging data generated by the first tracked device; and

a continually updated second set of medical imaging data generated by the second tracked device;

an image guidance controller configured to;

perform a spatial registration of the first set of visual medical imaging data with the second set of medical imaging data based on the continually updated first spatial state data and the continually updated second spatial state data; and

compose the first set of visual medical imaging data and the second set of medical imaging data based on the spatial registration; and

an output module configured to transmit the combined displayable data to a display unit.

17. The system of claim 16, wherein the continually-updated input is updated at least ten times per second.

18. The system of claim 16, wherein the first set of visual medical imaging data comprises an ultrasound data captured by an ultrasound wand.

19. The system of claim 16, wherein the second set of medical imaging data comprises a three-dimensional model of at least a portion of the second device.

20. The system of claim 16, wherein the first spatial state information is sent from spatial measurement equipment configured to track the first device.

21. The system of claim 16, wherein the real-time displayable data comprises a two-dimensional representation of the first set of visual medical imaging data and the second set of medical imaging data, combined based on the spatial registration.

22. The system of claim 16, wherein the first device comprises an ultrasound wand.

23. The system of claim 16, wherein the second device comprises an ablation needle.

24. A method for producing medical image guidance, comprising:

receiving, at an imaging system, electronic tracking data comprising a first spatial state of a first tracked device;

receiving, at the imaging system, electronic tracking data comprising a second spatial state of a second tracked device;

receiving, at the imaging system, a first set of real-time visual medical imaging data generated by the first tracked device;

receiving at the imaging system a second set of real-time medical imaging data generated by the second tracked device;

combining, using the imaging system, the first set of visual medical imaging data and the second set of medical imaging data into combined displayable data based on the electronic tracking data comprising the first spatial state and the electronic tracking data comprising the second spatial state; and

causing the display of the combined displayable data.

25. The method of claim 24, wherein receiving electronic tracking data comprises continually receiving electronic tracking data.

26. The method of claim 24, wherein the image guidance data comprises a two-dimensional representation of the first set of real-time visual medical imaging data and the second set of real-time medical imaging data combined based on the relative registration of the first set of real-time visual medical imaging data to the second set of real-time medical imaging data in three-dimensional space.

27. The method of claim 24, wherein the first set of real-time visual medical imaging data comprises an ultrasound data captured by an ultrasound wand.

28. The method of claim 24, wherein second set of real-time medical imaging data comprises a three-dimensional model of at least a portion of the second device.

29. The method of claim 24, wherein the first tracked device comprises an ultrasound wand and the second tracked device comprises an ablation needle.