System and method of providing real-time dynamic imagery of a medical procedure site using multiple modalities

• US 9,659,345 B2
• Filed: 07/08/2013
• Issued: 05/23/2017
• Est. Priority Date: 08/02/2006
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: Razzaque, Sharif A, Keller, Kurtis P, Ackerman, Jeremy D, State, Andrei, Green, Caroline K
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US13/936,951
• Publication Number:
  US 20140016848A1
• Time in Patent Office: 1,415 Days
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: G06T 3/16 Spatio-temporal transformations, e.g. video cubism
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method of providing composite real-time dynamic imagery of a medical procedure site from multiple modalities which continuously and immediately depicts the current state and condition of the medical procedure site synchronously with respect to each modality and without undue latency is disclosed. The composite real-time dynamic imagery may be provided by spatially registering multiple real-time dynamic video streams from the multiple modalities to each other. Spatially registering the multiple real-time dynamic video streams to each other may provide a continuous and immediate depiction of the medical procedure site with an unobstructed and detailed view of a region of interest at the medical procedure site at multiple depths. As such, a surgeon, or other medical practitioner, may view a single, accurate, and current composite real-time dynamic imagery of a region of interest at the medical procedure site as he/she performs a medical procedure, and thereby, may properly and effectively implement the medical procedure.

Claims

1. A method of medical procedure image guidance, comprising the steps of:

receiving real-time first spatial state data associated with a first medical device;

obtaining first medical imaging data associated with the first medical device;

receiving real-time second spatial state data associated with a second medical device that is different from the first medical device;

obtaining second medical imaging data associated with the second medical device;

determining, using one or more processors, a spatial relationship between the first medical device and the second medical device based at least in part on the first spatial state data and the second spatial state data; and

causing a display device to display composite medical imaging data comprising at least a portion of the first medical imaging data and at least a portion of the second medical imaging data based at least in part on the determined spatial relationship between the first medical device and the second medical device.

2. The method of claim 1, further comprising spatially registering the first medical imaging data with the second medical imaging data based at least on the first spatial state data associated with the first medical device and the second spatial state data associated with the second medical device.

3. The method of claim 1, wherein the first medical device is usable in at least one of laparoscopy, hysteroscopy, thoroscopy, arthoscopy, colonoscopy, bronchoscopy, cycstoscopy, proctosigmoidoscopy, esophagogastroduodenoscopy, and colposcopy.

4. The method of claim 1, wherein the first medical device is usable in at least one of ultrasonography, magnetic resonance imaging, x-ray imaging, computed tomography, and optical wavefront imaging.

5. The method of claim 1, wherein the first medical imaging data comprises video.

6. The method of claim 1, wherein the first medical device comprises a two-dimensional modality.

7. The method of claim 1, wherein, the first medical device comprises a three-dimensional modality.

8. The method of claim 1, wherein the second medical device comprises a two-dimensional modality.

9. The method of claim 1, wherein the second medical device comprises a three-dimensional modality.

10. The method of claim 1, wherein the second medical device comprises an invasive medical device.

11. A system for medical procedure image guidance, comprising:

an image guidance system, wherein the image guidance system is configured to;

receive real-time first spatial state data associated with a first medical device;

obtain first medical imaging data associated with the first medical device;

receive real-time second spatial state data associated with a second medical device that is different from the first medical device;

obtain second medical imaging data associated with the second medical device;

determine a spatial relationship between the first medical device and the second medical device based at least in part on the first spatial state data and the second spatial state data; and

cause a display device to display composite medical imaging data comprising at least a portion of the first medical imaging data and at least a portion of the second medical imaging data based at least in part on the determined spatial relationship between the first medical device and the second medical device.

12. The system of claim 11, wherein the image guidance system is configured to spatially register the first medical imaging data with the second medical imaging data based at least on the first spatial state data associated with the first medical device and the second spatial state data associated with the second medical device.

13. The system of claim 11, wherein the first medical device comprises a plurality of first modalities.

14. The system of claim 11, wherein the second medical device comprises a plurality of second modalities.

15. The system of claim 11, wherein the first medical device is usable in at least one of ultrasonography, magnetic resonance imaging, x-ray imaging, computed tomography, and optical wavefront imaging.

16. The system of claim 11, wherein the first medical device is usable in at least one of laparoscopy, hysteroscopy, thoroscopy, arthoscopy, colonoscopy, bronchoscopy, cycstoscopy, proctosigmoidoscopy, esophagogastroduodenoscopy, and colposcopy.

17. A computer-readable non-transitory storage medium comprising instructions operable to execute on a computer system, said instructions when executing on said computer system cause the computer system to:

receive real-time first spatial state data associated with a first medical device;

obtain first medical imaging data associated with the first medical device;

receive real-time second spatial state data associated with a second medical device that is different from the first medical device;

obtain second medical imaging data associated with the second medical device;

determine a spatial relationship between the first medical device and the second medical device based at least in part on the first spatial state data and the second spatial state data; and

cause a display device to display composite medical imaging data comprising at least a portion of the first medical imaging data and at least a portion of the second medical imaging data based at least in part on the determined spatial relationship between the first medical device and the second medical device.