SYSTEM AND METHOD OF PROVIDING REAL-TIME DYNAMIC IMAGERY OF A MEDICAL PROCEDURE SITE USING MULTIPLE MODALITIES

• US 20080030578A1
• Filed: 08/02/2007
• Published: 02/07/2008
• Est. Priority Date: 08/02/2006
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: Ackerman, Jeremy, State, Andrei, Keller, Kurtis, Razzaque, Sharif, Green, Caroline
• Granted Patent: US 7,728,868 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: G06T 3/16 Spatio-temporal transformations, e.g. video cubism
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system and method of providing composite real-time dynamic imagery of a medical procedure site from multiple modalities which continuously and immediately depicts the current state and condition of the medical procedure site synchronously with respect to each modality and without undue latency is disclosed. The composite real-time dynamic imagery may be provided by spatially registering multiple real-time dynamic video streams from the multiple modalities to each other. Spatially registering the multiple real-time dynamic video streams to each other may provide a continuous and immediate depiction of the medical procedure site with an unobstructed and detailed view of a region of interest at the medical procedure site at multiple depths. As such, a surgeon, or other medical practitioner, may view a single, accurate, and current composite real-time dynamic imagery of a region of interest at the medical procedure site as he/she performs a medical procedure, and thereby, may properly and effectively implement the medical procedure.

Claims

1. A method of providing real-time dynamic imagery of a medical procedure site, comprising the steps of:

receiving a first real-time dynamic video stream of a scene based on a first modality from a first video source;

receiving a second real-time dynamic video stream of the scene based on a second modality from a second video source;

spatially registering the first real-time dynamic video stream and the second real-time dynamic video stream, wherein the first real-time dynamic video stream and the second real-time dynamic video stream align to form a composite representation of the scene; and

generating a composite real-time dynamic video stream of the scene from the composite representation.

2. The method of claim 1, wherein the spatially registering uses information other than an anatomical characteristic.

3. The method of claim 1, wherein the spatially registering uses information other than a reference coordinate system.

4. The method of claim 1, wherein the spatially registering uses information other than the position of a subject of a medical procedure.

5. The method of claim 1, wherein the first video source comprises a first spatial state, and the second video source comprises a second spatial state.

6. The method of claim 5, wherein the spatially registering uses the first spatial state and the second spatial state, and wherein the first real-time dynamic video stream is synchronized with the second real-time dynamic video stream based on the first spatial state and the second spatial state.

7. The method of claim 1, wherein the first modality comprises a two-dimensional modality.

8. The method of claim 1, wherein the first modality comprises a three-dimensional modality.

9. The method of claim 1, wherein the second modality comprises a two-dimensional modality.

10. The method of claim 1, wherein the second modality comprises a three-dimensional modality.

11. The method of claim 1, wherein the first modality comprises a plurality of first modalities.

12. The method of claim 11, wherein one of the plurality of first modalities comprises endoscopy, and wherein the endoscopy comprises a modality selected from a group consisting of:

laparoscopy, hysteroscopy, thoroscopy, arthoscopy, colonoscopy, bronchoscopy, cycstoscopy, proctosigmoidoscopy, esophagogastroduodenoscopy, and colposcopy.

13. The method of claim 1, wherein the second modality comprises a plurality of second modalities.

14. The method of claim 13, wherein one of the plurality of second modalities comprises a modality selected from a group consisting of:

ultrasonography, magnetic resonance imaging, x-ray imaging, computed tomography, and optical wavefront imaging.

15. A system of providing real-time dynamic imagery of a medical procedure site, comprising:

a control system, wherein the control system is adapted to;

receive a first real-time dynamic video stream of a scene based on a first modality from a first video source;

receive a second real-time dynamic video stream of the scene based on a second modality from a second video source;

spatially register the first real-time dynamic video stream and the second real-time dynamic video stream, wherein the first real-time dynamic video stream and the second real-time dynamic video stream align to form a composite representation of the scene; and

generate a composite real-time dynamic video stream of the scene from the composite representation.

16. The system of claim 15, wherein the control system is adapted to spatially register using information other than anatomical characteristics.

17. The system of claim 15, wherein the control system is adapted to spatially register using information other than a reference coordinate system.

18. The method of claim 15, wherein the control system is adapted to spatially register using information other than the position of a subject of a medical procedure.

19. The system of claim 15, wherein the first video source comprises a first spatial state, and the second video source comprises a second spatial state.

20. The system of claim 19, wherein the control system is adapted to spatially register using the first spatial state and the second spatial state, and wherein the first real-time dynamic video stream is synchronized with the second real-time dynamic video stream based on the first spatial state and the second spatial state.

21. The system of claim 15, wherein the first modality comprises a two-dimensional modality.

22. The system of claim 15, wherein the first modality comprises a three-dimensional modality.

23. The system of claim 15, wherein the second modality comprises a two-dimensional modality.

24. The system of claim 15, wherein the second modality comprises a three-dimensional modality.

25. The system of claim 15, wherein the first modality comprises a plurality of first modalities.

26. The system of claim 25, wherein one of the plurality of first modalities comprises endoscopy, and wherein the endoscopy comprises a modality selected from a group consisting of:

laparoscopy, hysteroscopy, thoroscopy, arthoscopy, colonoscopy, bronchoscopy, cycstoscopy, proctosigmoidoscopy, esophagogastroduodenoscopy, and colposcopy.

27. The system of claim 15, wherein the second modality comprises a plurality of second modalities.

28. The system of claim 27, wherein one of the plurality of second modalities comprises a modality selected from a group consisting of:

ultrasonography, magnetic resonance imaging, x-ray imaging, computed tomography, and optical wavefront imaging.

29. A computer-readable medium comprising instructions for instructing a computer to:

receive a first real-time dynamic video stream of a scene based on a first modality from a first video source;

receive a second real-time dynamic video stream of the scene based on a second modality from a second video source;

spatially register the first real-time dynamic video stream and the second real-time dynamic video stream, wherein the first real-time dynamic video stream and the second real-time dynamic video stream align to form a composite representation of the scene; and

generate a composite real-time dynamic video stream of the scene from the composite representation.