Image annotation in image-guided medical procedures

• US 8,554,307 B2
• Filed: 01/26/2011
• Issued: 10/08/2013
• Est. Priority Date: 04/12/2010
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: Razzaque, Sharif, State, Andrei
• Primary Examiner(s): Jung, Unsu
• Assistant Examiner(s): SIRIPURAPU, RAJEEV P
• Application Number:
  US13/014,596
• Publication Number:
  US 20110251483A1
• Time in Patent Office: 986 Days
• Field of Search: 600/424, 382/128
• US Class Current: 600/424
• CPC Class Codes: A61B 18/1477 Needle-like probes
• CPC Class Codes: A61B 2017/00207 with hand gesture control or hand gesture recognition
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/748 Selection of a region of interest, e.g. using a graphics tablet
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4494 Means for identifying the diagnostic device
• CPC Class Codes: A61B 6/466 adapted to display 3D data
• CPC Class Codes: A61B 6/468 allowing annotation or message recording
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/466 adapted to display 3D data 3D imaging and stereoscopic displays in devices for short-range imaging with acoustic waves G01S15/8993 and G01S7/52068, respectively; 3D image rendering G06T15/00; manipulating 3D models for computer graphics G06T19/00
• CPC Class Codes: A61B 8/468 allowing annotation or message recording
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment

Abstract

Presented herein are methods, systems, devices, and computer-readable media for image annotation in image-guided medical procedures. Some embodiments herein allow physicians or other operators to use one or more medical devices in order to define annotations in 3D space. These annotations may later be displayed to the physician or operator in 3D space in the position in which they were first drawn or otherwise generated. In some embodiments, the operator may use various available medical devices, such as needles, scalpels, or even a finger in order to define the annotation. Embodiments herein may allow an operator to more conveniently and efficiently annotate visualizable medical data.

Claims

1. A method for image annotation in image guided medical procedures, comprising:

determining, with one or more computing devices, position and orientation of a first medical device based at least on data from a first 3D tracking unit coupled with the first medical device;

automatically selecting, using the one or more computing devices, a 2D plane within a 3D space based at least on the position and orientation of the first medical device;

selecting visualizable medical data in the 2D plane from a 3D medical data set of a patient based at least on the position and orientation of the first medical device;

displaying on one or more displays the 3D space and the visualizable medical data in the 2D plane;

determining, with the one or more computing devices, position and orientation of a second medical device over time based at least on data from a second 3D tracking unit coupled with the second medical device;

generating an annotation in the 3D space on the visualizable medical data in the 2D plane based at least on the position and orientation of the second medical device over time;

generating image guidance information, with the one or more computing devices, based at least on the annotation in the 3D space; and

displaying, on one or more displays, a graphical rendering of the image guidance information.

2. The method of claim 1, wherein the method further comprises generating virtual ink at the intersection of an axis associated with the second medical device and the visualizable medical data in the 2D plane;

and generating the annotation in the 3D space comprises generating the annotation in the 3D space based at least on the virtual ink.

3. The method of claim 1, wherein generating the annotation in the 3D space comprises determining an intersection of an axis associated with the second medical device over time and the visualizable medical data in the 2D plane.

4. The method of claim 3, wherein generating the annotation in the 3D space comprises generating a spline based on the intersection of the axis associated with the second medical device over time and the visualizable medical data in the 2D plane.

5. The method of claim 1, further comprising determining position and orientation of the visualizable medical data in the 2D plane within the 3D space.

6. The method of claim 1, wherein

determining the position and orientation of the second medical device over time, comprises determining the position and orientation of the second medical device over time relative to a 2D screen displaying the visualizable medical data in the 2D plane;

and

generating the annotation in the 3D space comprises determining the annotation in the 3D space based at least in part on the position and orientation of the second medical over time relative to the 2D screen.

7. The method of claim 1,

wherein determining the position and orientation of the second medical device over time comprises determining the position and orientation of the second medical device over time relative to the one or more displays displaying the visualizable medical data in the 2D plane.

8. The method of claim 7, further comprising:

determining an intersection of an axis associated with the second medical device and the visualizable medical data in the 2D plane displayed on the one or more displays,

wherein determining the annotation in the 3D space comprises determining the annotation in the 3D space based at least on the intersection of the axis associated with the second medical device and the visualizable medical data in the 2D plane displayed on the one or more displays.

9. The method of claim 1, wherein determining position and orientation of the second medical device over time comprises receiving input from a touch screen and determining position and orientation based on input from the touch screen.

10. The method of claim 1, wherein determining position and orientation of the second medical device over time comprises receiving input from a remote pointer and determining position and orientation based on input from the remote pointer.

20. The method of claim 1, wherein at least a portion of the second medical device is inserted into the patient.

21. The method of claim 1, wherein the first medical device is an ultrasound wand.

11. A system for image annotation in image guided medical procedures, comprising one or more computing devices, said computing devices being configured to:

determine position and orientation of a first medical device based at least on data from a first 3D tracking unit coupled with the first medical device;

select a 2D plane of visualizable medical data from a 3D medical data set of a patient based at least on the position and orientation of the first medical device;

determine position and orientation over time for a second medical device based at least on data from a second 3D tracking unit coupled with the second medical device;

generate an annotation in 3D space on the visualizable medical data of the 2D plane of visualizable medical data based at least on the position and orientation over time for the second medical device;

generate image guidance information based at least on the annotation in 3D space; and

display, on one or more displays, a graphical rendering of the image guidance information.

12. The system of claim 11, wherein generating the annotation in the 3D space comprises determining an intersection of an axis associated with the second medical device over time and the 2D plane of visualizable medical data and generating a spline based on the intersection of the axis associated with the second medical device over time and the 2D plane of visualizable medical data.

13. The system of claim 11, wherein

determining position and orientation over time for the second medical device, comprises determining the position and orientation over time for the second medical device relative to a 2D screen displaying the 2D plane of visualizable medical data;

and

generating the annotation in the 3D space comprises determining the annotation in the 3D space based at least in part on the position and orientation over time for the second medical device relative to the 2D screen.

14. The system of claim 11, wherein the computing devices are further configured to:

display a projection of the 2D plane of visualizable medical data on a display, and

wherein determining position and orientation over time for the second medical device, comprises determining position and orientation over time of the second medical device relative to the displayed projection of the 2D plane of visualizable medical data.

15. The system of claim 14, wherein the computing devices are further configured to:

determine an intersection of an axis associated with the second medical device and the displayed projection of the 2D plane of visualizable medical data,

wherein generating the annotation in the 3D space comprises generating the annotation in the 3D space based on the intersection of the axis associated with the second medical device and the displayed projection of the 2D plane of visualizable medical data.

16. A non-transient computer-readable medium comprising computer-executable instructions for image annotation in image guided medical procedures, said computer-executable instructions, when executing on one or more computing devices, cause the one or more computing devices to:

determine position and orientation of a first medical device;

select one or more planes of visualizable medical data of a patient based at least on the position and orientation of the first medical device;

determine position and orientation over time for a second medical device;

generate an annotation in the 3D space based on the position and orientation over time of the second medical device and the one or more planes of visualizable medical data;

generate image guidance information, with the one or more computing devices, based at least on the annotation in the 3D space; and

display, on one or more displays, a graphical rendering of the image guidance information.

17. The non-transient computer-readable medium of claim 16, wherein determining the position and orientation of the first medical device comprises determining position and orientation over time for the first medical device.

18. The non-transient computer-readable medium of claim 17, wherein generating the annotation in the 3D space comprises generating a non-planar annotation in the 3D space based on the position and orientation over time for the second medical device and the position and orientation over time for the first medical device.

19. The non-transient computer-readable medium of claim 16, wherein generating the annotation in the 3D space comprises generating a spline based on the position and orientation of the first medical device and the position and orientation over time for the second medical device.