Image annotation in image-guided medical procedures

• US 9,107,698 B2
• Filed: 10/07/2013
• Issued: 08/18/2015
• Est. Priority Date: 04/12/2010
• Status: Active Grant

• Current Assignee: Inneroptic Technology, Inc.
• Original Assignee: Inneroptic Technology, Inc.
• Inventors: Razzaque, Sharif, State, Andrei
• Primary Examiner(s): SIRIPURAPU, RAJEEV P
• Application Number:
  US14/047,628
• Publication Number:
  US 20140094687A1
• Time in Patent Office: 680 Days
• Field of Search: 600/424, 382/128
• US Class Current: 1/1
• CPC Class Codes: A61B 18/1477 Needle-like probes
• CPC Class Codes: A61B 2017/00207 with hand gesture control or hand gesture recognition
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/748 Selection of a region of interest, e.g. using a graphics tablet
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4494 Means for identifying the diagnostic device
• CPC Class Codes: A61B 6/466 adapted to display 3D data
• CPC Class Codes: A61B 6/468 allowing annotation or message recording
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: A61B 8/466 adapted to display 3D data 3D imaging and stereoscopic displays in devices for short-range imaging with acoustic waves G01S15/8993 and G01S7/52068, respectively; 3D image rendering G06T15/00; manipulating 3D models for computer graphics G06T19/00
• CPC Class Codes: A61B 8/468 allowing annotation or message recording
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment

Abstract

Presented herein are methods, systems, devices, and computer-readable media for image annotation in image-guided medical procedures. Some embodiments herein allow physicians or other operators to use one or more medical devices in order to define annotations in 3D space. These annotations may later be displayed to the physician or operator in 3D space in the position in which they were first drawn or otherwise generated. In some embodiments, the operator may use various available medical devices, such as needles, scalpels, or even a finger in order to define the annotation. Embodiments herein may allow an operator to more conveniently and efficiently annotate visualizable medical data.

Claims

1. A method for image annotation in image guided medical procedures, comprising:

determining, with one or more computing devices, position and orientation of a first medical device;

determining position and orientation of an imaging area based at least on the position and orientation of the first medical device;

causing one or more displays to display a perspective view of an image corresponding to the imaging area within a virtual 3D space based at least on the position and orientation of the first medical device relative to a position of a user;

tracking, with the one or more computing devices, a position of a second medical device; and

causing the one or more displays to display an annotation in the virtual 3D space based at least in part on an intersection of an axis of the second medical device with the imaging area.

2. The method of claim 1, wherein the second medical device comprises at least one of a needle, a stylus, an ultrasound transducer, a cauterizer, a scalpel, and a glove.

3. The method of claim 1, wherein causing the one or more displays to display the annotation comprises causing the one or more displays to display the annotation based at least on the intersection of the at least a portion of the second medical device with the imaging area over time.

4. The method of claim 1, wherein the position of the user comprises a predetermined location.

5. A system for image annotation in image-guided medical procedures, comprising:

one or more computing devices in communication with one or more displays, a first medical device, and a second medical device, wherein the one or more computing devices are configured to;

determine position and orientation of the first medical device;

determine position and orientation of an imaging area based at least on the position and orientation of the first medical device;

cause the one or more displays to display a perspective view of an image corresponding to the imaging area within a virtual 3D space based at least on the position and orientation of the first medical device relative to a position of a user;

track a position of the second medical device; and

cause the one or more displays to display an annotation in the virtual 3D space based at least on an intersection of an axis of the second medical device with the imaging area.

6. The system of claim 5, wherein the intersection comprises contact between the at least a portion of the second medical device and the one or more displays.

7. The system of claim 5, wherein the second medical device comprises at least one of a needle, a stylus, an ultrasound transducer, a cauterizer, or a scalpel.

8. The system of claim 5, wherein the second medical device comprises a glove.

9. The system of claim 5, wherein the one or more computing devices are further configured to cause the one or more displays to display virtual ink at the intersection of the at least a portion of the second medical device and the perspective view of the imaging area.

10. The system of claim 5, wherein to cause the one or more displays to display the annotation, the one or more computing devices are configured to cause the one or more displays to display the annotation based at least on the intersection of the at least a portion of the second medical device with the imaging area over time.

11. The system of claim 5, wherein to track the position of the second medical device the one or more computing devices are configured to receive input from a touch screen and determine position based on the input from the touch screen.

12. The system of claim 5, wherein to track position of the second medical device the one or more computing devices are configured to receive input from a remote pointer and determine position based on the input from the remote pointer.

13. The system of claim 5, wherein the position of the user comprises a predetermined location.

14. A non-transient computer-readable medium comprising computer-executable instructions, said computer-executable instructions, when executing on one or more computing devices, cause the one or more computing devices to:

determine position and orientation of a first medical device;

determine position and orientation of an imaging area based at least on the position and orientation of the first medical device;

cause one or more displays to display a perspective view of an image corresponding to the imaging area within a virtual 3D space based at least on the position and orientation of the first medical device relative to a position of a user;

track a position of a second medical device; and

cause the one or more displays to display an annotation in the virtual 3D space based at least on an intersection of an axis of the second medical device with the imaging area.

15. The non-transient computer-readable medium of claim 14, wherein causing the one or more displays to display the annotation comprises causing the one or more displays to display the annotation based at least on the intersection of the at least a portion of the second medical device with the imaging area over time.

16. The non-transient computer-readable medium of claim 14, wherein the position of the user comprises a predetermined location.