Feedback during surgical events

• US 8,886,577 B2
• Filed: 01/22/2013
• Issued: 11/11/2014
• Est. Priority Date: 11/18/2009
• Status: Active Grant

• Current Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Original Assignee: Empire Technology Development LLC (Allied Inventors Management, LLC)
• Inventors: Mangione-Smith, William H.
• Primary Examiner(s): VINCENT, DAVID ROBERT
• Application Number:
  US13/747,309
• Publication Number:
  US 20130138599A1
• Time in Patent Office: 658 Days
• Field of Search: 706/12, 706/45, 706/62
• US Class Current: 706/12
• CPC Class Codes: A61B 2017/00115 with audible or visual output
• CPC Class Codes: A61B 2017/00119 alarm; indicating an abnormal situation
• CPC Class Codes: A61B 2017/00128 related to intensity or progress of surgical action
• CPC Class Codes: A61B 2034/104 Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 8/0841 for locating instruments
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61B 90/98 using electromagnetic means, e.g. transponders
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

Technology is disclosed for communicating surgical information. The technology can receive data from one or more sensors coupled to one or more tools in operation by a user, generate a virtual model of a four-dimensional virtual real-time space, receive motion history data for the user'"'"'s previous movements, generate a prediction of movement of the first tool in each of the four dimensions in relation to the first vital entity and a prediction score, and output an indication of the prediction score.

Claims

1. A method for communicating surgical information, comprising:

receiving, during a procedure, first data from one or more sensors coupled to one or more tools;

generating a virtual model of a four-dimensional virtual real-time space including a representation of a first vital entity, wherein a first tool is mapped, based on a portion of the first data, in the virtual real-time space in relation to at least the representation of the first vital entity;

determining, based on the first vital entity, at least one probabilistic model, from a plurality of probabilistic models, that applies to the first data and the four-dimensional virtual real-time space;

applying the determined at least one probabilistic model to obtain;

a prediction of a movement of the first tool in the procedure in relation to the representation of the first vital entity, and

a prediction score, wherein the prediction score indicates either a severity of a warning or an estimated reliability of the prediction of the movement of the first tool in the procedure;

outputting an indication of the prediction score;

receiving, from one or more sensors, additional data collected during the procedure indicating an actual outcome of the movement of the first tool in the procedure; and

automatically updating, based at least in part on the actual outcome of the movement of the first tool in the procedure, one or more of the determined at least one probabilistic model.

2. The method of claim 1, wherein the applying includes accessing historical information from a data store to predict movement of the first tool.

3. The method of claim 1, wherein the indication of the prediction score comprises a hazard warning signal.

4. The method of claim 3, wherein the hazard warning signal comprises at least one of a temperature signal, a vibration frequency, or a sound pitch.

5. The method of claim 1, further comprising measuring auditory information, ultrasound information, or visual information.

6. The method of claim 1, wherein the virtual model includes a length dimension, a width dimension, a height dimension, and a time dimension.

7. The method of claim 1, further comprising receiving second data from a second sensor coupled to a second tool, wherein the second tool is mapped in the virtual real-time space in relation to the first vital entity and a second vital entity via the second sensor.

8. The method of claim 7, further comprising:

applying the virtual real-time space against at least a first probabilistic model and a second probabilistic model to obtain third data about movement of the second tool in relation to the first vital entity and the second vital entity; and

outputting the third data to an output device to convey haptic information about the second tool.

9. A computer-readable storage medium storing computer-readable instructions that, when executed, perform a method comprising:

receiving, during procedure, first data from one or more sensors coupled to one or more tools;

generating a virtual model of a four-dimensional virtual real-time space including a representation of a first vital entity, wherein a first tool is mapped, based on a portion of the first data, in the virtual real-time space in relation to at least the representation of the first vital entity;

determining, based on the first vital entity, at least one probabilistic model, from a plurality of probabilistic models, that applies to the first data and the four-dimensional virtual real-time space;

applying the determined at least one probabilistic model to obtain;

a prediction of a movement of the first tool in the procedure in relation to the representation of the first vital entity, and

a prediction score, wherein the prediction score indicates either a severity of a warning or an estimated reliability of the prediction of the movement of the first tool in the procedure;

outputting an indication of the prediction score;

receiving, from the one or more sensors, additional data collected during the procedure indicating an actual outcome of the movement of the first tool in the procedure; and

automatically updating, based at least in part on the actual outcome of the movement of the first tool in the procedure, one or more of the determined at least one probabilistic model.

10. The computer-readable storage medium of claim 9, wherein the applying includes accessing historical information from a data store to predict movement of the first tool.

11. The computer-readable storage medium of claim 9, wherein the outputting includes outputting second data comprising a hazard warning signal.

12. The computer-readable storage medium of claim 11, wherein the hazard warning signal comprises at least one of a temperature signal, a vibration frequency, or a sound pitch.

13. The computer-readable storage medium of claim 9, wherein a first sensor is configured to measure auditory information, ultrasound, or visual information.

14. The computer-readable storage medium of claim 9, wherein the virtual model includes a length dimension, a width dimension, a height dimension, and a time dimension.

15. The computer-readable storage medium of claim 9, further comprising instructions for receiving second data from a second sensor coupled to a second tool, wherein the second tool is mapped in the virtual real-time space in relation to the first vital entity and a second vital entity via the second sensor.

16. The computer-readable storage medium of claim 15, further comprising instructions for:

applying the virtual real-time space against at least a first probabilistic model and a second probabilistic model to obtain third data about movement of the second tool in relation to the first vital entity and a second vital entity; and

outputting the third data to an output device to convey haptic information about the second tool.

17. A system for communicating surgical information, comprising:

a component configured to receive, during a procedure, data from one or more sensors coupled to one or more tools in operation by a user;

a component configured to generate a virtual model of a four-dimensional virtual real-time space including a representation of a first vital entity, wherein a first tool is mapped, based on a portion of the data, in the virtual real-time space in relation to at least the representation of the first vital entity;

a component configured to receive, from a data store, motion history data for the user'"'"'s previous movements;

a component configured to determine, based on the first vital entity and the motion history data, at least one probabilistic model that applies to the motion history data and the four-dimensional virtual real-time space;

a component configured to generate, using the determined probabilistic model and based on the received user'"'"'s previous movements and the received data;

a prediction of a movement of the first tool in the procedure in relation to the first vital entity, and

a prediction score, wherein the prediction score indicates either a severity of a warning or an estimated reliability of the prediction of the movement of the first tool in the procedure;

a component configured to output an indication of the prediction score; and

a component configured to receive, from the one or more sensors, additional data collected during the procedure indicating an actual outcome of the movement of the first tool in the procedure, and to automatically update, based at least in part on the actual outcome of the movement of the first tool in the procedure, one or more of the determined at least one probabilistic models.

18. The system of claim 17, wherein historical information from a data store is accessed to predict movement of the first tool.

19. The system of claim 17, wherein the output comprises a hazard warning signal.

20. The system of claim 19, wherein the hazard warning signal comprises at least one of a temperature signal, a vibration frequency, or a sound pitch.