OPERATING ROOM BLACK-BOX DEVICE, SYSTEM, METHOD AND COMPUTER READABLE MEDIUM FOR EVENT AND ERROR PREDICTION

• US 20180122506A1
• Filed: 03/24/2016
• Published: 05/03/2018
• Est. Priority Date: 03/26/2015
• Status: Active Grant

• Current Assignee: Surgical Safety Technologies Inc.
• Original Assignee: Surgical Safety Technologies Inc.
• Inventors: GRANTCHAROV, Teodor Pantchev, YANG, Kevin Lee
• Granted Patent: US 11,322,248 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: G06F 1/12 Synchronisation of different clock signals provided by a plurality of clock generators
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: H04L 63/0421 Anonymous communication, i.e. the party's identifiers are hidden from the other party or parties, e.g. using an anonymizer

Abstract

A multi-channel recorder/encoder for collecting, integrating, synchronizing and recording medical or surgical data received as independent live or real-time data streams from a plurality of hardware units. The medical or surgical data relating to a live or real-time medical procedure. Example hardware units include a control interface, cameras, sensors, audio devices, and patient monitoring hardware. Further example systems may include a cloud based platform incorporating the encoder.

Claims

1. A system for collecting and processing medical or surgical data comprising:

a plurality of hardware units for collecting real-time medical or surgical data streams having a control interface coupled by a network to cameras, sensors, audio devices, and patient monitoring hardware, the real-time medical or surgical data streams relating to a real-time medical procedure within an operating or clinical site;

an encoder with a network server for synchronizing and recording the real-time medical or surgical data streams to a common clock or timeline to generate a session container file;

the network server configured to control a multi-nodal perception engine to;

generate a protocol for data extraction from the session container file;

process the data using the protocol to extract patterns for time-stamped clinical events within the session container file, each time-stamped clinical event associated with a confidence level;

generate an interface indicator for a temporal sequence of the time-stamped clinical events within the session container file and error assessments, the interface indicator identifying each of the time-stamped clinical events and the associated confidence levels; and

generate a predictive data model for refining protocol generation using support vector machines or artificial intelligence network data structures with neural networks for modelling correlation of data for interference and feature extraction.

2. The system of claim 1, further comprising device middleware and hardware to establishes a secure reliable connection using a network infrastructure for communication with the encoder and the hardware units, the device middleware and hardware for translating, connecting, and formatting the real-time medical or surgical data streams received independently from the hardware units.

3. The system of claim 2, wherein the device middleware and hardware implements data conformity and accurate synchronization for the real-time medical or surgical data streams using network protocols for clock synchronization between the hardware units to assist the encoder to generate the session container file.

4. The system of claim 1, wherein the encoder and device middleware and hardware are operable to interface with third party devices to receive additional data feeds as part of the real-time medical or surgical data streams.

5. The system of claim 1, further comprising a central control station accessible using the control interface, the control station configured to control processing of the data streams in response to input control comprising play/pause, stop session, record session, move to session frame, split-display, recording status indicator, and log file.

6. The system of claim 1, wherein network infrastructure provides increased fail-over and redundancy for the real-time medical or surgical data streams from the hardware units.

7. The system of claim 1, further comprising a storage area network for storing data container files of the real-time medical or surgical data streams until scheduled transmission.

8. The system of claim 1, wherein the encoder implements identity anonymization and encryption to the medical or surgical data.

9. The system of claim 1, wherein the encoder processes the real-time medical or surgical data streams to generate measurement metrics relating to the medical procedure.

10. The system of claim 1, wherein the real-time medical or surgical data streams correlates to a timeline, wherein the encoder detects events within the real-time medical or surgical data streams at corresponding times on the timeline, and tags and timestamps the session container file with the events, the timestamps corresponding to times on the timeline.

11. The system of claim 1, further comprising an intelligent dashboard interface for annotation and tagging of the synchronized medical or surgical data streams, wherein the intelligent dashboard may implement a viewer with playback viewing for reviewing content and interface controls for tagging content.

12. The system of claim 11, wherein the intelligent dashboard is multi-dimensional in that the union of all dimension variables for the medical procedure may indicate a specific set of one or more applicable annotation dictionaries or coding templates.

13. The system of claim 12 wherein example variables that may be used to determine the annotation and tagging dictionary may be:

the type of medical procedure being performed, the aspect of the procedure that is being analyzed, the geographic area/region where the procedure is being performed.

34. The system of claim 1, wherein the time-stamped clinical events within the session container file is stored with associated metadata for duration and frequency of each time-stamped clinical event.

35. The system of claim 1, wherein the network server uses patterns for time-stamped clinical events within the session container file to identify and extract features from the session container file for correlation or spectral analysis based on temporal nature of the time-stamped clinical events within the session container file.

36. The system of claim 1, wherein the network server identifies frequent temporal events as patterns leading to adverse events or errors in the timeline and develops predictive models to identify critical events during the real-time medical procedures.

37. The system of claim 1, wherein the network server groups the time-stamped clinical events within the session container file into technical and non-technical events.

38. The system of claim 1, wherein the interface indicator comprises an audio representation of the sequence of the time-stamped clinical events.

39. The system of claim 1, wherein the network server configures the multi-nodal perception engine for filtering the time-stamped clinical events within the session container file using machine learning with feature extraction for event correlation using computational intelligence, the multi-nodal perception engine interfacing with distributed hardware units.

40. The system of claim 1, wherein the network server generates the predictive data model by generating an Attribute-Relation File Format and the artificial intelligence network data structures and creates the temporal sequence of the time-stamped clinical events within the session container file, the temporal sequence relating to predicted locations of surgical error, wherein the interface indicator receives rating indicia for the error assessments of the time-stamped clinical events.

41. The system of claim 1, wherein the network server implements post-processing of the time-stamped clinical events within the session container file for comparative processing for outcome links to generate hazard maps for the interface indicator.

14. A multi-channel encoder for collecting, integrating, synchronizing and recording medical or surgical data streams onto a single interface with a common timeline or clock, the medical or surgical data streams received as independent real-time or live data streams from a plurality of hardware units, the encoder having a network server for scheduling transmission of session file containers for the recordings, the encoder processing the medical or surgical data streams to generate measurement metrics relating to a real-time medical procedure, the encoder configured to generate a protocol for data extraction from the session container file;

process the data using the protocol to define patterns for time-stamped clinical events within the session container file;

generate an interface indicator for a visual sequence of the time-stamped clinical events within the session container file and correspondence assessments;

generate a predictive data model for refining protocol generation using support vector machines or artificial intelligence network data structures.

15. The encoder of claim 14, wherein the encoder generates as output a single session transport file using lossless compression operations.

16. The encoder of claim 15, wherein the encoder detects completion of a recording of the data streams and securely encrypts the single transport file.

17. The encoder of claim 14, wherein the encoder implements identity anonymization to the medical or surgical data.

18. The encoder of claim 14, the data streams comprising audio, video, text, metadata, quantitative, semi-quantitative, and data feeds.

19. A method for collecting and processing medical or surgical data comprising:

receiving, at a multi-channel encoder, a plurality of live or real-time independent input feeds from one or more data capture devices located in an operating room or other patient intervention area, the input feeds relating to a live or real-time medical procedure;

synchronizing, by the encoder, the plurality of live independent input feeds onto a single interface with a common timeline or clock;

recording the synchronized input feeds using a network server;

generating, by the encoder, an output session file using the synchronized input feeds;

transmitting the output session file using the network server;

generating a protocol for data extraction from the session container file;

processing the data using the protocol to define patterns for time-stamped clinical events within the session container file;

generating an interface indicator for a visual sequence of the time-stamped clinical events within the session container file and correspondence assessments; and

generating a predictive data model for refining protocol generation using support vector machines or artificial intelligence network data structures.

20. The method of claim 19, further comprising processing the data streams for identity anonymization.

21. The method of claim 19 further comprising routing the data streams using a switch router to the encoder.

22. A cloud based system for collecting and processing medical or surgical data comprising:

an encoder having a control interface for, in response to receiving a control command, triggering collection of real-time medical or surgical data streams by smart devices including cameras, sensors, audio devices, and patient monitoring hardware, the medical or surgical data relating to a real-time medical procedure within an operating or clinical site, the encoder for authenticating the smart devices, the smart devices synchronizing the real-time medical or surgical data streams by embedding timestamp markers within the real-time medical or surgical data streams, the timestamp markers generated by each smart device by a device clock;

a media management hub server with middleware and hardware for translating, connecting, formatting, and recording the real-time medical or surgical data streams to generate session container files on network accessible storage devices;

wireless network infrastructure to provide a secure network connection between the encoder, the smart devices and the media management hub server for communication of the real-time medical or surgical data streams;

a central content server for storing and distributing the session container files and providing a two-way communication interface for the media management hub to implement a file transfer handshake for the session container files, the central content server configured to;

generate a protocol for data extraction from the session container file;

process the data using the protocol to define patterns for time-stamped clinical events within the session container file;

generate an interface indicator for a visual sequence of the time-stamped clinical events within the session container file and correspondence assessments;

generate a predictive data model for refining protocol generation using support vector machines or artificial intelligence network data structures; and

switching or gateway hardware to transmit the session container files from the media management hub to the central content server.

23. The cloud based system of claim 22, wherein the media management hub server broadcasts clock data to the smart devices for synchronization of the device clocks.

24. The cloud based system of claim 22, wherein the encoder provides a user interface to receive the control command and display real-time visual representations of the medical or surgical data.

25. The cloud based system of claim 22, wherein the media management hub server aggregates, packages, compresses and encrypts the real-time data streams to generate the session container files.

26. The cloud based system of claim 22, wherein the media management hub server manages the smart devices based on location, schedule, zone and requirements.

27. The cloud based system of claim 22, wherein the media management hub server receives operating status data from the smart devices to generate a management interface with a visual representation of the operating status data for the smart devices, the operating status data including online, offline, running capture, and on-board storage.

28. The cloud based system of claim 27, wherein the media management hub server processes the operating status data to detect smart devices operating outside of normal conditions and in response generating an alert notification of the detected smart devices operating outside of normal conditions.

29. The cloud based system of claim 22, wherein the media management hub server implements a device communication interface for the smart devices to implement a device data transfer handshake for the real-time medical or surgical data streams.

30. The cloud based system of claim 22, wherein the media management hub server authenticates the smart devices.

31. The cloud based system of claim 22, further comprising a computational intelligence platform for receiving the session container files to construct an analytics model to identify clinical factors within the session container files for predictions, costs and safety hazards, the analytics model providing a network for extracting features, correlations and event behaviour from the session container files that involve multivariable data sets with time-variant parameters.

32. The cloud based system of claim 22, further comprising a training or education server to receive the session container files, process the session container files to identify root causes of adverse patient outcomes and generate a training interface to communicate training data using the identified root causes and the session container files.

33. The cloud based system of claim 22, wherein the smart devices include motion tracking devices for markerless motion tracking of objects within the operating or clinical site, the system further comprising a processor configured to convert captured motion data from the motion tracking devices into data structures identifying human factors, workflow design and chain-of-events.