Automated clinical documentation system and method

• US 10,546,655 B2
• Filed: 08/08/2018
• Issued: 01/28/2020
• Est. Priority Date: 08/10/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Owen, Donald E., Almendro Barreda, Daniel Paulino, Sharma, Dushyant
• Primary Examiner(s): Gilliard, Delomia L
• Application Number:
  US16/058,951
• Publication Number:
  US 20190051382A1
• Time in Patent Office: 538 Days
• Field of Search: None
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: G06F 16/637 Administration of user profiles, e.g. generation, initialization, adaptation or distribution
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06K 19/07762 the adhering arrangement making the record carrier wearable, e.g. having the form of a ring, watch, glove or bracelet record carriers for insertion in the human body for medical purposes A61B90/00; record carriers adapted for attachment to animals A01K11/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/52 : Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G10L 15/08 : Speech classification or search
• CPC Class Codes: G10L 15/1815 : Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 : Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 : Speaker identification or verification techniques
• CPC Class Codes: G10L 2021/02082 : the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 : Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0232 : Processing in the frequency domain
• CPC Class Codes: G10L 21/0272 : Voice signal separating
• CPC Class Codes: G10L 25/48 : specially adapted for particular use
• CPC Class Codes: G11B 27/10 : Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G16B 50/00 : ICT programming tools or database systems specially adapted for bioinformatics
• CPC Class Codes: G16H 10/20 : for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 10/60 : for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 30/00 : ICT specially adapted for the handling or processing of medical images computed tomography A61B6/03
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/20 : for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/60 : for the operation of medical equipment or devices
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 80/00 : ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: G16Y 20/00 : Information sensed or collected by the things
• CPC Class Codes: H04L 51/02 : using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/222 : using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source
• CPC Class Codes: H04R 1/326 : for microphones H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 1/406 : microphones
• CPC Class Codes: H04R 2420/07 : Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/02 : for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453
• CPC Class Codes: H04R 3/12 : for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 2400/15 : Aspects of sound capture and related signal processing for recording or reproduction

Abstract

A method, computer program product, and computing system for tracking encounter participants is executed on a computing device and includes obtaining encounter information of a patient encounter, wherein the encounter information includes machine vision encounter information obtained via one or more machine vision systems. The machine vision encounter information is processed to identify one or more humanoid shapes.

Claims

1. A computer-implemented method executed on a computing device, comprising:

obtaining encounter information of a patient encounter, wherein the encounter information includes machine vision encounter information obtained via one or more machine vision systems;

processing the machine vision encounter information to identify one or more humanoid shapes, wherein processing the machine vision encounter information includes identifying a first humanoid shape of the one or more humanoid shapes and tracking the movement of the first humanoid shape within a monitored space, and adding a second humanoid shape within the monitored space;

steering one or more audio recording beams toward one of the first and second humanoid shapes to capture audio encounter information;

processing the encounter information to generate at least a portion of an encounter transcript; and

processing at least the portion of the encounter transcript to populate at least a portion of a medical record associated with the patient encounter.

2. The computer-implemented method of claim 1 wherein tracking the movement of the one or more humanoid shapes within the monitored space includes:

removing an existing humanoid shape from the one or more humanoid shapes when the existing humanoid shape leaves the monitored space.

3. The computer-implemented method of claim 1 wherein the audio encounter information is included within the encounter information.

4. The computer-implemented method of claim 1 wherein the one or more machine vision systems includes one or more of:

an RGB imaging system;

an infrared imaging system;

an ultraviolet imaging system;

a laser imaging system;

an X-ray imaging system;

a SONAR imaging system;

a RADAR imaging system; and

a thermal imaging system.

5. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

obtaining encounter information of a patient encounter, wherein the encounter information includes machine vision encounter information obtained via one or more machine vision systems;

processing the machine vision encounter information to identify one or more humanoid shapes, wherein processing the machine vision encounter information includes identifying a first humanoid shape of the one or more humanoid shapes and tracking the movement of the first humanoid shape within a monitored space, and adding a second humanoid shape within the monitored space;

steering one or more audio recording beams toward one of the first and second humanoid shapes to capture audio encounter information;

processing the encounter information to generate at least a portion of an encounter transcript; and

processing at least the portion of the encounter transcript to populate at least a portion of a medical record associated with the patient encounter.

6. The computer program product of claim 5 wherein tracking the movement of the one or more humanoid shapes within the monitored space includes:

removing an existing humanoid shape from the one or more humanoid shapes when the existing humanoid shape leaves the monitored space.

7. The computer program product of claim 5 wherein the audio encounter information is included within the encounter information.

8. The computer program product of claim 5 wherein the one or more machine vision systems includes one or more of:

an RGB imaging system;

an infrared imaging system;

an ultraviolet imaging system;

a laser imaging system;

an X-ray imaging system;

a SONAR imaging system;

a RADAR imaging system; and

a thermal imaging system.

9. A computing system including a processor and memory configured to perform operations comprising:

obtaining encounter information of a patient encounter, wherein the encounter information includes machine vision encounter information obtained via one or more machine vision systems;

processing the machine vision encounter information to identify one or more humanoid shapes, wherein processing the machine vision encounter information includes identifying a first humanoid shape of the one or more humanoid shapes and tracking the movement of the first humanoid shape within a monitored space, and adding a second humanoid shape within the monitored space;

steering one or more audio recording beams toward one of the first and second humanoid shapes to capture audio encounter information;

processing the encounter information to generate at least a portion of an encounter transcript; and

processing at least the portion of the encounter transcript to populate at least a portion of a medical record associated with the patient encounter.

10. The computing system of claim 9 wherein tracking the movement of the one or more humanoid shapes within the monitored space includes:

removing an existing humanoid shape from the one or more humanoid shapes when the existing humanoid shape leaves the monitored space.

11. The computing system of claim 9 wherein the audio encounter information is included within the encounter information.

12. The computing system of claim 9 wherein the one or more machine vision systems includes one or more of:

an RGB imaging system;

an infrared imaging system;

an ultraviolet imaging system;

a laser imaging system;

an X-ray imaging system;

a SONAR imaging system;

a RADAR imaging system; and

a thermal imaging system.