Medical environment monitoring system

• US 10,489,661 B1
• Filed: 03/08/2017
• Issued: 11/26/2019
• Est. Priority Date: 03/08/2016
• Status: Active Grant

• Current Assignee: Ocuvera LLC
• Original Assignee: Ocuvera LLC
• Inventors: Rush, Benjamin D., Brown-Kramer, Joshua M., Sabalka, Lucas A., Gansemer, Brian S., Bauer, Paul B., Unterseher, Andrew R., Origas, Benjamin J., Durham, Douglas W., Wilsman, Tony R.
• Primary Examiner(s): Shin, Soo
• Application Number:
  US15/452,855
• Time in Patent Office: 993 Days
• Field of Search: None
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/0476 Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths

Abstract

A system and a method are described for monitoring a medical care environment. In one or more implementations, a method includes identifying a first subset of pixels within a field of view of a camera as representing a bed. The method also includes identifying a second subset of pixels within the field of view of the camera as representing an object (e.g., a patient) proximal to the bed. The method also includes determining an orientation of the object within the bed. In implementations, the method further includes issuing an electronic communication notification directing a user to view a display monitor associated with the object based upon the orientation of the object. In some implementations, the method further includes breaking out a particular portion of the display associated with the object. In some implementations, the method also includes determining whether to issue future notifications based upon feedback to the notification.

Claims

1. A system for monitoring a medical care environment,

the system comprising:

an imaging device having an imaging plane, the imaging device configured to receive at least one of a depth frame image, a color frame, or an infrared frame image representing a three-dimensional environment corresponding to a physical space;

at least one display monitor associated with one or more subjects, the at least one display monitor comprising a plurality of display portions, where each display portion is configured to receive the respective at least one of the depth frame image, the color frame image, or the infrared frame image associated with each of the one or more subjects, and display the at least one of the depth frame image, the color frame image, or the infrared frame image;

at least one computing device in communication with the imaging device, the at least one computing device including;

a memory configured to store one or more modules;

a processor coupled to the memory, the processor is configured to execute the one or more modules to cause the processor to;

classify, via a machine learning classifier, a subset of pixels within the at least one of the depth frame image, the color frame image, or the infrared frame image as representing a body part of the one or more subjects;

generate an orientation signal of the body part with respect to an image plane using the classified subset of pixels;

determine at least one state of the one or more subjects based upon the orientation signal;

generate an electronic communication notification to view the at least one display monitor based upon the at least one state of the one or more subjects;

generate a zoomed-in image of at least one display portion of the plurality of display portions within the at least one display monitor based upon the at least one state of the one or more subjects;

receive feedback from an electronic device indicative of a validity of the electronic communication notification;

adjust via a machine learning process, an electronic communication alert sensitivity level for the determined state based upon the feedback; and

determine whether to generate future electronic communication alerts for the determined state of the one or more subjects based on the adjusted electronic communication alert sensitivity level.

2. The system as recited in claim 1, wherein the processor is further configured to execute the one or more modules to cause the processor to process, via another processing technique, the at least one of the depth frame image, the color frame image, or the infrared frame image to generate another orientation signal of the body part with respect to the image plane;

determine a probability distribution based upon at least the orientation signal and the another orientation signal via a fusion engine; and

determine the at least one state of the one or more subjects based upon the probability distribution.

3. The system as recited in claim 1, wherein the processor is further configured to execute the one or more modules to cause the processor to determine whether the one or more subjects is at least one of transitioning from a bed or transitioning within the bed.

4. The system as recited in claim 1, wherein the processor is further configured to execute the one or more modules to cause the processor to determine an orientation of the one or more subjects within a seating platform.

5. The system as recited in claim 1, wherein the processor is further configured to execute the one or more modules to cause the processor to process the at least one of the depth frame image, the color frame image, or the infrared frame image to detect at least one of a presence or an absence of a gesture;

and provide an indication associated with the at least one of the presence or the absence of the gesture.

6. The system as recited in claim 1, wherein the machine learning classifier comprises a cascading Haar classifier.

7. A method for monitoring a subject within a medical care environment, the method comprising:

classifying, by a processor, a first subset of pixels within a field of view of an imaging device as representing an object of interest, the first subset of pixels representing a portion of a three-dimensional environment;

classifying, via a machine learning classifier, a second subset of pixels within the field of view of the imaging device as representing the subject within the field of view, the second subset of pixels representing a portion of the three-dimensional environment;

determining, by the processor using the machine learning classifier, at least one state associated with the subject based upon the first subset of pixels and the second subset of pixels;

generating an electronic communication notification to view a display monitor associated with the subject based upon the at least one state of the subject;

generating a zoomed-in image of at least one display portion within the display monitor based upon the at least one state of the subject;

receiving feedback from an electronic device indicative of a validity of the electronic communication notification;

adjusting, via a machine learning process, an electronic communication alert sensitivity level for the determined state based upon the feedback; and

determining whether to generate future electronic communication alerts for the determined state of the one or more subjects based on the adjusted electronic communication alert sensitivity level.

8. The method as recited in claim 7, further comprising filtering noise from the first subset of pixels and the second subset of pixels.

9. The method as recited in claim 7, wherein the machine learning classifier comprises a cascading Haar classifier.

10. A method for monitoring a medical care environment, the method comprising:

receiving at least one of a depth frame image, a color frame image, or an infrared frame image associated with one or more subjects, the at least one of the depth frame image, the color frame image, or the infrared frame image captured by an imaging device having an image plane;

displaying, via at least one display monitor associated with the one or more subjects, the at least one of the depth frame image, the color frame image, or the infrared frame image;

classifying, via a first machine learning classifier, a body part associated with the one or more subjects from the at least one of the depth frame image, the color frame image, or the infrared frame image to generate a first orientation signal of a body part with respect to the image plane;

classifying, via a second machine learning classifier, the at least one of the depth frame image, the color frame image, or the infrared frame image to generate a second orientation signal of the body part with respect to the image plane, wherein the second machine learning classifier is different from the first machine learning classifier;

determining a probability distribution based upon at least the first orientation signal and the second orientation signal via a fusion engine;

determining at least one state of the one or more subjects based upon the probability distribution;

generating an electronic communication notification to view the at least one display monitor based upon the at least one state of the one or more subjects;

generating a zoomed-in image of a display portion of a plurality of display portions within the at least one display monitor based upon the at least one state of the subject;

receiving feedback from an electronic device indicative of a validity of the electronic communication notification;

adjusting, via a machine learning process, an electronic communication alert sensitivity level for the determined state based upon the feedback; and

determining whether to generate future electronic communication alerts for the determined state of the one or more subjects based on the adjusted electronic communication alert sensitivity level.

11. The method as recited in claim 10, further comprising processing, via a third machine learning classifier, the at least one of the depth frame image, the color frame image, or the infrared frame image to generate a third orientation signal of the body part with respect to the image plane, wherein determining a probability distribution based upon at least the first orientation signal and the second orientation signal via a fusion engine comprises determining the probability distribution based upon at least the first orientation signal, the second orientation signal, and the third orientation signal via the fusion engine, wherein the third machine learning classifier is different from the first machine learning classifier and the second machine learning classifier.

12. The method as recited in claim 10, further comprising filtering noise from the at least one of the depth frame image, the color frame image, or the infrared frame image.

13. The method as recited in claim 10, wherein determining at least one state comprises determining an activity level associated with the subject.

14. The method as recited in claim 10, further comprising identifying a position of at least one of a second subject or an object relative to the one or more subjects.