Medical environment monitoring system

• US 10,229,491 B1
• Filed: 07/14/2016
• Issued: 03/12/2019
• Est. Priority Date: 10/16/2012
• Status: Active Grant

• Current Assignee: Ocuvera LLC
• Original Assignee: Ocuvera LLC
• Inventors: Rush, Benjamin D., Brown-Kramer, Joshua M., Sabalka, Lucas A., Gansemer, Brian S., Upton, Clay T., Bauer, Paul B., Unterseher, Andrew R., Origas, Benjamin J., Durham, Douglas W.
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US15/210,478
• Time in Patent Office: 971 Days
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A system and a method are described for monitoring a medical care environment. In one or more implementations, a method includes identifying a plurality of potential locations within a field of view of a camera as representing a bed and/or a seating platform. The method also includes identifying a first subset of pixels from the plurality of potential locations as representing a bed and/or a seating platform. The method also includes identifying a second subset of pixels within the field of view of the camera as representing an object (e.g., a subject, such as a patient, medical personnel; bed; chair; patient tray; medical equipment; etc.) proximal to the bed and/or seating platform. The method also includes determining an orientation of the object with respect to the bed and/or seating platform. In implementations, the method further includes issuing an electronic communication alert based upon the orientation of the object.

Claims

1. A system for monitoring a medical care environment, the system comprising:

an imaging device having an imaging plane, the imaging device configured to receive a plurality of depth frame images comprising a plurality of pixels, the plurality of depth frame images representing a three-dimensional environment corresponding to a physical space, the physical space including at least one of a bed or a seating platform;

at least one computing device in communication with the imaging device, the at least one computing device including;

a memory configured to store one or more modules;

a processor coupled to the memory, the processor configured by a fusion engine and the one or more modules to cause the processor to;

generate a depth mask representing the physical space based upon the plurality of depth frame images

identify, via one or more object location algorithms, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask;

identify, via one or more object identification algorithms, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations;

identify one or more second pixels of the plurality of pixels as representing a portion of a body of a subject proximate to the at least one of the bed or the seating platform;

process, via a first processing technique, the one or more second pixels to generate a first possible orientation signal of the portion of the body with respect to the image plane, the first possible orientation signal representing a position of the portion of the body with respect to the at least one of the bed or the seating platform;

process, via a second processing technique via the fusion engine, the one or more second pixels to generate a second possible orientation signal of the portion of the body with respect to the image plane, the second possible orientation signal representing the position of the portion of the body with respect to the at least one of the bed or the seating platform;

determine a probability distribution based upon the first possible orientation signal and the second possible orientation signal via the fusion engine;

determine a transitional state of the subject based upon the probability distribution; and

issue an electronic communication alert based upon the determined transitional state of the subject.

2. The system as recited in claim 1, wherein the processor configured by a fusion engine and the one or more modules to cause the processor identify, via one or more object location algorithms, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask comprises generating a Boolean array from the depth mask and identifying maximal rectangles that represent at least one of a plurality of potential bed locations or a plurality of potential seating platform locations.

3. The system as recited in claim 1, wherein the processor configured by a fusion engine and the one or more modules to cause the processor to identify, via one or more object identification algorithms, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations comprises modeling, via one or more gradient ascent algorithms, the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations and selecting the at least one of the plurality of potential bed locations or the plurality of potential seating platform with the highest fitness as representing the at least one of the bed or the seating platform.

4. The system as recited in claim 1, wherein the processor is further configured to execute the one or more modules to cause the processor to receive feedback about a validity of the electronic communication alert.

5. The system as recited in claim 1, wherein the processor is further configured to:

receive feedback about an accuracy of at least one of the depth mask, the one or more first pixels, or the one or more second pixels; and

dynamically adjust at least one of the depth mask, the one or more object identification algorithms, the one or more object location algorithms, the one or more first pixels, or the one or more second pixels.

6. The system as recited in claim 1, wherein the processor is further configured to execute the one or more modules to cause the processor to process the depth frame image to detect at least one of a presence or an absence of a gesture;

and provide an indication associated with the at least one of the presence or the absence of the gesture.

7. The system as recited in claim 1, wherein the processor is further configured to determine an electronic communication alert sensitivity level associated with the subject based upon at least one characteristic of the subject.

8. The system as recited in claim 1, wherein the at least one computing device comprises a computer cluster, the computer cluster comprising a plurality of cluster nodes configured to access a shared memory.

9. A method for monitoring a subject within a medical care environment, the method comprising:

generating, via a processor configured by a fusion engine, a depth mask representing the medical care environment based upon a plurality of depth frame images received from an imaging device, the plurality of depth frame images representing a three-dimensional environment corresponding to a physical space, the physical space including at least one of a bed or a seating platform, the plurality of depth frame images including a plurality of pixels;

identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask;

identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations;

identifying, via the processor, one or more second pixels of the plurality of pixels as representing a portion of a body of a subject proximate to the at least one of the bed or the seating platform;

processing, via a first processing technique, the one or more second pixels to generate a first possible orientation signal of the portion of the body with respect to the image plane, the first possible orientation signal representing a position of the portion of the body with respect to the at least one of the bed or the seating platform;

processing, via a second processing technique via the fusion engine, the one or more second pixels to generate a second possible orientation signal of the portion of the body with respect to the image plane, the second possible orientation signal representing the position of the portion of the body with respect to the at least one of the bed or the seating platform;

determining a probability distribution based upon the first possible orientation signal and the second possible orientation signal via the fusion engine;

determining a transitional state of the subject based upon the probability distribution; and

issuing an electronic communication alert based upon the determined transitional state of the subject.

10. The method as recited in claim 9, wherein identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask comprises generating a Boolean array from the depth mask and identifying maximal rectangles that represent at least one of a plurality of potential bed locations or a plurality of potential seating platform locations.

11. The method as recited in claim 9, wherein identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations comprises modeling, via one or more gradient ascent algorithms, the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations and selecting the at least one of the plurality of potential bed locations or the plurality of potential seating platform with the highest fitness as representing the at least one of the bed or the seating platform.

12. The method as recited in claim 9, further comprising:

tracking the one or more second pixels through the plurality of depth frame images;

determining an activity parameter of the subject based upon the tracked one or more second pixels; and

issuing an activity electronic communication alert indicating the activity parameter when the activity parameter is outside of an activity threshold.

13. The method as recited in claim 9, further comprising receiving feedback about a validity of the electronic communication alert.

14. The method as recited in claim 9, further comprising determining an electronic communication alert sensitivity level associated with the subject based upon at least one characteristic of the subject.

15. A method for monitoring a medical care environment, the method comprising:

receiving a plurality of depth frame images comprising a plurality of pixels the plurality of depth frame images captured by an imaging device having an image plane, the plurality of depth frame images representing a three-dimensional environment corresponding to a physical space, the physical space including at least one of a bed or a seating platform;

generating, via a processor configured by a fusion engine, a depth mask representing the physical space based upon a plurality of depth frame images received from an imaging device;

identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask;

identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations;

identifying, via the processor, one or more second pixels of the plurality of pixels as representing a body part of a subject proximate to the at least one of the bed or the seating platform;

processing, via a first processing technique, the one or more second pixels to generate a first possible orientation signal of the body part with respect to the image plane, the first possible orientation signal representing a position of the body part with respect to the at least one of the bed or the seating platform;

processing, via a second processing technique via the fusion engine, the one or more second pixels to generate a second possible orientation signal of the body part with respect to the image plane, the second possible orientation signal representing the position of the body part with respect to the at least one of the bed or the seating platform;

determining, via the processor, a probability distribution based upon the first possible orientation signal and the second possible orientation signal via the fusion engine;

determining a transitional state of the subject based upon the probability distribution; and

issuing an electronic communication alert based upon the determined transitional state of the subject.

16. The method as recited in claim 15, further comprising:

receiving feedback about an accuracy of at least one of the depth mask, the one or more first pixels, or the one or more second pixels; and

dynamically adjusting at least one of the generating, via a processor configured by a fusion engine, a depth mask representing the physical space, the identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels, or the identifying, via the processor, one or more second pixels of the plurality of pixels in response to the feedback received.

17. The method as recited in claim 15, wherein identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask comprises generating a Boolean array from the depth mask and identifying maximal rectangles that represent at least one of a plurality of potential bed locations or a plurality of potential seating platform locations.

18. The method as recited in claim 15, further comprising:

processing, via a third processing technique via the fusion engine, the one or more second pixels to generate a third possible orientation signal of the body part with respect to the image plane, the third possible orientation signal representing the position of the body part with respect to the at least one of the bed or the seating platform;

processing, via a fourth processing technique via the fusion engine, the one or more second pixels to generate a fourth possible orientation signal of the body part with respect to the image plane, the fourth possible orientation signal representing the position of the body part with respect to the at least one of the bed or the seating platform; and

determine the probability distribution based upon the first possible orientation signal, the second possible orientation signal, the third possible orientation signal, and the fourth possible orientation signal via the fusion engine.

19. The method as recited in claim 15, further comprising identifying one or more third pixels of the plurality of pixels as representing at least one of a second subject or an object relative to the subject.

20. The method as recited in claim 19, further comprising:

receiving feedback about an accuracy of the one or more third pixels; and

dynamically adjusting the identifying one or more third pixels of the plurality of pixels as representing at least one of a second subject or an object relative to the subject.