Apparatus and method for determination of medication location
First Claim
1. A method for confirming a pill in the mouth of a user, comprising:
- predefining one or more steps associated with confirming a pill in the mouth of a user, the one or more steps comprising at least proper identification of a medication pill, proper positioning of the face of the user, proper positioning of the open mouth of the user, and proper showing of whether there is a pill in the mouth of the user;
displaying on a display one or more guidance instructions for properly positioning of a face of the user relative to the image capture camera;
capturing a first set of one or more video frames by an image capture camera associated with the display;
determining, by one or more processors in near real time from the first set of one or more video frames, that the user has properly positioned the face of the user in response to the one or more guidance instructions;
confirming, by the one or more processors in near real time, that the face of the user is an expected face;
in response to a confirmation that the face of the user is the expected face, displaying on the display one or more guidance instructions for positioning the medication pill relative to the image capture camera upon determination that the user has properly positioned the face of the user;
capturing a second set of one or more video frames by the image capture device;
determining, by the one or more processors in near real time, that the user has properly positioned the medication pill within the field of view in response to the one or more guidance instructions for positioning the medication pill, and that the medication pill is an expected medication pill from the second set of one or more video frames;
setting a bottom portion of the face of the user as a region of interest, the region of interest including the mouth of the user, responsive to determining that the user has properly positioned their face in response to the one or more guidance instructions for positioning of the face of the user, that the user is the expected user, and that the medication pill is the expected medication pill;
determining, by the one or more processors in near real time, that the mouth of the user in the region of interest is an open mouth from the second set of one or more video frames;
subsequent to determining that the mouth of the user is the open mouth, extracting an image of the open mouth from the second set of one or more video frames, and classifying, by the one or more processors, the open mouth from the extracted image as a mouth without a pill therein;
responsive to classifying the open mouth as the mouth without a pill therein, displaying on the display guidance for positioning the open mouth of the user relative to the image capture camera; and
displaying on the display guidance to the user to place the medication pill in their mouth.
3 Assignments
0 Petitions
Accused Products
Abstract
A method, apparatus and computer program stored on a non-volatile computer readable storage medium for confirming a pill in the mouth of a user. The computer program causing a general purpose computer to perform the steps of capturing one or more images of a user by an image capture device, confirming the position of the face of the user within the captured image by measuring a size of the face, and setting a predetermined portion of the face of the user to be a region of interest. An open mouth of the user is confirmed within the region of interest, and the open mouth of the user is classified as one of a mouth with a pill therein and a mouth without a pill therein.
147 Citations
18 Claims
-
1. A method for confirming a pill in the mouth of a user, comprising:
-
predefining one or more steps associated with confirming a pill in the mouth of a user, the one or more steps comprising at least proper identification of a medication pill, proper positioning of the face of the user, proper positioning of the open mouth of the user, and proper showing of whether there is a pill in the mouth of the user; displaying on a display one or more guidance instructions for properly positioning of a face of the user relative to the image capture camera; capturing a first set of one or more video frames by an image capture camera associated with the display; determining, by one or more processors in near real time from the first set of one or more video frames, that the user has properly positioned the face of the user in response to the one or more guidance instructions; confirming, by the one or more processors in near real time, that the face of the user is an expected face; in response to a confirmation that the face of the user is the expected face, displaying on the display one or more guidance instructions for positioning the medication pill relative to the image capture camera upon determination that the user has properly positioned the face of the user; capturing a second set of one or more video frames by the image capture device; determining, by the one or more processors in near real time, that the user has properly positioned the medication pill within the field of view in response to the one or more guidance instructions for positioning the medication pill, and that the medication pill is an expected medication pill from the second set of one or more video frames; setting a bottom portion of the face of the user as a region of interest, the region of interest including the mouth of the user, responsive to determining that the user has properly positioned their face in response to the one or more guidance instructions for positioning of the face of the user, that the user is the expected user, and that the medication pill is the expected medication pill; determining, by the one or more processors in near real time, that the mouth of the user in the region of interest is an open mouth from the second set of one or more video frames; subsequent to determining that the mouth of the user is the open mouth, extracting an image of the open mouth from the second set of one or more video frames, and classifying, by the one or more processors, the open mouth from the extracted image as a mouth without a pill therein; responsive to classifying the open mouth as the mouth without a pill therein, displaying on the display guidance for positioning the open mouth of the user relative to the image capture camera; and displaying on the display guidance to the user to place the medication pill in their mouth. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A system for confirming a pill in the mouth of a user the system comprising:
-
an image acquisition device operable to capture one or more video frames of a user; a display operable to display guidance for positioning of a face of the user relative to the image acquisition device in accordance with the captured one or more video frames, to display one or more guidance instructions for performing one or more steps associated with proper medication pill positioning; and a processor operable to; output, to the display, the one or more guidance instructions for positioning the face of the user relative to the image acquisition device in accordance with the captured one or more video frames, and guidance for positioning the open mouth of the user relative to the image acquisition device, receive, from the image acquisition device, a first one or more video frames of the user performing one or more steps associated with properly positioning the face of the user in response to the display on the display of the guidance instructions for positioning the face of the user and the open mouth of the user; determine in near real time whether the user has properly positioned their face within the field of view in response to the one or more guidance instructions for positioning the face of the user; receive, from the image acquisition device a second one or more video frames of the user; determine, in accordance with the second one or more video frames in near real time, whether the user has properly positioned the medication pill in response to the guidance instructions for performing one or more steps associated with proper medication pill positioning, thus allowing the processor to further confirm that the medication pill is an expected medication pill; set a bottom portion of the face of the user as a region of interest, the region of interest including the mouth of the user, responsive to determining that the user has properly positioned their face in response to the one or more guidance instructions for positioning of the face of the user, classify in near real time, based on the second one or more video frames, the mouth of the user in the region of interest is an open mouth, subsequent to determining that the mouth of the user is the open mouth, extracting an image of the open mouth from the second one or more video frames, and classifying the open mouth from the extracted image as one of a mouth with a pill therein and a mouth without a pill therein, and output, to the display, guidance to the user to place a medication pill in their mouth when the open mouth of the user is classified as a mouth without a pill therein. - View Dependent Claims (7, 8, 9, 10, 11, 12)
-
-
13. A non-volatile computer readable storage medium having a computer program stored thereon, the computer program, when executed by a computer, causing the computer to perform the actions of:
-
predefining one or more steps associated with confirming a pill in the mouth of a user, the one or more steps comprising at least proper identification of a medication pill, proper positioning of the face of the user, proper positioning of the open mouth of the user, and proper showing of whether there is a pill in the mouth of the user; displaying on a display one or more guidance instructions for properly positioning of a face of the user relative to the image capture camera; capturing a first set of one or more video frames by an image capture camera associated with the display; determining, in near real time from the first set of one or more video frames, whether the user has properly positioned the face of the user relative to the image capture camera in response to the one or more guidance instructions; confirming, in near real time, that the face of the user is an expected face; in response to a confirmation that the face of the user is the expected face, displaying on the display one or more guidance instructions for positioning of the medication pill relative to the image capture camera upon determination that the user has properly positioned the face of the user; capturing a second set of one or more video frames of a user by an image capture device associated with the display; determining, in near real time, whether the user has properly positioned the medication pill within the field of view in response to the one or more guidance instructions for positioning the medication, and that the medication pill is an expected medication pill from the second set of one or more video frames; setting a bottom portion of the face of the user as a region of interest, the region of interest including the mouth of the user, responsive to determining that the user has properly positioned their face in response to the one or more guidance instructions for positioning of the face of the user, that the user is the expected user, and that the medication pill is the expected medication pill; capturing a third set of one or more video frames by the image capture device; confirming that the mouth of the user is an open mouth of the user within the region of interest from the third set of one or more video frames in near real time; subsequent to determining that the mouth of the user is the open mouth, extracting an image of the open mouth from the third set of one or more video frames, and classifying the open mouth of the user as one of a mouth with a pill therein and a mouth without a pill therein in near real time; and responsive to classifying the open mouth of the user as the mouth without a pill therein, displaying on the display guidance for positioning the open mouth of the user relative to the image capture device and displaying on the display guidance to the user to place the medication pill in their mouth. - View Dependent Claims (14, 15, 16, 17, 18)
-
Specification