METHOD AND APPARATUS FOR IDENTIFICATION

• US 20190205615A1
• Filed: 11/13/2018
• Published: 07/04/2019
• Est. Priority Date: 01/04/2012
• Status: Active Grant

• Current Assignee: AIC Innovations Group, Inc.
• Original Assignee: AIC Innovations Group, Inc.
• Inventors: Hanina, Adam, Guan, Lei
• Granted Patent: US 10,565,431 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A method and system for de-identifying a video sequence are provided. The method may include the steps of capturing a video sequence, comprising a number of individual frames, including one or more users performing one or more actions, and using activity recognition to recognize one of the one or more actions. One or more of the plurality of frames may be defined as comprising the recognized one or more actions, and a portion of the one or more of the plurality of frames may be identified to remain visible. The non-identified portions of the one or more of the plurality of frames and the non-defined frames may be de-identified. This method may be applied to the determination of whether a user has ingested a medication pill.

Claims

1. A method for de-identifying a video sequence, comprising the steps of:

capturing a video sequence, comprising a plurality of individual frames, including one or more users performing one or more actions;

using activity recognition to recognize one of the one or more actions;

defining one or more of the plurality of frames as comprising the recognized one or more actions;

identifying a portion of the one or more of the plurality of frames to remain visible; and

de-identifying the non-identified portions of the one or more of the plurality of frames and the non-defined frames.

2. The method of claim 1, wherein the de-identifying comprises blurring the image.

3. The method of claim 1, further comprising the step of highlighting the visible portions of the one or more plurality of frames.

4. The method of claim 1, wherein the one or more actions comprises ingestion of a medication.

5. The method of claim 1, further comprising the step of transmitting the de-identified one or more of the plurality of frames to a remote storage and computing location.

6. The method of claim 1, wherein the step of using activity recognition to recognize one of the one or more actions further comprises the steps of:

determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence; and

determining, through object recognition, the identity of an object and its surroundings in a second plurality of frames of the video sequence.

7. The method of claim 1, wherein the one or more actions comprise micro movements of one or more body parts.

8. The method of claim 1, wherein the step of using activity recognition to recognize one of the one or more actions further comprises the steps of:

defining a plurality of detection methods for recognizing one of the one or more actions; and

applying each of the plurality of detection methods to a subset of the frames of the captured video sequence.

9. The method of claim 8, wherein each of the plurality of detection methods comprise searching for an object positioned at a different orientation.

16. The method of claim 8, wherein the step of using activity recognition to recognize the placement of the medication pill in the mouth of the user further comprises the steps of:

defining a plurality of detection methods for recognizing the placement of the medication pill in the mouth of the user; and

applying each of the plurality of detection methods to a subset of the frames of the captured video sequence.

17. The method of claim 16, wherein each of the plurality of detection methods comprise searching for a medication pill positioned in the mouth of the user at a different orientation.

10. A method for de-identifying a video sequence of a user ingesting a medication pill, comprising the steps of:

capturing a video sequence, comprising a plurality of individual frames, including one or more users performing the action of putting the medication pill in their mouth;

using activity recognition to recognize the placement of the medication pill in the mouth of the user;

defining one or more of the plurality of frames as comprising the recognized action of placing the medication pill in the mouth of the user;

identifying a portion of the one or more of the plurality of frames to remain visible corresponding to the portion of the one or more of the plurality of frames corresponding to the location of placing the medication pill in the mouth of the user; and

de-identifying the non-identified portions of the one or more of the plurality of frames and the non-defined frames so that the portion of the one or more of the plurality of frames corresponding to the location of placing the medication pill in the mouth of the user remains visible.

11. The method of claim 10, wherein the de-identifying comprises blurring the image.

12. The method of claim 10, further comprising the step of highlighting the visible portions of the one or more plurality of frames.

13. The method of claim 10, further comprising the step of transmitting the de-identified one or more of the plurality of frames to a remote storage and computing location.

14. The method of claim 10, wherein the step of using activity recognition to recognize one of the one or more actions further comprises the steps of:

determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence; and

determining, through object recognition, the identity of an object and its surroundings in a second plurality of frames of the video sequence.

15. The method of claim 10, further comprising the steps of capturing a video sequence, comprising a plurality of individual frames, including one or more users performing the action of swallowing the medication pill;

using activity recognition to recognize one or more micro movements of one or more body parts associated with the swallowing of the medication pill by the user;

defining one or more of the plurality of frames as comprising the recognized one or more micro movements of the one or more body parts of the user;

identifying a portion of the one or more of the plurality of frames to remain visible corresponding to the portion of the one or more of the plurality of frames corresponding to one or more micro movements of the one or more body parts of the user; and

de-identifying the non-identified portions of the one or more of the plurality of frames and the non-defined frames so that the portion of the one or more of the plurality of frames corresponding to one or more micro movements of the one or more body parts the of the user remains visible.

18. A system for de-identifying a video sequence of a user ingesting a medication pill, comprising the steps of:

a video capture device for capturing a video sequence, comprising a plurality of individual frames, including one or more users performing the action of putting the medication pill in their mouth;

a processor associated with the video capture device using activity recognition to recognize the placement of the medication pill in the mouth by determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence, and determining, through object recognition, the identity of an object and its surroundings in a second plurality of frames of the video sequence, defining one or more of the plurality of frames as comprising the recognized action of placing the medication pill in the mouth of the user, identifying a portion of the one or more of the plurality of frames to remain visible corresponding to the portion of the one or more of the plurality of frames corresponding to the location of placing the medication pill in the mouth of the user, and de-identifying the non-identified portions of the one or more of the plurality of frames and the non-defined frames so that the portion of the one or more of the plurality of frames corresponding to the location of placing the medication pill in the mouth of the user remains visible.

19. The system of claim 18, wherein the de-identifying comprises blurring the image.

20. The system of claim 18, wherein the processor further applies each of a plurality of detection methods for recognizing the placement of the medication pill in the mouth of the user to a subset of the frames of the captured video sequence.