Method and apparatus for identification

• US 9,256,776 B2
• Filed: 11/12/2012
• Issued: 02/09/2016
• Est. Priority Date: 11/18/2009
• Status: Active Grant

• Current Assignee: AIC Innovations Group, Inc.
• Original Assignee: AI CURE Technologies Incorporated
• Inventors: Hanina, Adam, Guan, Lei
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): MESSMORE, JONATHAN R
• Application Number:
  US13/674,209
• Publication Number:
  US 20130169781A1
• Time in Patent Office: 1,184 Days
• Field of Search: 348/77, 382/264
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A method and system for de-identifying a video sequence are provided. The method may include the steps of capturing a video sequence, comprising a number of individual frames, including one or more users performing one or more actions, and using activity recognition to recognize one of the one or more actions. One or more of the plurality of frames may be defined as comprising the recognized one or more actions, and a portion of the one or more of the plurality of frames may be identified to remain visible. The non-identified portions of the one or more of the plurality of frames and the non-defined frames may be de-identified. This method may be applied to the determine of whether a user has ingested a medication pill.

Claims

1. A method for de-identifying a video sequence, comprising:

capturing, in a video capture device, a video sequence comprising a plurality of individual frames, the plurality of individual frames including images of one or more users administering medication;

performing, by a processor, activity recognition on the captured video sequence to recognize the administration of medication within the plurality of individual frames;

designating one or more frames of the plurality of frames as comprising the recognized administration of medication, wherein at least one of the plurality of frames is non-designated;

identifying a first portion of the one or more designated frames to remain visible, wherein the first portion comprises the recognized administration of medication; and

de-identifying a second portion of the one or more designated frames and de-identifying the at least one non-designated frame so that the first portion comprising the administration of medication remains visible.

2. The method of claim 1, wherein the de-identifying comprises blurring the image.

3. The method of claim 1, further comprising highlighting the visible portions of the one or more plurality of frames.

4. The method of claim 1, wherein the administration of medication comprises ingestion of the medication.

5. The method of claim 1, further comprising transmitting the de-identified one or more of the plurality of frames to a remote storage and computing location.

6. The method of claim 1, wherein performing activity recognition to recognize administration of medication further comprises:

determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence; and

determining, through object recognition, the identity of the object and its surroundings in a second plurality of frames of the video sequence.

7. The method of claim 1, wherein the administration of medication comprise micro movements of one or more body parts.

8. The method of claim 1, wherein the performing activity recognition to recognize the administration of medication further comprises:

defining a plurality of detection methods for recognizing the administration of medication; and

applying each of the plurality of detection methods to a subset of the frames of the captured video sequence.

9. The method of claim 8, wherein each detection method comprises searching for an object positioned at a different orientation.

10. A method for de-identifying a video sequence of a user ingesting a medication pill, comprising the steps of:

capturing a first video sequence in a video capture device, the first video sequence comprising a plurality of individual frames, the plurality of individual frames including images of one or more users performing the action of putting the medication pill in their mouth;

performing, by a processor, activity recognition on the captured video sequence to recognize the placement of the medication pill in the mouth of the user;

designating one or more frames of the plurality of frames as comprising the recognized action of placing the medication pill in the mouth of the user, wherein at least one frame is non-designated;

identifying a first portion of the one or more designated frames to remain visible, wherein the first portion comprises the recognized placement of the medication pill in the mouth of the user; and

de-identifying a second portion of the one or more designated frames and de-identifying the at least one non-designated frame so that the first portion comprising the placement of the medication pill in the mouth of the user remains visible.

11. The method of claim 10, wherein the de-identifying comprises blurring the image.

12. The method of claim 10, further comprising highlighting the visible portions of the one or more plurality of frames.

13. The method of claim 10, further comprising transmitting the de-identified second portion and at least one non-designated frame to a remote storage and computing location.

14. The method of claim 10, wherein performing activity recognition further comprises:

determining, through object recognition, the identity of an object for a first plurality of frames of the video sequence; and

determining, through object recognition, the identity of the object and its surroundings in a second plurality of frames of the video sequence.

15. The method of claim 10, further comprising capturing a second video sequence, the second video sequence comprising a plurality of individual frames, the plurality of individual frames of the second video sequence including images of one or more users performing the action of swallowing the medication pill;

performing, by the processor, activity recognition on the second video sequence to recognize one or more micro movements of one or more body parts associated with the swallowing of the medication pill by the user;

designating one or more frames of the second video sequence as comprising the recognized one or more micro movements of the one or more body parts of the user, wherein at least one frame of the second video sequence is non-designated;

identifying a first portion of the one or more frames of the second video sequence to remain visible, wherein first portion of the one or more frames of the second video sequence comprises the recognized one or more micro movements of the one or more body parts of the user; and

de-identifying a second portion of the one or more frames of the second video sequence and de-identifying the at least one non-designated frame of the second video sequence so that the first portion of the one or more frames of the second video sequence remains visible.

16. The method of claim 10, wherein performing activity recognition to recognize the placement of the medication pill in the mouth of the user further comprises:

defining a plurality of detection methods for recognizing the placement of the medication pill in the mouth of the user; and

applying each of the plurality of detection methods to a subset of the frames of the captured first video sequence.

17. The method of claim 16, wherein each of the plurality of detection methods comprise searching for a medication pill positioned in the mouth of the user at a different orientation.

18. A system for de-identifying a video sequence of a user ingesting a medication pill, the system comprising:

a video capture device operable to capture a video sequence comprising a plurality of individual frames, the plurality of individual frames including images of one or more users performing the action of putting the medication pill in their mouth;

a processor associated with the video capture device operable to perform activity recognition on the captured video sequence to recognize the placement of the medication pill in the mouth by;

determining, through object recognition, the identity of a first object for a first plurality of frames of the video sequence,

determining, through object recognition, the identity of a second object and its surroundings in a second plurality of frames of the video sequence,

designating one or more frames of the captured video sequence as comprising the recognized action of placing the medication pill in the mouth of the user, wherein at least one frame is non-designated,

identifying a first portion of the one or more designated frames of the captured video sequence to remain visible, wherein the first portion comprises the recognized placement of the medication pill in the mouth of the user, and

de-identifying a second portion of the one or more designated frames and de-identifying the at least one non-designated frame so that the first portion remains visible.

19. The system of claim 18, wherein the de-identifying comprises blurring the image.

20. The system of claim 18, wherein the processor is further operable to apply each of a plurality of detection methods for recognizing the placement of the medication pill in the mouth of the user to a subset of the frames of the captured video sequence.