Systems and method for activity monitoring

• US 9,710,700 B2
• Filed: 09/15/2016
• Issued: 07/18/2017
• Est. Priority Date: 02/26/2013
• Status: Expired due to Fees

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Hyde, Roderick A., Kare, Jordin T., Wood, Victoria Y. H., Young, Christopher Charles, Peterson, Nels R., Bilet, Maxime Jean Jerome, Wood, Jr., Lowell L., Ishikawa, Muriel Y., Myhrvold, Nathan P., Pan, Tony S., Tegreene, Clarence T.
• Primary Examiner(s): Singh, Gurkanwaljit
• Application Number:
  US15/266,966
• Publication Number:
  US 20170004357A1
• Time in Patent Office: 306 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities
• CPC Class Codes: G06Q 30/018 Certifying business or products
• CPC Class Codes: G06Q 50/12 Hotels or restaurants
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

A system for activity monitoring includes a tracking component, an activity identification component, a procedural component, and a notification component. The tracking component tracks an individual and one or more objects in a work area using a three-dimensional tracking system. The activity identification component identifies an activity of the individual and any of the one or more objects that are affected by the activity. The procedural component determines whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects. The notification component provides a notification of a violation.

Claims

1. A non-transitory computer-readable storage medium comprising program code for causing one or more processors to perform a method for monitoring an activity, the method comprising:

tracking an individual in a work area using a three-dimensional tracking system comprising at least a camera to sense a movement of a second portion of the individual that is in view of the camera and a processor to predict a position of a first portion of the individual that is out of view of the camera based on the movement of the second portion of the individual;

tracking one or more object in the work area using the three-dimensional tracking system and a sensor to obtain data corresponding to the one or more objects within the work area;

identifying, using an activity monitoring device comprising a computing device with a processor, an activity of the individual and any of the one or more objects that are affected by the activity based on at least one of the movement of the second portion of the individual, the predicted position of the first portion of the individual, and the data corresponding to the one or more objects obtained by the sensor;

determining, using the activity monitoring device comprising the computing device with the processor, whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects; and

providing a notification of a violation to an output device.

2. The computer-readable storage medium of claim 1, wherein tracking the one or more objects comprises tracking a food item.

3. The computer-readable storage medium of claim 1, wherein tracking the one or more objects comprises tracking tableware.

4. The computer-readable storage medium of claim 1, wherein tracking the one or more objects comprises tracking a cleaning implement.

5. The computer-readable storage medium of claim 1, wherein tracking an individual and one or more objects in a work area comprises tracking the individual and the one or more objects in a kitchen.

6. The computer-readable storage medium of claim 1, wherein tracking comprises tracking an individual and one or more objects in a work area logically divided into two or more logical areas.

7. The computer-readable storage medium of claim 6, wherein the two or more logical areas comprise a raw meat area.

8. The computer-readable storage medium of claim 6, wherein the two or more logical areas comprise a hand washing area.

9. The computer-readable storage medium of claim 1, wherein the identified activity comprises contact between the individual and an object of the one or more objects.

10. The computer-readable storage medium of claim 1, wherein the identified activity comprises contact between an individual and a surface of the work area.

11. The computer-readable storage medium of claim 1, wherein the identified activity comprises contact between an object of the one or more objects and a surface of the work area.

12. The computer-readable storage medium of claim 1, wherein the identified activity comprises an activity having a risk of contaminating an object of the one or more objects.

13. The computer-readable storage medium of claim 1, wherein the identified activity comprises an activity having a risk of contaminating at least a portion of the work area.

14. The computer-readable storage medium of claim 1, wherein a procedural rule of the one or more procedural rules defines violation of a food handling procedure and determining comprises determining whether the activity violates the food handling procedure.

15. The computer-readable storage medium of claim 1, wherein the procedural rule defines a combination of one or more work area conditions that comprise a violation and wherein determining whether the activity violates the procedural rule comprises comparing current work area conditions to the defined combination of work area conditions.

16. The computer-readable storage medium of claim 1, wherein providing a notification comprises indicating in the work area an occurrence of a violation in response to determining that the activity violates a procedural rule of the one or more procedural rules.

17. The computer-readable storage medium of claim 16, wherein indicating comprises visually indicating the occurrence of the violation.

18. The computer-readable storage medium of claim 17, wherein visually indicating comprises spotlighting one or more of an object of the one or more objects, the individual, and a portion of the work area using a spotlight.

19. The computer-readable storage medium of claim 17, wherein visually indicating comprises displaying on a display a diagram of the work area indicating the occurrence of the violation.

20. The computer-readable storage medium of claim 16, wherein indicating comprises audibly playing a sound indicating the occurrence of the violation.

21. A system for activity monitoring, comprising:

a tracking component to track an individual in a work area using a three-dimensional tracking system comprising at least a camera to sense a movement of a second portion of the individual that is in view of the camera and a processor to predict a position of a first portion of the individual that is out of view of the camera based on the movement of the second portion of the individual, and track one or more objects in the work area using the three-dimensional tracking system;

a sensor to obtain data corresponding to the one or more objects in the work area;

a computing device with a processor, the computing device including;

an activity identification component to identify, an activity of the individual and any of the one or more objects that are affected by the activity based on at least one of the movement of the second portion of the individual, the predicted position of the first portion of the individual, and the data corresponding to the objects obtained by the sensor; and

a procedural component to determine whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects; and

a notification component to provide a notification of a violation using a visual output within the work area.

22. The system of claim 21, further comprising a tracking storage component to store information regarding tracking of the individual and the one or more objects in the work area.

23. The system of claim 22, wherein the tracking storage component stores video footage of the work area.

24. The system of claim 21, wherein the three-dimensional tracking system comprises a tracer detection component to detect a tracer material, the tracer material comprising a material that transfers to a contacting surface in response to contact with the tracer material.

25. The system of claim 24, wherein the three-dimensional tracking system comprises a tracer trigger component to trigger detectability of the tracer material.

26. The system of claim 21, wherein the notification component provides the notification by automatically issuing a citation in response to the procedural component determining that the activity violates the one or more procedural rules.