System and method for activity monitoring

• US 9,449,219 B2
• Filed: 02/26/2013
• Issued: 09/20/2016
• Est. Priority Date: 02/26/2013
• Status: Expired due to Fees

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Bilet, Maxime Jean Jerome, Hyde, Roderick A., Ishikawa, Muriel Y., Kare, Jordin T., Myhrvold, Nathan P., Pan, Tony S., Peterson, Nels R., Tegreene, Clarence T., Wood, Lowell L. Jr., Wood, Victoria Y. H., Young, Christopher Charles
• Primary Examiner(s): Singh, Gurkanwaljit
• Application Number:
  US13/777,548
• Publication Number:
  US 20140244344A1
• Time in Patent Office: 1,302 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities
• CPC Class Codes: G06Q 30/018 Certifying business or products
• CPC Class Codes: G06Q 50/12 Hotels or restaurants
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

A system for activity monitoring includes a tracking component, an activity identification component, a procedural component, and a notification component. The tracking component tracks an individual and one or more objects in a work area using a three-dimensional tracking system. The activity identification component identifies an activity of the individual and any of the one or more objects that are affected by the activity. The procedural component determines whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects. The notification component provides a notification of a violation.

Claims

1. A non-transitory computer-readable storage medium comprising program code for causing one or more processors to perform a method for activity monitoring, the method comprising:

tracking an individual using a three-dimensional tracking system comprising at least a camera to sense a movement of a second portion of the individual that is in view of the camera and a processor to predict a position of a first portion of the individual that is out of view of the three-dimensional tracking system based on the movement of the second portion of the individual;

tracking one or more objects in a work area using the three-dimensional tracking system comprising at least a sensor embedded in each of the one or more objects;

identifying, using the three-dimensional tracking system, an activity of the individual and any of the one or more objects that are affected by the activity;

determining, using an activity monitoring device comprising a computing device with a processor, whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects; and

providing a notification of a violation to an output device.

2. The computer-readable storage medium of claim 1, wherein tracking the one or more objects comprises tracking a food item.

3. The computer-readable storage medium of claim 1, wherein tracking the one or more objects comprises tracking tableware.

4. The computer-readable storage medium of claim 1, wherein tracking the one or more objects comprises tracking a cleaning implement.

5. The computer-readable storage medium of claim 1, wherein tracking an individual and one or more objects in a work area comprises tracking the individual and the one or more objects in a kitchen.

6. The computer-readable storage medium of claim 1, wherein tracking comprises tracking an individual and one or more objects in a work area logically divided into two or more logical areas.

7. The computer-readable storage medium of claim 6, wherein the two or more logical areas comprise a raw meat area.

8. The computer-readable storage medium of claim 6, wherein the two or more logical areas comprise a hand washing area.

9. The computer-readable storage medium of claim 1, wherein the identified activity comprises contact between the individual and an object of the one or more objects.

10. The computer-readable storage medium of claim 1, wherein the identified activity comprises contact between an individual and a surface of the work area.

11. The computer-readable storage medium of claim 1, wherein the identified activity comprises contact between an object of the one or more objects and a surface of the work area.

12. The computer-readable storage medium of claim 1, wherein the identified activity comprises an activity having a risk of contaminating an object of the one or more objects.

13. The computer-readable storage medium of claim 1, wherein the identified activity comprises an activity having a risk of contaminating at least a portion of the work area.

14. The computer-readable storage medium of claim 1, wherein a procedural rule of the one or more procedural rules defines violation of a food handling procedure and determining comprises determining whether the activity violates the food handling procedure.

15. The computer-readable storage medium of claim 1, wherein the procedural rule defines a combination of one or more work area conditions that comprise a violation and wherein determining whether the activity violates the procedural rule comprises comparing current work area conditions to the defined combination of work area conditions.

16. The computer-readable storage medium of claim 1, wherein providing a notification comprises indicating in the work area an occurrence of a violation in response to determining that the activity violates a procedural rule of the one or more procedural rules.

17. The computer-readable storage medium of claim 16, wherein indicating comprises visually indicating the occurrence of the violation.

18. The computer-readable storage medium of claim 17, wherein visually indicating comprises spotlighting one or more of an object of the one or more objects, the individual, and a portion of the work area using a spotlight.

19. The computer-readable storage medium of claim 17, wherein visually indicating comprises displaying on a display a diagram of the work area indicating the occurrence of the violation.

20. The computer-readable storage medium of claim 16, wherein indicating comprises audibly playing a sound indicating the occurrence of the violation.

21. The computer-readable storage medium of claim 1, the method further comprising maintaining a history for the one or more objects, the individual, and portions of the work area.

22. The computer-readable storage medium of claim 21, wherein determining whether the activity violates a procedural rule is based on the history.

23. The computer-readable storage medium of claim 21, the method further comprising determining a contamination status of one or more of an object of the one or more objects, a portion of the work area, and the individual based on the history.

24. The computer-readable storage medium of claim 1, the method further comprising maintaining a database of violating activities and one or more work area conditions associated with each violating activity.

25. The computer-readable storage medium of claim 24, the method further comprising analyzing the database to identify a combination of the associated workplace conditions having a correlation to a violating activity.

26. The computer-readable storage medium of claim 24, the method further comprising analyzing the database to identify a procedure having a high correlation to a violating activity.

27. The computer-readable storage medium of claim 24, the method further comprising analyzing the database to identify a physical layout feature having a high correlation to a violating activity.

28. The computer-readable storage medium of claim 24, the method further comprising analyzing the database to identify an individual having a high correlation to a violating activity.

29. The computer-readable storage medium of claim 24, the method further comprising detecting a combination of the associated workplace conditions having a correlation to a violating activity and providing a warning.

30. A system for activity monitoring, the system comprising:

a tracking component to;

track an individual in a work area using a three-dimensional tracking system comprising at least a camera to sense a movement of a second portion of the individual that is in view of the camera and a processor to predict a position of a first portion of the individual that is out of view of the three-dimensional tracking system based on the movement of the second portion of the individual;

tracking one or more objects in a work area using the three-dimensional tracking system comprising at least a sensor embedded in each of the one or more objects;

an activity identification component to identify, using the three-dimensional tracking system, an activity of the individual and any of the one or more objects that are affected by the activity;

a procedural component comprising a computing device with a processor to determine whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects; and

a notification component to provide a notification of a violation using a visual output within the work area.

31. The system of claim 30, further comprising a tracking storage component to store information regarding tracking of the individual and the one or more objects in the work area.

32. The system of claim 31, wherein the tracking storage component stores video footage of the work area.

33. The system of claim 32, wherein the tracking storage component stores video footage comprising video footage corresponding to a violation.

34. The system of claim 31, wherein the tracking storage component stores abstracted information regarding one or more of the work area, the activity of the individual, and the one or more objects.

35. The system of claim 30, wherein the three-dimensional tracking system comprises a tracer detection component to detect a tracer material, the tracer material comprising a material that transfers to a contacting surface in response to contact with the tracer material.

36. The system of claim 35, wherein the three-dimensional tracking system comprises a tracer trigger component to trigger detectability of the tracer material.

37. The system of claim 30, wherein the notification component provides the notification by automatically issuing a citation in response to the procedural component determining that the activity violates the one or more procedural rules.

38. The system of claim 37, wherein the citation indicates violation of a health violation.

39. A method for activity monitoring, the method comprising:

tracking an individual using a three-dimensional tracking system comprising at least a camera to sense a movement of a second portion of the individual that is in view of the camera and a processor to predict a position of a first portion of the individual that is out of view of the three-dimensional tracking system based on the movement of the second portion of the individual;

tracking one or more objects in a work area using the three-dimensional tracking system comprising at least a sensor embedded in each of the one or more objects;

identifying, using the three-dimensional tracking system, an activity of the individual and any of the one or more objects that are affected by the activity;

determining, using an activity monitoring device comprising a computing device with a processor, whether the activity violates one or more procedural rules pertaining to one or more of the individual, the work area, and the one or more objects; and

providing a notification of a violation using a visual output within the work area.