Activity recognition in multi-entity environments

• US 9,460,350 B2
• Filed: 06/29/2012
• Issued: 10/04/2016
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: Washington State University Office Of Commercialization, WSU Research Foundation
• Original Assignee: Washington State University
• Inventors: Crandall, Aaron Spence, Cook, Diane J.
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Traktovenko, Ilya
• Application Number:
  US13/538,882
• Publication Number:
  US 20130006899A1
• Time in Patent Office: 1,558 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A physical environment is equipped with a plurality of non-obtrusive sensors (e.g., motion sensors). As a plurality of residents perform various activities within the physical environment, sensor readings are received from one or more of the sensors. Based on the sensor readings, each of the plurality of residents is identified and locations of each of the plurality of residents are tracked.

Claims

1. A method comprising:

defining a graph including vertices and edges, wherein each vertex corresponds to a sensor in a physical environment and each edge represents a direct path between two vertices with no other vertices between the two vertices along the path;

defining a set of logical rules for identifying and tracking entities within the physical environment based on the graph, wherein the rules include;

a rule for determining that an entity has moved from one location in the physical environment to another location within the physical environment based at least in part on the edges of the graph; and

a rule for determining that the entity has moved from the one location in the physical environment to the another location within the physical environment based at least in part on the entity moving more recently within the physical environment than another entity that is a candidate for a detected move to the another location;

receiving first data from a first sensor in the physical environment, the first sensor corresponding to a first vertex in the graph, the first sensor including a first type of sensor;

determining a current location of a first entity based at least in part on the first data;

receiving second data from a second sensor in the physical environment, the second sensor corresponding to a second vertex in the graph;

determining a current location of a second entity based at least in part on the second data;

receiving third data from a third sensor in the physical environment, the third sensor corresponding to a third vertex in the graph, the third sensor including a second, different type of sensor, wherein the third vertex is connected to the first vertex by a first edge of the graph and the third vertex is connected to the second vertex by a second edge of the graph; and

executing instructions on a computer processor to update the current location of the first entity within the physical environment based at least in part on the first vertex being connected to the third vertex by the first edge of the graph and the first entity moving more recently in the physical environment than the second entity.

2. A method as recited in claim 1, further comprising:

analyzing the first data from the first sensor and the third data from the third sensor using behaviometrics; and

identifying, based at least in part on the analyzing, the first entity as a specific entity.

3. A method as recited in claim 1, further comprising:

analyzing the first data from the first sensor and the third data from the third sensor using a Viterbi algorithm; and

identifying, based at least in part on the analyzing, the first entity as a specific entity.

4. A method as recited in claim 1, wherein a supervised learning technique is used to learn a mapping from sensor event contexts to the first entity based on training data.

5. A method as recited in claim 4, wherein the supervised learning technique comprises a naï

ve Bayes classifier.

6. A method as recited in claim 4, wherein the supervised learning technique comprises a hidden Markov model.

7. A method as recited in claim 1, wherein the rules further include a rule for identifying a new entity, wherein the rule for identifying the new entity is based at least in part on receiving data from a sensor at a location with no known adjacent entities.

8. A method as recited in claim 1, wherein the rules further include a rule for determining that an entity has exited the physical environment, wherein the rule for determining that the entity has exited the physical environment is based, at least in part, on the entity:

moving to a location adjacent to a doorway, or

failing to generate a sensor event for a predetermined period of time.

9. A method as recited in claim 1, wherein the first type of sensor includes a motion sensor and the second type of sensor includes a light sensor.

10. A method as recited in claim 1, wherein the first type of sensor includes a light sensor and the second type of sensor includes a motion sensor.

11. A method comprising:

defining a transition matrix for a physical environment, wherein;

the physical environment includes a plurality of sensors; and

the transition matrix represents, for each particular sensor of the plurality of sensors, a plurality of probabilities, each probability being a probability of an entity moving from a location associated with another sensor of the plurality of sensors to a location associated with the particular sensor;

determining a location of a first entity;

determining a location of a second entity;

receiving a sensor event from a first sensor in the physical environment;

based at least in part on receiving the sensor event;

determining, based on the transition matrix, a first probability that the first entity moved from the determined location of the first entity to a location associated with the first sensor; and

determining, based at least in part on the transition matrix, a second probability that the second entity moved from the determined location of the second entity to the location associated with the first sensor; and

identifying the first entity within the physical environment as an entity that caused the sensor event based at least in part on the first probability being greater than the second probability.

12. A method as recited in claim 11, wherein the first probability that the first entity moved from the determined location of the first entity to the location associated with the first sensor is based at least in part on training data that indicates a relative frequency with which an entity moves between the determined location of the first entity and the location associated with the first sensor.

13. A method as recited in claim 11, further comprising updating the transition matrix based at least in part on data observed within the physical environment.