Knowledge transfer in smart environments

• US 9,251,463 B2
• Filed: 06/29/2012
• Issued: 02/02/2016
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: WSU Research Foundation
• Original Assignee: WSU Research Foundation
• Inventors: Cook, Diane J.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Traktovenko, Ilya
• Application Number:
  US13/538,932
• Publication Number:
  US 20130006906A1
• Time in Patent Office: 1,313 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Activity templates are generated from one or more existing smart environments (e.g., source spaces) based on sensor data from the one or more existing smart environments that corresponds to known activities. A target activity template is then generated for a new smart environment, e.g., the target space. The source space activity templates are then mapped to the target activity templates to enable recognition of activities based on sensor data received from the target space.

Claims

1. A method comprising:

generating source activity templates from a plurality of source spaces, wherein generating source activity templates from a plurality of source spaces comprises;

receiving sensor event data from a plurality of source spaces;

recognizing activities based on contiguous sequences of sensor events from the plurality of source spaces; and

combining similar activities to form source activity templates;

generating target activity templates for a target space;

mapping the source activity templates to the target activity templates based in part on at least one of;

spatial similarities between the activities of the source activity templates and activities associated with the target activity templates, or

temporal similarities between the activities of the source activity templates and activities associated with the target activity templates; and

labeling the target activity templates to enable recognition of activities based on sensor data from the target space.

2. A method as recited in claim 1, wherein generating source activity templates from a plurality of source spaces further comprises:

performing sensor selection for each source activity template to identify particular sensors in the plurality of source spaces that are associated with each source activity template.

3. A method as recited in claim 1, wherein generating target activity templates from a target space comprises:

receiving sensor event data from a target space;

portioning the sensor event data into activities based on sensor location; and

combining similar activities to form target activity templates.

4. A method as recited in claim 1, wherein mapping the source activity templates to the target activity templates comprises:

initializing a sensor mapping matrix;

initializing an activity mapping matrix;

computing sensor mapping probabilities; and

maximizing activity mapping probabilities.

5. A method as recited in claim 4, further comprising iteratively repeating the computing and maximizing.

6. A method as recited in claim 4, wherein initializing the sensor mapping matrix comprises mapping a source sensor to a target sensor when the source sensor and the target sensor have a same location tag.

7. A method as recited in claim 4, wherein initializing the activity mapping matrix comprises mapping a source activity to a target activity when a similarity between the source activity and the target activity exceeds a threshold value.

8. A method as recited in claim 7, wherein the similarity between the source activity and the target activity is based on activity start time.

9. A method as recited in claim 7, wherein the similarity between the source activity and the target activity is based on activity duration.

10. A method as recited in claim 7, wherein the similarity between the source activity and the target activity is based on activity location.

11. A method as recited in claim 7, wherein the similarity between the source activity and the target activity is based on activity structure in terms of sensors.

12. One or more non-transitory computer readable media encoded with computer-executable instructions that, when executed, configure a computer system to perform a method as recited in claim 1.

13. A computing device comprising:

a processor;

a memory communicatively coupled to the processor;

an activity recognition module, at least partially stored in the memory and executable on the processor, to recognize activities based on received sensor data from a smart environment;

an activity model abstraction module to generate source activity templates based on activities recognized from sensor data received from a plurality of smart environments; and

a mapping module to map the source activity templates to target activity templates associated with a new smart environment based in part on at least one of;

spatial similarities between the source activity templates and the target activity templates, or

temporal similarities between the source activity templates and the target activity templates.

14. A system comprising:

one or more processors;

one or more computer-readable media;

a plurality of source spaces in a first smart environment;

a plurality of target spaces in a second smart environment;

processor-executable instructions maintained on the one or more computer-readable media which, when executed by the one or more processors, program the one or more processors to;

receive labeled sensor event data from the source spaces;

determine source activities from the labeled sensor event data based at least in part on grouping contiguous sequences of sensor events occurring in proximity to one another;

generate a source activity template for the first smart environment based at least in part on the source activities;

receive unlabeled sensor event data from the target spaces;

determine target activities from the unlabeled sensor event data based at least in part on grouping contiguous sequences of sensor events occurring in proximity to one another;

generate a target activity template for the second environment based at least in part on the source activities; and

map the source activity template to the target activity template based at least in part on at least one of;

spatial similarities between the source activity template and the target activity template, or

temporal similarities between the source activity template and the target activity template.

15. A system as recited in claim 14, wherein the one or more processors are further programmed to label the target activity template.

16. A system as recited in claim 14, wherein mapping the source activity template to the target activity template, further comprises:

initializing a sensor mapping matrix to represent a mapping between a first sensor in the plurality of source spaces and a second sensor in the plurality of target spaces, wherein the first sensor is associated with the source activity template and the second sensor is associated with the target activity template;

initializing an activity mapping matrix to represent a mapping between the source activity template and the target activity template;

determining sensor mapping probabilities based at least in part on a probability that the first sensor corresponds to the second sensor; and

maximizing activity mapping probabilities based at least in part on the sensor mapping probabilities.

17. A system as recited in claim 16, wherein initializing a sensor mapping matrix to represent the mapping between the first sensor in the plurality of source spaces and the second sensor in the plurality of target spaces, further comprises determining a mapping value based at least in part on a first location tag associated with the first sensor and a second location tag associated with the second sensor.

18. A system as recited in claim 16, wherein determining sensor mapping probabilities based at least in part on a probability that the first sensor corresponds to the second sensor, further comprises determining a probability value based at least in part on a relationship between a plurality of source activity templates associated with the first sensor and a plurality of target activity templates associated with the second sensor.

19. A system as recited in claim 14, wherein determining source activities from the labeled sensor event data is based in part on at least one of a naï

ve Bayes classifier, hidden Markov model, and conditional field model.

20. A system as recited in claim 14, wherein generating the target activity template for the second environment further comprises combining the target activities into the target activity template based at least in part on a similarity value associated with the target activities exceeding a predetermined threshold.