Automatically learning and controlling connected devices

• US 10,270,609 B2
• Filed: 05/29/2015
• Issued: 04/23/2019
• Est. Priority Date: 02/24/2015
• Status: Active Grant

• Current Assignee: BrainofT, Inc.
• Original Assignee: BrainofT, Inc.
• Inventors: Saxena, Ashutosh, Koppula, Hema Swetha, Wu, Chenxia, Sener, Ozan
• Primary Examiner(s): Hartman, Jr., Ronald D
• Application Number:
  US14/725,989
• Publication Number:
  US 20160248847A1
• Time in Patent Office: 1,425 Days
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A first input is received from a plurality of sensors. A first state including a first location based on the first input is determined. The first state is associated with a first probability. A second input is received from the plurality of sensors. A second state including a second location is determined based on the second input associated with a second probability. It is determined that the second state corresponds to an actual state based on a transition model and the second probability. The transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state. A rule to change a state of at least one network connected device is triggered based on the second state.

Claims

1. A system for controlling network connected devices, including:

a processor configured to;

receive a first input from a plurality of sensors;

determine a first state including a first location based on the first input, wherein the first state is associated with a first probability;

receive a second input from the plurality of sensors;

determine a second state including a second location based on the second input associated with a second probability;

determine that the second state corresponds to an actual state based on a transition model and the second probability, wherein the transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state; and

trigger a rule to change a state of at least one network connected device based on the second state, wherein changing the state of the at least one network connected device includes controlling the at least one network connected device based on the second state; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions.

2. The system of claim 1, wherein the plurality of sensors includes a camera.

3. The system of claim 1, wherein the first state is a first state of a human subject and the second state is a second state of the human subject.

4. The system of claim 1, wherein the first state includes a first event and the second state includes a second event.

5. The system of claim 4, wherein the second activity is one of a plurality of human activities predefined to be recognizable.

6. The system of claim 4, wherein the second activity has been detected by computer vision analysis of a video image.

7. The system of claim 1, wherein the second probability identifies a likelihood the second state is the actual state.

8. The system of claim 1, wherein the first state has been identified as a previous valid state.

9. The system of claim 1, wherein determining that the second state corresponds to the actual state includes determining that the second probability meets a threshold value.

10. The system of claim 1, wherein the rule is based on a previous observation associated with the second state.

11. The system of claim 1, wherein the rule is based on a feedback from a previous trigger of the rule.

12. The system of claim 1, wherein the second state triggers a plurality of rules that conflict and the triggered rule has been selected for triggering based on a comparison of priority values of the plurality of rules.

13. The system of claim 1, wherein the transition model was generated using deep learning to analyze a history of state transitions.

14. The system of claim 1, wherein the processor is further configured to determine that a first sensor data and a second sensor data are associated with a same detected subject.

15. The system of claim 1, wherein determining that the second state corresponds to the actual state includes determining a correlation between components of the second state.

16. The system of claim 1, wherein the first state and the second state have been determined using machine learning.

17. The system of claim 1, wherein the rule was automatically generated based on observations detected using the plurality of the sensors.

18. The system of claim 1, wherein the first location identifies a physical location within a building.

19. The system of claim 1, wherein the first input and the second input are included in streaming inputs received from the plurality of sensors.

20. The system of claim 1, wherein changing the state of the at least one network connected device based on the second state includes sending a command to the at least one network connected device.

21. A method for controlling network connected devices, including:

receiving a first input from a plurality of sensors;

determining a first state including a first location based on the first input, wherein the first state is associated with a first probability;

receiving a second input from the plurality of sensors;

determining a second state including a second location based on the second input associated with a second probability;

using a processor to determine that the second state corresponds to an actual state based on a transition model and the second probability, wherein the transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state; and

triggering a rule to change a state of at least one network connected device based on the second state, wherein changing the state of the at least one network connected device includes controlling the at least one network connected device based on the second state.

22. A computer program product for controlling network connected devices, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a first input from a plurality of sensors;

determining a first state including a first location based on the first input, wherein the first state is associated with a first probability;

receiving a second input from the plurality of sensors;

determining a second state including a second location based on the second input associated with a second probability;

determining that the second state corresponds to an actual state based on a transition model and the second probability, wherein the transition model associates the first state with the second state and indicates a likelihood of a transition from the first state to the second state; and

triggering a rule to change a state of at least one network connected device based on the second state, wherein changing the state of the at least one network connected device includes controlling the at least one network connected device based on the second state.