Remote state following device

• US 10,282,621 B2
• Filed: 07/09/2017
• Issued: 05/07/2019
• Est. Priority Date: 07/09/2016
• Status: Active Grant

• Current Assignee: Grabango Co.
• Original Assignee: Grabango Co.
• Inventors: Glaser, William, Van Osdol, Brian
• Primary Examiner(s): Crawford, Jason
• Application Number:
  US15/644,803
• Publication Number:
  US 20180014382A1
• Time in Patent Office: 667 Days
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: H04L 12/2809 indicating that an appliance service is present in a home automation network monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51
• CPC Class Codes: H04L 12/2814 Exchanging control software or macros for controlling appliance services in a home automation network arrangements for maintenance or administration involving configuration of the network and network elements H04L41/08
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H05B 47/105 in response to determined parameters
• CPC Class Codes: H05B 47/125 by using cameras
• CPC Class Codes: H05B 47/19 via wireless transmission
• CPC Class Codes: H05B 47/1965 using handheld communication devices
• CPC Class Codes: Y02B 20/40 Control techniques providing energy savings, e.g. smart controller or presence detection

Abstract

A system and method for a remote state following device that includes an electronic device with a controllable operating state; an imaging device; and control system that when targeted at a control interface interprets a visual state from the control interface, and modifies the operating state in coordination with the visual state.

Claims

1. A system comprising:

an electronic device with a controllable operating state;

an imaging device that captures image data of an environment and that has an imaging resolution that isolates visual state of a targeted control interface in a field of view of the imaging device;

a control interface that comprises a pairing identifier;

a control system communicatively coupled to the imaging device and the electronic device, the control system comprising a visually monitored interface control mode with configuration to;

identify the pairing identifier in the field of view of the image data and thereby detect the control interface,

interpret a visual state of the control interface, and

modulate the operating state in coordination with the visual state.

2. The system of claim 1, wherein there are at least two types of detectable control interfaces, and wherein a first type of control interface has a first set of visual states that are mapped to two distinct operating states and a second type of control interface that has a second set of visual states that are mapped to a range of operating states.

3. The system of claim 1, wherein the imaging device captures image data of an environment, wherein the control system comprises configuration to configure a sub-region of image data as a location of the control interface.

4. The system of claim 3, wherein the sub-region of the image data is automatically configured as the location of the control interface.

5. The system of claim 3, wherein the control system is further configured to detect a second control interface in a second sub-region of image data, interpret the visual state of the second control interface, and wherein modulation of the operating state is further modulated in coordination with the visual state of the second control interface.

6. The system of claim 1, wherein the imaging device comprises an optical system with a narrow field of view.

7. The system of claim 1, further comprising a pairing indicator that is configured to signal pairing state of the system.

8. The system of claim 1, wherein the control interface is a non-electric element.

9. The system of claim 1, wherein the control interface comprises at least one interaction region, wherein the interpretation of visual state comprises configuration to detect a user gesture within the interaction region, and wherein the a detected gesture and the interaction region map to a targeted operating state.

10. The system of claim 1, wherein the control system further comprises a pairing mode, wherein during a pairing mode, the control system is configured to detect the pairing identifier in a sub-region of the image data, and upon detection of the pairing identifier, register that sub-region for detection of a control interface.

11. The system of claim 1, wherein configuration to interpret a visual state of the control interface comprises configuration to detect a set of visual physical states of a control interface.

12. The system of claim 1, wherein configuration to interpret a visual state of the control interface comprises configuration to detect a visual gesture with an interaction region of the control interface.

13. The system of claim 1, further comprising an ambient activity sensor, and wherein the control system comprises activating the visually monitored interface control mode upon activation of the ambient activity sensor.

15. The system of claim 1, wherein the electronic device is a lighting device, and wherein the operating state is an illumination state of the lighting device.

16. The system of claim 15, wherein the control interface is a bi-stable light switch.

17. The system of claim 1, wherein the electronic device is a home automation hub, and wherein the operating state specifies the operating state of a set of connected devices.

18. The system of claim 1, wherein the electronic device comprises operating states comprising an on-state and an off-state.

14. The system of 1, further comprising a directable housing that at least partially encases the imaging device and couples to the electronic device.

19. A system comprising:

an electronic device with a controllable operating state;

an imaging device with an imaging resolution that isolates visual state of a targeted control interface in a field of view of the imaging device;

a control system communicatively coupled to the imaging device and the electronic device, the control system comprising a visually monitored interface control mode with configuration to;

interpret a visual state of the control interface, and

modulate the operating state in coordination with the visual state; and

the control system further comprising a pairing mode, wherein during a pairing mode, the control system is configured to detect a pairing identifier in a sub-region of the image data, and upon detection of the pairing identifier, register that sub-region for detection of the control interface.

20. The system of claim 19, wherein there are at least two types of detectable control interfaces, and wherein a first type of control interface has a first set of visual states that are mapped to two distinct operating states and a second type of control interface that has a second set of visual states that are mapped to a range of operating states.

21. The system of claim 19, wherein the sub-region of the image data is automatically configured as the location of the control interface.

22. The system of claim 19, wherein the control system is further configured to detect a second control interface in a second sub-region of image data, interpret the visual state of the second control interface, and wherein modulation of the operating state is further modulated in coordination with the visual state of the second control interface.

23. The system of claim 19, wherein the imaging device comprises an optical system with a narrow field of view.

24. The system of claim 19, further comprising a pairing indicator that is configured to signal pairing state of the system.

25. The system of claim 19, further comprising a control interface.

26. The system of claim 25, wherein the control interface is a non-electric element.

27. The system of claim 25, wherein the control interface comprises at least one interaction region, wherein the interpretation of visual state comprises configuration to detect a user gesture within the interaction region, and wherein a detected gesture and the interaction region map to a targeted operating state.

28. The system of claim 19, wherein configuration to interpret a visual state of the control interface comprises configuration to detect a set of visual physical states of a control interface.

29. The system of claim 19, wherein configuration to interpret a visual state of the control interface comprises configuration to detect a visual gesture with an interaction region of the control interface.

30. The system of claim 19, further comprising an ambient activity sensor, and wherein the control system comprises activating the visually monitored interface control mode upon activation of the ambient activity sensor.

32. The system of claim 19, wherein the electronic device is a lighting device, and wherein the operating state is an illumination state of the lighting device.

33. The system of claim 32, wherein the control interface is a bi-stable light switch.

34. The system of claim 19, wherein the electronic device is a home automation hub, and wherein the operating state specifies the operating state of a set of connected devices.

35. The system of claim 19, wherein the electronic device comprises operating states comprising an on-state and an off-state.

31. The system of 26, further comprising a directable housing that at least partially encases the imaging device and couples to the electronic device.