Systems and methods to operate controllable devices with gestures and/or noises

• US 10,295,972 B2
• Filed: 04/29/2016
• Issued: 05/21/2019
• Est. Priority Date: 04/29/2016
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Laurent, Patryk, Izhikevich, Eugene
• Primary Examiner(s): Karim, Ziaul
• Application Number:
  US15/143,397
• Publication Number:
  US 20170315519A1
• Time in Patent Office: 1,117 Days
• Field of Search: 700275
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: H04L 12/2809 indicating that an appliance service is present in a home automation network monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72415 for remote control of appliances
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 8/005 Discovery of network devices, e.g. terminals

Abstract

Gesture recognition systems that are configured to provide users with simplified operation of various controllable devices such as, for example, in-home controllable devices. In one implementation, the gesture recognition system automatically configures itself in order to determine the respective physical locations and/or identities of controllable devices as well as an operating mode for controlling the controllable devices through predetermined gesturing. In some implementations, the gesture recognition systems are also configured to assign boundary areas associated with the controllable devices. Apparatus and methods associated with the gesture recognition systems are also disclosed.

Claims

1. A non-transitory computer-readable storage medium having computer-readable instructions stored thereon, that when executed by a processor configure the processor to:

operate in an auto-configuration mode, in the auto-configuration mode the processor being configured to execute the computer-readable instructions to,

discover a location of at least one of a plurality of controllable devices within a detecting area based on a signal transmitted by a respective one of the plurality of controllable devices, the signal comprising audio and visual changes pertaining to the respective one of the plurality of controllable devices within the detecting area to avoid false positives; and

operate in an operation mode, the operation mode being different from the auto-configuration mode, and in the operation mode the processor being configured to execute the computer-readable instructions to,

detect an audio noise from a user and a body gesture from the user in order to command an operation of the at least one of the plurality of controllable devices in the detecting area,

assign a boundary area to each of the plurality of the controllable devices by associating a first boundary area with a first unique identifier and a different second boundary area with a different second unique identifier, the first and second unique identifiers being associated to a first and second controllable devices, respectively, and

operate at least one of the plurality of controllable devices in response to the detection of the at least one of the audio noise and the body gesture from the user.

2. The non-transitory computer-readable storage medium of claim 1, wherein

the detection of the at least one of the audio noise or the body gesture further comprises detection of the at least one of the audio noise or the body gesture within the assigned boundary area.

3. The non-transitory computer-readable storage medium of claim 1, further comprising:

determining the unique identifiers of each of the plurality of the controllable devices further comprises wireless detection within a wireless network of the first unique identifier and the second unique identifier.

4. The non-transitory computer-readable storage medium of Claim 1, wherein

the at least one of the audio noise or the body gesture is directed towards a respective one of the plurality of the controllable devices.

5. The non-transitory computer-readable storage medium of claim 1, wherein,

the user includes a motion performed by at least one of a person, animal, or object.

6. The non-transitory computer-readable storage medium of claim 1, wherein,

the signal is detected within the detecting area by at least one of a camera sensor or a microphone.

7. The non-transitory computer-readable storage medium of claim 1, wherein the processor is further configured to execute the computer-readable instructions to,

learn an association between the at least one of the noise and the body gesture from the user and the operation of the one or more of the plurality of the controllable devices while in the operational mode such that, upon detecting the at least one of the audio noise and the body gesture from the user within the detecting area the at least one of the plurality of controllable devices performs the operation.

8. A method for operating a gesture detection system, the method comprising:

operating in an auto-configurable mode, in the auto-configuration mode a processor being configured to execute computer-readable instructions to;

discover a location of at least one of a plurality of controllable devices within a detecting area based on a signal transmitted by a respective one of the plurality of controllable devices, the signal comprising audio and visual changes pertaining to the respective one of the plurality of controllable devices within the detecting area to avoid false positives; and

operating in an operation mode, the operation mode being different from the auto-configuration mode, and in the operation mode the processor being configured to execute the computer-readable instructions to,

detect an audio noise from a user and a body gesture from the user in order to command an operation of the at least one of the plurality of controllable devices within the detecting area,

assign a boundary area to each of the plurality of the controllable devices by associating a first boundary area with a first unique identifier and a different second boundary area with a different second unique identifier, the first and second unique identifiers being associated to a first and second controllable devices, respectively, and

operate at least one of the plurality of controllable devices in response to the detection of the at least one of the audio noise and the body gesture from the user.

9. The method of claim 8, further comprising:

learning an association between the at least one of the noise and the body gesture from the user and the operation of the one or more of the plurality of the controllable devices while in the operational mode such that, upon detecting the at least one of the audio noise and the body gesture from the user within the detecting area, the at least one of the plurality of the controllable devices performs the operation.

10. The method of claim 8, further comprising:

assigning a boundary area to at least one of the plurality of the controllable device, wherein the detecting of the at least one of the audio noise or the body gesture comprises detecting the at least one of the audio noise or the body gesture being performed within the boundary area.

11. A system for detecting gestures for operation of a plurality of controllable devices within a detecting area, the system comprising:

at least one detection device;

a processing apparatus in communication with the at least one detection device; and

a non-transitory computer readable storage medium having a plurality of instructions stored thereon, that when executed by the processing apparatus, cause the processing apparatus to,

operate in an auto-configuration mode, in the auto-configuration mode the processing apparatus being configured to execute the instructions to,

discover a location of the plurality of controllable devices within the detecting area based on a signal transmitted by a respective one of the plurality of controllable devices, the signal comprising audio and visual changes pertaining to the respective one of the plurality of controllable devices within the detecting area to avoid false positives; and

operate in an operation mode, the operation mode being different from the auto-configuration mode, and in the operation mode the processing apparatus being configured to execute the instructions to,

detect an audio noise from a user and a body gesture from the user in order to command operation of the at least one of the plurality of controllable devices in the detecting area,

assign a boundary area to each of the plurality of the controllable devices by associating a first boundary area with a first unique identifier and a different second boundary area with a different second unique identifier, the first and second unique identifiers being associated to a first and second controllable devices, respectively, and

operate at least one of the plurality of controllable devices in response to the detection of the at least one of the audio noise and the body gesture from the user.

12. The system of claim 11, wherein the processing apparatus is further configured to execute the computer-readable instructions to

the act of detecting the at least one of the audio noise or the body gesture form the user comprises detecting the at least one of the audio noise and the body gesture within the boundary area.

13. The system of claim 12, wherein the processing apparatus is further configured to execute the computer-readable instructions to

learn an association between the at least one of the audio noise from the user or the body gesture from the user and the operation of the at least one of the plurality of controllable devices while in the operational mode such that, upon detecting the at least one of the audio noise and the body gesture from the user within the detecting area, the at least one of the plurality of controllable devices performs the operation.