Method and apparatus for controlling smart device

• US 10,007,354 B2
• Filed: 02/19/2016
• Issued: 06/26/2018
• Est. Priority Date: 02/26/2015
• Status: Active Grant

• Current Assignee: Xiaomi Inc. (Xiaomi Corp.)
• Original Assignee: Xiaomi Inc. (Xiaomi Corp.)
• Inventors: Liang, Xin, Fan, Li
• Primary Examiner(s): Watko, Julie Anne
• Application Number:
  US15/048,242
• Publication Number:
  US 20160252967A1
• Time in Patent Office: 858 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/70 Device selection
• CPC Class Codes: G08C 2201/92 Universal remote control
• CPC Class Codes: G08C 2201/93 Remote control using other portable devices, e.g. mobile phone, PDA, laptop
• CPC Class Codes: H04L 12/2809 indicating that an appliance service is present in a home automation network monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

The present disclosure relates to a method and an apparatus for controlling a smart device. The method includes: when a first hand gesture is detected, preparing to switch a control relationship according to the first hand gesture and a first mapping relationship, the first mapping relationship including a corresponding relationship between hand gestures and respective control functions; when a second hand gesture is detected, determining a target controlled device according to the second hand gesture and a second mapping relationship, the second mapping relationship including a corresponding relationship between hand gestures and locations of respective controlled devices; establishing a first control relationship with the target controlled device; acquiring a third hand gesture; and controlling, according to the third hand gesture and the first mapping relationship, the target controlled device to perform an operation corresponding to the third hand gesture.

Claims

1. A method for use in a controlling device, comprising:

when a first hand gesture is detected, releasing a control relationship with a first controlled device according to the first hand gesture and a first mapping relationship, the first mapping relationship including a corresponding relationship between hand gestures and respective control functions, the first controlled device being controlled by the controlling device before the first hand gesture is detected;

when a second hand gesture is detected, determining a second controlled device according to the second hand gesture and a second mapping relationship, the second mapping relationship including a corresponding relationship between hand gestures and locations of respective controlled devices, wherein when the first hand gesture and the second hand gesture are performed by two hands, respectively, the determining of the second controlled device according to the second hand gesture and the second mapping relationship further comprises;

when the first hand gesture and the second hand gesture are detected approximately at the same time, determining the second controlled device according to the second hand gesture and the second mapping relationship;

after the control relationship with the first controlled device is released, establishing a control relationship with the second controlled device;

acquiring a third hand gesture; and

controlling, according to the third hand gesture and the first mapping relationship, the second controlled device to perform an operation corresponding to the third hand gesture.

2. The method according to claim 1, further comprising:

acquiring a first positioning straight line and a second positioning straight line, each of the first positioning straight line and the second positioning straight line being an extension line of a direction pointed by a finger of a user when the user respectively points to a controlled device at a first location and a second location;

determining a location of the controlled device according to an intersection point of the first positioning straight line and the second positioning straight line; and

establishing the second mapping relationship between a hand gesture and the location of the controlled device.

3. The method according to claim 1, further comprising:

establishing the first mapping relationship between a hand gesture and a control function, wherein the hand gesture is used to control the control function in different controlled devices.

4. The method according to claim 1, wherein when the first hand gesture and the second hand gesture are performed by the same hand, the determining of the second controlled device according to the second hand gesture and the second mapping relationship further comprises:

when the detection of the second hand gesture and the detection of the first hand gesture are separated by a predetermined amount of time, determining the second controlled device according to the second hand gesture and the second mapping relationship.

5. The method according to claim 1, further comprising:

releasing the control relationship with the second controlled device; and

establishing a control relationship with the first controlled device.

6. A controlling device, comprising:

one or more processors; and

a memory for storing instructions executable by the one or more processors;

wherein the one or more processors are configured to;

when a first hand gesture is detected, release a control relationship with a first controller device according to the first hand gesture and a first mapping relationship, the first mapping relationship including a corresponding relationship between hand gestures and respective control functions, the first controlled device being controlled by the controlling device before the first hand gesture is detected;

when a second hand gesture is detected, determine a second controlled device according to the second hand gesture and a second mapping relationship, the second mapping relationship including a corresponding relationship between hand gestures and locations of respective controlled devices;

after the control relationship with the first controlled device is released, establish a control relationship with the second controlled device;

acquire a third hand gesture; and

control, according to the third hand gesture and the first mapping relationship, the second controlled device to perform an operation corresponding to the third hand gesture;

wherein when the first hand gesture and the second hand gesture are performed by two hands respectively, the one or more processors are further configured to;

when the first hand gesture and the second hand gesture are detected approximately at the same time, determine the second controlled device according to the second hand gesture and the second mapping relationship.

7. The controlling device according to claim 6, wherein the one or more processors are further configured to:

acquire a first positioning straight line and a second positioning straight line, each of the first positioning straight line and the second positioning straight line being an extension line of a direction pointed by a finger of a user when the user respectively points to a controlled device at a first location and a second location;

determine a location of the controlled device according to an intersection point of the first positioning straight line and the second positioning straight line; and

establish the second mapping relationship between a hand gesture and the location of the controlled device.

8. The controlling device according to claim 6, wherein the one or more processors are further configured to:

establish the first mapping relationship between a hand gesture and a control function, wherein the hand gesture is used to control the control function in different controlled devices.

9. The controlling device according to claim 6, wherein when the first hand gesture and the second hand gesture are performed by the same hand, the one or more processors are configured to:

when the detection of the second hand gesture and the detection of the first hand gesture are separated by a predetermined amount of time, determine the second controlled device according to the second hand gesture and the second mapping relationship.

10. The controlling device according to claim 6, wherein the one or more processors are further configured to:

release the first control relationship with the second controlled device; and

establish a control relationship with the first controlled device.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of a controlling device, cause the controlling device to perform:

when a first hand gesture is detected, releasing a control relationship with a first controlled device according to the first hand gesture and a first mapping relationship, the first mapping relationship including a corresponding relationship between hand gestures and respective control functions, the first controlled device being controlled by the controlling device before the first hand gesture is detected;

when a second hand gesture is detected, determining a second controlled device according to the second hand gesture and a second mapping relationship, the second mapping relationship including a corresponding relationship between hand gestures and locations of respective controlled devices, wherein when the first hand gesture and the second hand gesture are performed by two hands, respectively, the determining of the second controlled device according to the second hand gesture and the second mapping relationship further comprises;

when the first hand gesture and the second hand gesture are detected approximately at the same time, determining the second controlled device according to the second hand gesture and the second mapping relationship;

after the control relationship with the first controlled device is released, establishing a first control relationship with the second controlled device;

acquiring a third hand gesture; and

controlling, according to the third hand gesture and the first mapping relationship, the second controlled device to perform an operation corresponding to the third hand gesture.