Systems and methods for recording custom gesture commands

• US 9,996,164 B2
• Filed: 09/22/2016
• Issued: 06/12/2018
• Est. Priority Date: 09/22/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gummadi, Bapineedu Chowdary, Babbadi, Venkata A Naidu, Agnihotri, Hem, Tiwari, Anurag
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US15/273,284
• Publication Number:
  US 20180081447A1
• Time in Patent Office: 628 Days
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264

Abstract

Methods and apparatuses are disclosed for recording a custom gesture command. In an aspect, at least one camera of a user device captures the custom gesture command, wherein the custom gesture command comprises a physical gesture performed by a user of the user device, a user interface of the user device receives user selection of one or more operations of the user device, and the user device maps the custom gesture command to the one or more operations of the user device.

Claims

1. A method for recording a custom gesture command, comprising:

capturing, by a front-facing camera and a rear-facing camera of a handheld user device, the custom gesture command, wherein the custom gesture command comprises a facial gesture and a hand gesture performed by a user of the handheld user device, and wherein the front-facing camera captures the facial gesture of the user and the rear-facing camera captures the hand gesture of the user;

displaying, by a user interface of the handheld user device, an instruction to select a sequence of operations of the handheld user device to be mapped to the custom gesture command;

in response to the instruction to select the sequence of operations, receiving, at the user interface of the handheld user device, user selection of the sequence of operations of the handheld user device to be mapped to the custom gesture command; and

mapping, by the handheld user device, the custom gesture command to the sequence of operations of the handheld user device.

2. The method of claim 1, further comprising:

entering, by the handheld user device, a custom gesture command learn mode before the custom gesture command is captured; and

exiting, by the handheld user device, the custom gesture command learn mode after the custom gesture command is mapped to the sequence of operations.

3. The method of claim 1, further comprising:

capturing, by the handheld user device, information identifying the user of the handheld user device.

4. The method of claim 3, wherein the information identifying the user of the handheld user device comprises voice recognition data representing a voice of the user, facial recognition data representing a face of the user, login information of the user, or any combination thereof.

5. The method of claim 3, further comprising:

mapping, by the handheld user device, the information identifying the user to the custom gesture command and the sequence of operations of the handheld user device.

6. The method of claim 1, further comprising:

capturing, by at least one camera of the handheld user device, a second custom gesture command, wherein the second custom gesture command comprises a physical gesture performed by a second user of the handheld user device;

receiving, at the user interface of the handheld user device, user selection of a one or more operations of the handheld user device; and

mapping, by the handheld user device, the second custom gesture command to the one or more operations of the handheld user device.

7. The method of claim 6, wherein the second custom gesture command is different from the custom gesture command and the one or more operations of the handheld user device are the same as the sequence of operations of the handheld user device.

8. The method of claim 1, further comprising:

detecting, by the handheld user device, the custom gesture command;

identifying, by the handheld user device, the sequence of operations of the handheld user device mapped to the custom gesture command; and

performing, by the handheld user device, the sequence of operations of the handheld user device mapped to the custom gesture command.

9. The method of claim 8, further comprising:

identifying, by the handheld user device, the user based on information identifying the user of the handheld user device stored in a user profile on the handheld user device,

wherein the sequence of operations of the handheld user device are identified based on matching the custom gesture command to the information identifying the user.

10. The method of claim 8, wherein the custom gesture command is detected by the front-facing camera and the rear-facing camera of the handheld user device.

11. The method of claim 8, further comprising:

entering a custom gesture command input mode before detecting the custom gesture command; and

exiting the custom gesture command input mode after performing the sequence of operations of the handheld user device mapped to the custom gesture command.

12. The method of claim 11, wherein the custom gesture command input mode is entered in response to detection of a voice command from the user, and wherein the custom gesture command input mode is exited in response to detection of a voice command from the user.

13. The method of claim 11, wherein the custom gesture command input mode is entered in response to detection of a first gesture command performed by the user, and wherein the custom gesture command input mode is exited in response to detection of a second gesture command performed by the user.

14. The method of claim 13, wherein the first gesture command and the second gesture command are custom gesture commands.

15. The method of claim 8, further comprising:

aborting the performance of the sequence of operations of the handheld user device in response to receiving a voice command from the user.

16. An apparatus for recording a custom gesture command, comprising:

a front-facing camera and a rear-facing camera of a handheld user device;

a user interface of the handheld user device; and

at least one processor of the handheld user device configured to;

cause the front-facing camera and the rear-facing camera to capture the custom gesture command, wherein the custom gesture command comprises a facial gesture and a hand gesture performed by a user of the handheld user device, and wherein the front-facing camera captures the facial gesture of the user and the rear-facing camera captures the hand gesture of the user;

cause the user interface to display an instruction to select a sequence of operations of the handheld user device to be mapped to the custom gesture command;

cause the user interface to receive, in response to the instruction to select the sequence of operations, user selection of the sequence of operations of the handheld user device to be mapped to the custom gesture command; and

map the custom gesture command to the sequence of operations of the handheld user device.

17. The apparatus of claim 16, wherein the at least one processor is further configured to:

capture information identifying the user of the handheld user device.

18. The apparatus of claim 17, wherein the information identifying the user of the handheld user device comprises voice recognition data representing a voice of the user, facial recognition data representing a face of the user, login information of the user, or any combination thereof.

19. The apparatus of claim 17, wherein the at least one processor is further configured to:

map the information identifying the user to the custom gesture command and the sequence of operations of the handheld user device.

20. The apparatus of claim 16, wherein the at least one processor is further configured to:

cause at least one camera to capture a second custom gesture command, wherein the second custom gesture command comprises a physical gesture performed by a second user of the handheld user device;

cause the user interface to receive user selection of one or more operations of the handheld user device; and

map the second custom gesture command to the one or more operations of the handheld user device.

21. The apparatus of claim 16, wherein the at least one processor is further configured to:

detect the custom gesture command;

identify the sequence of operations of the handheld user device mapped to the custom gesture command; and

perform the sequence of operations of the handheld user device mapped to the custom gesture command.

22. The apparatus of claim 21, wherein the at least one processor is further configured to:

identify the user based on information identifying the user of the handheld user device stored in a user profile on the handheld user device,

wherein the sequence of operations of the handheld user device are identified based on a match between the custom gesture command and the information identifying the user.

23. The apparatus of claim 21, wherein the at least one processor is further configured to:

enter a custom gesture command input mode before detection of the custom gesture command; and

exit the custom gesture command input mode after performance of the sequence of operations of the handheld user device mapped to the custom gesture command.

24. The apparatus of claim 23, wherein the custom gesture command input mode is entered in response to detection of a voice command from the user, and wherein the custom gesture command input mode is exited in response to detection of a voice command from the user.

25. The apparatus of claim 23, wherein the custom gesture command input mode is entered in response to detection of a first gesture command performed by the user, and wherein the custom gesture command input mode is exited in response to detection of a second gesture command performed by the user.

26. An apparatus for recording a custom gesture command, comprising:

at least one a front-facing image capturing means and a rear-facing image capturing means of a handheld user device;

a user interface means of the handheld user device; and

a processing means of the handheld user device configured to;

cause the front-facing image capturing means and the rear-facing image capturing means to capture the custom gesture command, wherein the custom gesture command comprises a gesture and a hand gesture performed by a user of the handheld user device, and wherein the front-facing image-capturing means captures the facial gesture of the user and the rear-facing image-capturing means captures the hand gesture of the user;

cause the user interface means to display an instruction to select a sequence of operations of the user device to be mapped to the custom gesture command;

cause the user interface means to receive, in response to the instruction to select the sequence of operations, user selection of the sequence of operations of the handheld user device to be mapped to the custom gesture command; and

map the custom gesture command to the sequence of operations of the handheld user device.

27. A non-transitory computer-readable medium storing computer-executable instructions for recording a custom gesture command, the computer-executable instructions comprising:

at least one instruction instructing a front-facing camera and a rear-facing camera of a handheld user device to capture the custom gesture command, wherein the custom gesture command comprises a facial gesture and a hand gesture performed by a user of the handheld user device, and wherein the front-facing camera captures the facial gesture of the user and the rear-facing camera captures the hand gesture of the user;

at least one instruction instructing a user interface of the handheld user device to display an instruction to select a sequence of operations of the handheld user device to be mapped to the custom gesture command;

at least one instruction instructing the user interface of the handheld user device to receive, in response to the instruction to select the sequence of operations, user selection of the sequence of operations of the handheld user device to be mapped to the custom gesture command; and

at least one instruction instructing the handheld user device to map the custom gesture command to the sequence of operations of the handheld user device.