Human computer interaction using wearable device

• US 9,864,428 B2
• Filed: 06/28/2013
• Issued: 01/09/2018
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Zhang, Guangli
• Primary Examiner(s): DINH, DUC Q
• Application Number:
  US14/785,425
• Publication Number:
  US 20160070341A1
• Time in Patent Office: 1,656 Days
• Field of Search: 345169, 345173, 345174, 345175, 345156
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Embodiments of apparatus and methods for human-computer interaction are described. An apparatus for human-computer interaction may have one or more processors, multiple sensors to measure motion of a body part of a user, a communication module to communicate with a remote computing device, and an interpretation module to interpret the motion of the body part of the user to be associated with an indication of a user input to the remote computing device. The components may be encased in a body configured to be worn by the user. Other embodiments may be described and/or claimed.

Claims

1. A wearable apparatus for human-computer interaction, comprising:

one or more processors;

a sensor module coupled to the one or more processors to measure a motion of a body part of a user, wherein the user is interacting with an application of a computing device external to the apparatus;

an interpretation module, coupled with the one or more processors and the sensor module, and configured to;

interpret the motion of the body part of the user;

receive an indication of available actions based on a current state of the computing device or the application, wherein the indication of the available actions is received when the wearable apparatus enters a new state based on the current state of the computing device or the application prompting a new user interaction screen;

output the available actions for display on a display of the wearable apparatus;

associate the interpreted motion with an available action of the available actions; and

translate, using a trained gesture estimation model, the motion of the body part into an indication of a user input for the application based on the associated available action;

a communication module, coupled to the interpretation module, configured to;

send the indication of the user input to the computing device for the application, and

receive a feedback signal from the computing device notifying the user that the user input has been successfully accepted by the computing device; and

a body encasing the one or more processors, the sensor module, the communication module, and the interpretation module, wherein the body is configured for the apparatus to be worn by the user.

2. The wearable apparatus according to claim 1, wherein the body part of the user comprises at least one of an arm, a hand, or a finger.

3. The wearable apparatus according to claim 1, wherein the communication module comprises at least one of a line-of-sight wireless transmitter, an infrared transmitter, or a radio frequency transmitter.

4. The wearable apparatus according to claim 1, wherein the motion comprises at least one of acceleration, orientation, velocity, or positional change.

5. The wearable apparatus according to claim 1, wherein the sensor module is configured to measure a moving force applied to the body part of the user.

6. The wearable apparatus according to claim 1, wherein the sensor module is configured to measure a force generated by a finger tendon of the user.

7. The wearable apparatus according to claim 1, wherein the interpretation module is configured to interpret and translate a pattern of a plurality of moving forces applied to the body part of the user into the indication of the user input based at least in part on the trained gesture estimation model.

8. The wearable apparatus according to claim 7, wherein the plurality of moving forces are generated by a plurality of finger tendons of the user.

9. The wearable apparatus according to claim 7, wherein the trained gesture estimation model is customizable by the user.

10. The wearable apparatus according to claim 1, wherein the body is configured for the apparatus to be worn on a wrist, a neck, an arm, a leg, or a head of the user.

11. A method for human-computer interaction, comprising:

measuring, by a wearable computing device, motion of a body part of a user, wherein the user is interacting with an application of a computing device external to the wearable computing device;

receiving, at the wearable computing device, an indication of available actions based on a current state of the computing device or the application, wherein the indication of the available actions is received when the wearable computing device enters a new state based on the current state of the computing device or the application prompting a new user interaction screen;

displaying, by the wearable computing device, the available actions;

associating, by the wearable computing device, the measured motion with an available action of the available actions;

interpreting, by the wearable computing device, using a trained gesture estimation model, the motion of the body part of the user into an indication of a user input for the application based on the associated available action;

sending, by the wearable computing device, the indication of the user input to the computing device; and

receiving, by the wearable computing device, a feedback signal from the computing device notifying the user that the user input has been successfully accepted by the computing device.

12. The method according to claim 11, further comprising:

retrieving, by the wearable computing device, indications of the available actions from the computing device.

13. The method according to claim 11, wherein the measuring comprises at least one of measuring a moving force applied to the body part of the user, or measuring a force applied by a finger tendon of the user.

14. The method according to claim 11, wherein the interpreting comprises translating a pattern of a plurality of moving forces applied to the body part of the user into the indication of the user input based at least in part on a gesture estimation model.

15. The method according to claim 14, wherein the plurality of moving forces are generated by a plurality of finger tendons.

16. At least one non-transitory computer-readable storage medium, comprising a plurality of instructions, which when executed by at least one processor of a wearable computing device, cause the at least one processor to:

measure motion of a body part of a user, wherein the user is interacting with an application of a computing device external to the wearable computing device;

receive an indication of available actions based on a current state of the computing device or the application, wherein the indication of the available actions is received when the wearable computing device enters a new state based on the current state of the computing device or the application prompting a new user interaction screen;

output the available actions for display on a display of the wearable computing device;

associate the measured motion with an available action of the available actions;

interpret, using a trained gesture estimation model, the motion of the body part of the user into an indication of a user input for the application based on the associated available action;

send the indication of the user input to the computing device; and

receive, from the computing device, a feedback signal notifying the user that the user input has been successfully accepted by the computing device.

17. The at least one non-transitory computer-readable storage medium of claim 16, wherein the plurality of instructions further cause the at least one processor to:

retrieve indications of the available actions from the computing device.

18. The at least one non-transitory computer-readable storage medium of claim 16, wherein the plurality of instructions further cause the at least one processor to:

receive a feedback signal from the computing device in response to receiving the indication of the user input.

19. The at least one non-transitory computer-readable storage medium of claim 16, wherein the plurality of instructions that cause the at least one processor to measure the motion of the body part further comprise a plurality of instructions that cause the at least one processor to measure a moving force applied to the body part of the user or measure a force applied by a finger tendon of the user.

20. The at least one non-transitory computer-readable storage medium of claim 16, wherein the plurality of instructions that cause the at least one processor to interpret the motion comprise a plurality of instructions that cause the at least one processor to translate a pattern of a plurality of moving forces applied to the body part of the user into the indication of the user input based at least in part on the trained gesture estimation model.