GESTURE-BASED GRAPHICAL KEYBOARD FOR COMPUTING DEVICES

• US 20180267615A1
• Filed: 03/20/2017
• Published: 09/20/2018
• Est. Priority Date: 03/20/2017
• Status: Abandoned Application

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: DAQRI, LLC
• Inventors: Freeman, Jonathan Trevor, Kozlowski, Michael, Gupta, Noopur, Reyes, Anthony L., Aalto, Neil
• Application Number:
  US15/464,194
• Publication Number:
  US 20180267615A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A computing device provides augmented reality images of an environment in which the computing device is worn. The computing device is further configured to display a graphical keyboard for interacting with the computing device. The graphical keyboard may be displayed according to one or more configured keyboard layouts. The computing device further includes an inertial measurement unit, which provides input for manipulating the graphical keyboard. As a user of the computing device moves his or her body, or a portion thereof, corresponding graphical changes are made to the displayed graphical keyboard. In this way, by moving his or her body (or a portion thereof), the user is able to interact with, and provide input to, the computing device.

Claims

1. A computing device for displaying a graphical keyboard, the computing device comprising:

a machine-readable memory storing computer-executable instructions; and

at least one hardware processor in communication with the machine-readable memory that, when the computer-executable instructions are executed, configures a computing device to perform a plurality of operations, the plurality of operations comprising;

displaying, on a transparent display physically coupled with the at least one hardware processor, a graphical keyboard, the graphical keyboard having at least one selectable alphanumeric character;

acquiring, by at least one inertial measurement unit, a plurality of measurements indicating user movement of the computing device;

converting, by the at least one hardware processor, the plurality of measurements to obtain a plurality of vectors, at least one vector of the plurality of vectors associated with at least three axes of movement;

determining, by the at least one hardware processor, and based on the plurality of vectors, a command to perform with the displayed graphical keyboard; and

performing, by the at least one hardware processor, the determined command with the displayed graphical keyboard.

2. The computing device of claim 1, wherein the plurality of operations further comprises:

comparing one or more values of the plurality of vectors with a plurality of corresponding thresholds, at least one of the thresholds selected from the plurality of thresholds associated with the command to perform; and

the determining of the command to perform is further based on the comparison.

3. The computing device of claim 1, wherein the command to perform comprises a command to select the at least one selectable alphanumeric character.

4. The computing device of claim 1, wherein:

the command to perform comprises a command to submit the at least one selectable alphanumeric character; and

the plurality of operations further comprises displaying the submitted at least one selectable alphanumeric character at a predetermined location of the graphical keyboard.

5. The computing device of claim 1, wherein:

the displayed graphical keyboard comprises;

a horizontal arrangement of alphanumeric characters, the horizontal arrangement of alphanumeric characters being substantially parallel to a ground plane; and

at least one control element that indicates whether the user intends to submit the at least one selectable alphanumeric character for display on the transparent display physically coupled with the at least one hardware processor.

6. The computing device of claim 1, wherein:

the displayed graphical keyboard comprises;

a first plurality of alphanumeric characters, the first plurality of alphanumeric characters being a subset selected from a second plurality of alphanumeric characters, wherein each alphanumeric character of the first plurality of alphanumeric characters is displayed in a font size based on a relative position of the alphanumeric character; and

an indicator element that indicates an alphanumeric character selected from the first plurality of alphanumeric characters to be a submitted alphanumeric character, wherein the indicator element is associated with a predetermined position where the first plurality of alphanumeric characters are displayed.

7. The computing device of claim 1, wherein:

the displayed graphical keyboard comprises;

an elliptical arrangement of a plurality of alphanumeric characters, wherein;

a first position of the elliptical arrangement is associated with a largest font size and a second position is associated with a smallest font size;

each alphanumeric character of the plurality of alphanumeric characters is displayed at a two-dimensional coordinate relative to the transparent display physically coupled with the at least one hardware processor;

a first alphanumeric character of the plurality of alphanumeric characters is displayed with the largest font size at the first position;

a second alphanumeric character of the plurality of alphanumeric characters is displayed with the smallest font size at the second position; and

a set of alphanumeric characters selected from the plurality of alphanumeric characters, where each alphanumeric characters of the set is displayed with a font size having a value between the smallest font size and the largest font size.

8. A method for displaying a graphical keyboard on a computing device, the method comprising:

displaying, on a transparent display physically coupled with at least one hardware processor, a graphical keyboard, the graphical keyboard having at least one selectable alphanumeric character;

acquiring, by at least one inertial measurement unit, a plurality of measurements indicating user movement of the computing device;

converting, by the at least one hardware processor, the plurality of measurements to obtain a plurality of vectors, at least one vector of the plurality of vectors associated with at least three axes of movement;

determining, by the at least one hardware processor, and based on the plurality of vectors, a command to perform with the displayed graphical keyboard; and

performing, by the at least one hardware processor, the determined command with the displayed graphical keyboard.

9. The method of claim 8, wherein the method further comprises:

comparing one or more values of the plurality of vectors with a plurality of corresponding thresholds, at least one of the thresholds selected from the plurality of thresholds being associated with the command to perform; and

the determining of the command to perform is further based on the comparison.

10. The method of claim 8, wherein the command to perform comprises a command to select the at least one selectable alphanumeric character.

11. The method of claim 8, wherein:

the command to perform comprises a command to submit the at least one selectable alphanumeric character; and

the method further comprises displaying the submitted at least one selectable alphanumeric character at a predetermined location of the graphical keyboard.

12. The method of claim 8, wherein:

the displayed graphical keyboard comprises;

a horizontal arrangement of alphanumeric characters, the horizontal arrangement of alphanumeric characters being substantially parallel to a ground plane; and

at least one control element that indicates whether the user intends to submit the at least one selectable alphanumeric character for display on the transparent display physically coupled with the at least one hardware processor.

13. The method of claim 8, wherein:

the displayed graphical keyboard comprises;

a first plurality of alphanumeric characters, the first plurality of alphanumeric characters being a subset selected from a second plurality of alphanumeric characters, wherein each alphanumeric character of the first plurality of alphanumeric character is displayed in a font size based on a relative position of the alphanumeric character; and

an indicator element that indicates an alphanumeric character selected from the first plurality of alphanumeric characters to be a submitted alphanumeric character, wherein the indicator element is associated with a predetermined position where the first plurality of alphanumeric characters are displayed.

14. The method of claim 8, wherein:

the displayed graphical keyboard comprises;

an elliptical arrangement of a plurality of alphanumeric characters, wherein;

a first position of the elliptical arrangement is associated with a largest font size and a second position is associated with a smallest font size;

each alphanumeric character of the plurality of alphanumeric characters is displayed at a two-dimensional coordinate relative to the transparent display physically coupled with the at least one hardware processor;

a first alphanumeric character of the plurality of alphanumeric characters is displayed with the largest font size at the first position;

a second alphanumeric character of the plurality of alphanumeric characters is displayed with the smallest font size at the second position; and

a set of alphanumeric characters selected from the plurality of alphanumeric characters, where each alphanumeric character of the set is displayed with a font size having a value between the smallest font size and the largest font size.

15. A computer-readable medium having computer-executable instructions stored thereon that, when executed by at least one hardware processor, causes a computing device to perform a plurality of operations, the plurality of operations comprising;

displaying, on a transparent display physically coupled with the at least one hardware processor, a graphical keyboard, the graphical keyboard having at least one selectable alphanumeric character;

acquiring, by at least one inertial measurement unit, a plurality of measurements indicating user movement of the computing device;

converting, by the at least one hardware processor, the plurality of measurements to obtain a plurality of vectors, at least one vector of the plurality of vectors associated with at least three axes of movement;

determining, by the at least one hardware processor, and based on the plurality of vectors, a command to perform with the displayed graphical keyboard; and

performing, by the at least one hardware processor, the determined command with the displayed graphical keyboard.

16. The computer-readable medium of claim 15, wherein the plurality of operations further comprises:

comparing one or more values of the plurality of vectors with a plurality of corresponding thresholds, at least one of the thresholds selected from the plurality of thresholds being associated with the command to perform; and

the determining of the command to perform is further based on the comparison.

17. The computer-readable medium of claim 15, wherein:

the command to perform comprises a command to submit the at least one selectable alphanumeric character; and

the plurality of operations further comprises displaying the submitted at least one selectable alphanumeric character at a predetermined location of the graphical keyboard.

18. The computer-readable medium of claim 15, wherein:

the displayed graphical keyboard comprises;

a horizontal arrangement of alphanumeric characters, the horizontal arrangement of alphanumeric characters being substantially parallel to a ground plane; and

at least one control element that indicates whether the user intends to submit the at least one selectable alphanumeric character for display on the transparent display physically coupled with the at least one hardware processor.

19. The computer-readable medium of claim 15, wherein:

the displayed graphical keyboard comprises;

a first plurality of alphanumeric characters, the first plurality of alphanumeric characters being a subset selected from a second plurality of alphanumeric characters, wherein each alphanumeric character of the first plurality of alphanumeric character is displayed in a font size based on a relative position of the alphanumeric character; and

an indicator element that indicates an alphanumeric character selected from the first plurality of alphanumeric characters to be a submitted alphanumeric character, wherein the indicator element is associated with a predetermined position where the first plurality of alphanumeric characters are displayed.

20. The computer-readable medium of claim 15, wherein:

the displayed graphical keyboard comprises;

an elliptical arrangement of a plurality of alphanumeric characters, wherein;

a first position of the elliptical arrangement is associated with a largest font size and a second position is associated with a smallest font size;

each alphanumeric character of the plurality of alphanumeric characters is displayed at a two-dimensional coordinate relative to the transparent display physically coupled with at least one hardware processor;

a first alphanumeric character of the plurality of alphanumeric characters is displayed with the largest font size at the first position;

a second alphanumeric character of the plurality of alphanumeric characters is displayed with the smallest font size at the second position; and

a set of alphanumeric characters selected from the plurality of alphanumeric characters, where each alphanumeric character is displayed with a font size having a value between the smallest font size and the largest font size.