Enhanced character input using recognized gestures

• US 8,514,251 B2
• Filed: 06/23/2008
• Issued: 08/20/2013
• Est. Priority Date: 06/23/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan, MacDougall, Francis
• Primary Examiner(s): Amrany, Adi
• Application Number:
  US12/144,252
• Publication Number:
  US 20090315740A1
• Time in Patent Office: 1,884 Days
• Field of Search: 341/20, 345/619, 345/629, 345635-636
• US Class Current: 345/635
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Enhanced character input using recognized gestures, in which a user'"'"'s first and second gestures are recognized, and a control including radially disposed interaction elements is output. At least a portion of the interaction elements are associated with clusters of characters. When an interaction element is selected, the characters associated with the selected interaction element are disposed radially in relation to the selected interaction element. Using the control, the interaction element and a character associated with the selected interaction element are selected based on the user'"'"'s recognized first and second gestures, respectively, and the selected character is output.

Claims

1. A computer-implemented method comprising:

recognizing a user'"'"'s first and second gestures based on information from an image capture device;

determining a quantity and a position of interaction elements in a control such that the interaction elements may be placed to avoid covering one or more icons being displayed in a user interface;

outputting the control in the user interface based on the determined quantity and the determined position, wherein the control comprises;

the interaction elements being radially disposed, at least a portion of the interaction elements are associated with clusters of characters, wherein

the interaction elements in the control do not cover the one or more icons being displayed in the user interface, and

characters associated with a selected interaction element are disposed radially in relation to the selected interaction element when the interaction element is selected;

changing the quantity or position of the plurality of interaction elements and associated characters in the control in response to a change in quantity or position of icons being displayed in the user interface;

selecting, using the control, the interaction element and a character associated with the selected interaction element, based on the user'"'"'s recognized first and second gestures, respectively; and

outputting the selected character.

2. The method of claim 1, further comprising:

displaying a representation of the user in a central region of the control,

wherein the interaction elements are radially disposed in relation to the central region.

3. The method of claim 1, wherein selecting the interaction element and the character associated with the selected interaction element using the control further comprises interacting with the control.

4. The method of claim 1, wherein each character represents an alphabetic character, a numeric character, or a symbol character.

5. The method of claim 1, wherein the interaction elements further comprise:

a first interaction element associated with alphabetic characters only;

a second interaction element associated with numeric characters only; and

a third interaction element associated with symbolic characters only.

6. The method of claim 1, wherein the radially disposed interaction elements further comprise:

a first interaction element associated with a cluster of characters, and

a second interaction element associated with a function.

7. The method of claim 1, wherein the function comprises a character deletion function or a default function.

8. The method of claim 1, wherein selecting the interaction element further comprises determining whether the user'"'"'s recognized first gesture exceeds a predetermined magnitude.

9. The method of claim 1, wherein outputting the control further comprises hiding the characters associated with the selected interaction element when the users recognized first gesture moves away from the selected interaction element.

10. The method of claim 1, further comprising:

recognizing a user'"'"'s third and fourth gestures;

outputting, as the control;

supplemental interaction elements disposed radially in relation to the output selected character, of which at least a portion are associated with additional clusters of characters, and,

characters associated with a selected supplemental interaction element disposed radially in relation to the selected supplemental interaction element, when the supplemental interaction element is selected;

selecting, using the control, the supplemental interaction element and one of the characters associated with the selected supplemental interaction element, based on the user'"'"'s recognized third and fourth gestures, respectively; and

outputting the one of the characters associated with the selected supplemental interaction element.

11. The method of claim 10, further comprising hiding the output radially disposed interaction elements.

12. The method of claim 1, further comprising dynamically determining the characters associated with each cluster.

13. The method of claim 1, wherein the user'"'"'s first gesture is recognized from images of the user.

14. The method of claim 1, wherein the user'"'"'s gesture further comprises a single or multiple finger gesture;

a single hand gesture;

a single hand and arm gesture;

a single hand and arm, and body gesture;

a bimanual gesture;

a change in a head pose or posture;

a change in an eye position;

a change in facial expression;

or a change in a body pose or posture.

15. The method of claim 1, wherein the radially disposed interaction elements further comprise:

a first interaction element associated with characters;

A, B, C, D, E, and F;

a second interaction element associated with characters;

G, H, I, J, K, and L;

a third interaction element associated with characters;

M, N, O, P, Q, and R;

a fourth interaction element associated with characters;

S, T, U, V, W, and X;

a fifth interaction element associated with symbolic characters and characters;

Y and Z; and

a sixth interaction element associated with a backspace function.

16. The method of claim 1, further comprising displaying, in association with each interaction element, a visual indicia of the clusters of characters associated with the each interaction element.

17. The computer-implemented method of claim 1, wherein the radially disposed elements are disposed asymmetrically.

18. The computer-implemented method of claim 1, wherein the number of characters associated with each of the radially disposed interaction elements varies with the number of radially disposed interaction elements.

19. A non-transitory computer readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to:

recognize a user'"'"'s first and second gestures based on information from an image capture device;

determine a quantity and a position of interaction elements in a control such that the interaction elements may be placed to avoid covering one or more icons being displayed in a user interface;

output the control in a user interface based on the determined quantity and the determined position, wherein the control comprises;

the interaction elements are radially disposed, at least a portion of the interaction elements are associated with clusters of characters, wherein

the interaction elements in the control do not cover the one or more icons being displayed in the user interface, and

the characters associated with a selected interaction element are disposed radially in relation to a selected interaction element when the interaction element is selected;

change the quantity or position of the interaction elements and the characters in the control in response to a change in quantity or position of icons being displayed in the user interface;

determine that a character associated with the selected interaction element has been selected based on the user'"'"'s recognized first and second gestures, respectively; and

output the selected character.

20. A device comprising:

a user interface configured to output a control based on a determined quantity and a determined position of interaction elements, the control comprising;

radially disposed interaction elements, at least a portion of which are associated with clusters of characters, wherein

the interaction elements in the control do not cover one or more icons being displayed in the user interface, and

the characters associated with an interaction element are disposed radially in relation to the interaction element when the interaction element is selected; and

a processor configured to;

recognize a user'"'"'s first and second gestures based on information from an image capture device,

determine the quantity and the position of interaction elements in the control such that the interaction elements may be placed to avoid covering the one or more icons being displayed in the user interface,

change the quantity or position of the interaction elements and the characters in the control in response to a change in quantity or position of icons being displayed in the user interface,

select, using the control, the interaction element and a character associated with the interaction element, based on the user'"'"'s recognized first and second gestures, respectively, and

output the selected character.

21. The device of claim 20, further comprising:

a camera configured to generate images of the user,

wherein the user'"'"'s first and second gestures are recognized from the generated images.

22. A non-transitory computer readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

recognizing a user'"'"'s gesture based on information from an image capture device;

determining a quantity and a position of interaction elements for display such that the interaction elements are placed to avoid covering one or more icons being displayed in a user interface;

outputting a control in a user interface based on the determined quantity and the determined position, the control further comprises radially disposed interaction elements, each of which are associated with a character, wherein

the interaction elements in the control do not cover the one or more icons being displayed in the user interface;

changing the quantity or position of the interaction elements for display in response to a change in quantity or position of icons being displayed in the user interface;

selecting, using the control, an interaction element based on the user'"'"'s recognized gesture; and

outputting the character associated with the selected interaction element.

23. An apparatus comprising:

means for recognizing, via one or more images, a user'"'"'s first and second gestures;

means for determining a quantity and a position of interaction elements in a control such that the interaction elements may be placed to avoid one or more icons being displayed in a user interface;

means for outputting the control in the user interface based on the determined quantity and the determined position, the control further comprises;

radially disposed interaction elements, at least a portion of which are associated with clusters of characters wherein

the interaction elements in the control do not cover the one or more icons being displayed in the user interface, and

the characters associated with a selected interaction element are disposed radially in relation to the selected interaction element when the interaction element is selected;

means for changing the quantity or position of the interaction elements and the characters in the control in response to a change in quantity or position of icons being displayed in the user interface;

means for selecting, using the control, the interaction element and a character associated with the selected interaction element, based on the user'"'"'s recognized first and second gestures, respectively; and

means for outputting the selected character.

24. A computer-implemented method comprising:

recognizing a user'"'"'s first and second gestures based on information from an image capture device;

determining a quantity and a position of interaction elements for display such that the interaction elements are placed to avoid covering one or more icons being displayed in a user interface;

outputting a control in the user interface based on the determined quantity and the determined position, wherein the control comprises;

the interaction elements radially disposed around a single representation controlled by the user, at least a portion of the interaction elements being associated with clusters of characters, wherein

the interaction elements in the control do not cover the one or more icons being displayed in the user interface, and

the characters associated with a selected interaction element are disposed radially in relation to the selected interaction element when the interaction element is selected;

changing the quantity and position of the interaction elements of the control in the user interface in response to other icons being displayed in the user interface;

determining that a character associated with the selected interaction element has been selected based on the control and the user'"'"'s recognized first and second gestures; and

outputting the selected character.

25. The computer-implemented method of claim 24, wherein a quantity of characters associated with each of the radially disposed interaction elements varies with the quantity of radially disposed interaction elements.