GESTURE CONTROL TECHNIQUES FOR USE WITH DISPLAYED VIRTUAL KEYBOARDS

• US 20130239041A1
• Filed: 03/06/2012
• Published: 09/12/2013
• Est. Priority Date: 03/06/2012
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: DaCosta, Behram Mario
• Application Number:
  US13/413,061
• Publication Number:
  US 20130239041A1
• Time in Patent Office: Days
• US Class Current: 715/773
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A gesture control technique includes receiving data indicating one or more of a presence, location, position, motion, and direction of a user finger or the like from a location interface, such as a camera, touch pad, gyroscope or the like. A graphical user interface including a virtual keyboard is displayed on a display. The presence, location, position, motion and direction data compared to a view space location of the virtual keyboard on the display to determine if the presence, location, position, motion and direction data is associated with location on the virtual keyboard. The presence, location, position, motion and direction data are displayed overlaid on the virtual keyboard, if the presence, location, position, motion and direction data is associated with the virtual keyboard. In addition, one or more gestures are determined from one or more sets of presence, location, position, motion and direction data. Furthermore, one or more alphanumeric or control inputs of the virtual keyboard are determined from the one or more gestures.

Claims

1. A method comprising:

receiving, by a location interface, data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member;

displaying a graphical user interface including a virtual keyboard on a display;

determining, by a processing unit, if the presence, location, position, motion and direction data is associated with a view space location of the virtual keyboard on the display;

determining, by the processing unit, one or more gestures from one or more sets of presence, location, position, motion and direction data;

displaying the presence, location, position, motion and direction data on the display overlaid on the virtual keyboard, if the presence, location, position, motion and direction data is associated with the view space location of the virtual keyboard; and

determining, by the processing unit, one or more alphanumeric or control inputs from the virtual keyboard corresponding to the one or more gestures.

2. The method according to claim 1, further comprising sending, by the processing unit, the one or more alphanumeric or control keys to one or more programs.

3. The method according to claim 1, further comprising displaying the one or more alphanumeric or control keys in an input field of the graphical user interface on the display.

4. The method according to claim 1, wherein the presence, location, position, motion and direction data is displayed in a first format on the display, if the presence, location, position, motion and direction data is associated with the view space location of the virtual keyboard.

5. The method according to claim 4, further comprising displaying the presence, location, position, motion and direction data on the display in a second format overlaid on another corresponding portion of the graphical user interface, if the presence, location, position, motion and direction data is not associated with the view space location of the virtual keyboard.

6. The method according to claim 1, wherein the location interface for receiving the data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member comprises a camera.

7. The method according to claim 1, wherein the location interface for receiving the data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member comprises a touch pad.

8. The method according to claim 1, wherein the location interface for receiving the data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member comprises a gyroscope or accelerometer of a television remote control or smart phone.

9. The method according to claim 1, wherein the one or more gestures are further determined, by the processing unit, from a context, one or more previous gestures, one or more input from another source, one or more applications, or one or more data sets.

10. The method according to claim 1, wherein the one or more alphanumeric or control inputs are further determined, by the processing unit, from one or more of a layout of the virtual keyboard, a context, one or more previous alphanumeric or control inputs, one or more inputs from another source, one or more applications, one or more user preferences, one or more previous uses, one or more data sets, an interactive program guide, a spell check dictionary, one or more available menu choices, an autocomplete algorithm, or a disambiguation algorithm.

11. A system comprising:

a means for receiving data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member;

a means for displaying a graphical user interface including a virtual keyboard;

a means for determining if the presence, location, position, motion and direction data is associated with a view space location of the virtual keyboard on the display;

a means for determining one or more gestures from one or more sets of presence, location, position, motion and direction data;

a means for displaying the presence, location, position, motion and direction data on the display overlaid on the virtual keyboard, if the presence, location, position, motion and direction data is associated with the view space location of the virtual keyboard;

a means for disambiguously determining one or more alphanumeric or control inputs from the virtual keyboard corresponding to the one or more gestures; and

a means for displaying the one or more alphanumeric or control keys in an input field of the graphical user interface.

12. The system of claim 11, wherein the presence, location, position, motion and direction data is displayed in a first format, if the presence, location, position, motion and direction data is associated with the view space location of the virtual keyboard.

13. The system of claim 12, further comprising displaying the presence, location, position, motion and direction data in a second format overlaid on another corresponding portion of the graphical user interface, if the presence, location, position, motion and direction data is not associated with the view space location of the virtual keyboard.

14. The system of claim 11, wherein the means for receiving data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member is received relative to a first and second plane in space.

15. One or more computing device readable media including computing device executable instructions that when executed by a processing unit implement a gesture control process comprising:

receiving data indicating one or more of a presence, location, position, motion, and direction of a user gesturing member from a location interface;

displaying a graphical user interface including a virtual keyboard on a display;

determining if the presence, location, position, motion and direction data is associated with a view space location of the virtual keyboard on the display;

determining one or more gestures from one or more sets of presence, location, position, motion and direction data;

displaying the presence, location, position, motion and direction data on the display overlaid on the virtual keyboard, if the presence, location, position, motion and direction data is associated with the view space location of the virtual keyboard; and

determining one or more alphanumeric or control inputs from the virtual keyboard corresponding to the one or more gestures.

16. The one or more computing device readable media including computing device executable instructions that when executed by the processing unit implement the gesture control process according to claim 15, further comprising sending the one or more alphanumeric or control keys to one or more programs.

17. The one or more computing device readable media including computing device executable instructions that when executed by the processing unit implement the gesture control process according to claim 15, further comprising displaying the one or more alphanumeric or control keys in an input field of the graphical user interface.

18. The one or more computing device readable media including computing device executable instructions that when executed by the processing unit implement the gesture control process according to claim 15, wherein the one or more gestures are disambiguously determined from one or more of a context, one or more previous gestures, a layout of the virtual keyboard, one or more input from another source, one or more applications, or one or more data sets.

19. The one or more computing device readable media including computing device executable instructions that when executed by the processing unit implement the gesture control process according to claim 15, wherein the one or more alphanumeric or control inputs are disambiguously determined from one or more of a layout of the virtual keyboard, a context, one or more previous alphanumeric or control inputs, one or more inputs from another source, one or more applications, one or more user preferences, one or more previous uses, one or more data sets, an interactive program guide, a spell check dictionary, one or more available menu choices.

20. The one or more computing device readable media including computing device executable instructions that when executed by the processing unit implement the gesture control process according to claim 15, wherein the presence, location, position, motion and direction data of the user gesturing member is optically received relative to a first and second plane in space.