Gesture Keyboarding

• US 20100199228A1
• Filed: 02/23/2009
• Published: 08/05/2010
• Est. Priority Date: 01/30/2009
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tsunoda, Kudo, Geisner, Kevin, Perez, Kathryn Stone, Markovic, Relja, Latta, Stephen G., Bennett, Darren Alexander
• Application Number:
  US12/391,145
• Publication Number:
  US 20100199228A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

Systems, methods and computer readable media are disclosed for gesture keyboarding. A user makes a gesture by either making a pose or moving in a pre-defined way that is captured by a depth camera. The depth information provided by the depth camera is parsed to determine at least that part of the user that is making the gesture. When parsed, the character or action signified by this gesture is identified.

Claims

1. A method for providing keyboard-like input to a computer system that accepts gesture input, comprising:

receiving data comprising at least one image of a user creating an input gesture; and

parsing the data to determine at least one gestured character.

2. The method of claim 1, wherein each gestured character comprises an alphanumeric character.

3. The method of claim 1, wherein the input gesture comprises a gesture in a sign language.

4. The method of claim 3, wherein the sign language comprises American Sign Language (ASL).

5. The method of claim 1, wherein the user creates the input gesture using a physical prop.

6. The method of claim 5, wherein the physical prop comprises a keyboard that is not physically connected to the computer system.

7. The method of claim 1, wherein the user creates the input gesture by mimicking use of a phantom prop.

8. The method of claim 7, wherein the phantom prop is a phantom ball or a phantom keyboard.

9. The method of claim 8, wherein the user creates the input gesture by mimicking throwing the phantom ball at a plurality of characters displayed on the display device, and the data is parsed to determine the gestured character by determining where the phantom ball would intersect with the display device if it were a physical ball.

10. The method of claim 1, wherein the user creates an input gesture with two hands, the user'"'"'s first hand selecting a group of characters and the user'"'"'s second hand selecting an character from the group of characters.

11. The method of claim 10, wherein the gestured character comprises kanji.

12. The method of claim 1, further comprising:

comparing the gestured character against at least one previously gestured character;

determining that at least one error character exists in the set consisting of the gestured character and each previously gestured character; and

replacing each error character with a corresponding correct character.

13. A system for recognizing user movement as character input to a computing system, comprising:

a processor;

a data gatherer that receives image data corresponding to a user gesture; and

a gesture recognizer engine that determines at least one character that corresponds to the user gesture.

14. The system of claim 13, further comprising:

a feedback mechanism that sends feedback to the user based on the input gesture.

15. The system of claim 14, wherein the feedback consists of audio feedback, visual feedback, changing the color of a display element, lining of a display element, fading of a display element, strobing of a display element or a tracer pattern.

16. The system of claim 13, further comprising a display device that displays a plurality of characters and a cursor, wherein the gesture recognizer engine recognizes at least one gesture that corresponds to movement of the cursor on the display and at least one gesture that corresponds to selection of a displayed character in proximity to the cursor.

17. The system of claim 13, further comprising:

a media translator that converts each gestured character to a spoken equivalent or a text equivalent.

18. The system of claim 13, further comprising:

a language translator that translates each character from a first language to a second language.

19. The system of claim 13, wherein the data gatherer comprises a depth camera.

20. A computer readable storage medium, comprising computer readable instructions that when executed on a processor, cause the processor to perform the operations of:

receiving depth data from a depth camera, the depth data comprising at least one image of a user creating an input gesture; and

parsing the data to determine a gestured character.