Remote text input using handwriting

• US 20110254765A1
• Filed: 04/18/2010
• Published: 10/20/2011
• Est. Priority Date: 04/18/2010
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Brand, Michael
• Application Number:
  US12/762,336
• Publication Number:
  US 20110254765A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A method for user input includes capturing a sequence of positions of at least a part of a body, including a hand, of a user of a computerized system, independently of any object held by or attached to the hand, while the hand delineates textual characters by moving freely in a 3D space. The positions are processed to extract a trajectory of motion of the hand. Features of the trajectory are analyzed in order to identify the characters delineated by the hand.

Claims

1. A method for user input, comprising:

capturing a sequence of positions of at least a part of a body, including a hand, of a user of a computerized system, independently of any object held by or attached to the hand, while the hand delineates textual characters by moving freely in a 3D space;

processing the positions to extract a trajectory of motion of the hand; and

analyzing features of the trajectory in order to identify the characters delineated by the hand.

2. The method according to claim 1, wherein capturing the sequence comprises capturing three-dimensional (3D) maps of at least the part of the body and processing the 3D maps so as to extract the positions.

3. The method according to claim 2, wherein processing the 3D maps comprises finding 3D positions, and wherein processing the positions comprises projecting the 3D positions onto a two-dimensional (2D) surface in the 3D space to create a 2D projected trajectory, and wherein analyzing the features comprises analyzing the projected trajectory.

4. The method according to claim 1, wherein the motion of the hand delineates the characters by writing on a virtual markerboard.

5. The method according to claim 1, wherein the motion of the hand comprises words written without breaks between at least some of the characters, and wherein analyzing the features comprises extracting the characters from the motion independently of any end-of-character indications between the characters in the motion of the hand.

6. The method according to claim 5, wherein extracting the characters comprises applying a statistical language model to the words in order to identify the characters that are most likely to have been formed by the user.

7. The method according to claim 5, wherein each word is written in a continuous movement followed by an end-of-word gesture, and wherein extracting the characters comprises processing the trajectory of the continuous movement.

8. User interface apparatus, comprising:

a sensing device, which is configured to capture a sequence of positions of at least a part of a body, including a hand, of a user of the apparatus, independently of any object held by or attached to the hand, while the hand delineates textual characters by moving freely in a 3D space; and

a processor, which is configured to process the positions to extract a trajectory of motion of the hand, and to analyze features of the trajectory in order to identify the characters delineated by the hand.

9. The apparatus according to claim 8, wherein the sensing device is configured to capture three-dimensional (3D) maps of at least the part of the body, and wherein the processor is configured to process the 3D maps so as to extract the positions.

10. The apparatus according to claim 9, wherein the processor is configured to extract 3D positions from the 3D maps and to project the 3D positions onto a two-dimensional (2D) surface in the 3D space to create a 2D projected trajectory, and to analyze the projected trajectory in order to identify the characters.

11. The apparatus according to claim 8, wherein the motion of the hand delineates the characters by writing on a virtual markerboard.

12. The apparatus according to claim 8, wherein the motion of the hand comprises words written without breaks between at least some of the characters, and wherein the processor is configured to extract the characters from the motion independently of any end-of-character indications between the characters in the motion of the hand.

13. The apparatus according to claim 12, wherein the processor is configured to apply a statistical language model to the words in order to identify the characters that are most likely to have been formed by the user.

14. The apparatus according to claim 12, wherein each word is written in a continuous movement followed by an end-of-word gesture, and wherein the processor is configured to extract the characters by processing the trajectory of the continuous movement.

15. A computer software product, comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to identify a sequence of positions of at least a part of a body, including a hand, of a user of the computer, independently of any object held by or attached to the hand, while the hand delineates textual characters by moving freely in a 3D space, to process the positions to extract a trajectory of motion of the hand, and to analyze features of the trajectory in order to identify the characters delineated by the hand.

16. The product according to claim 15, wherein the instructions cause the computer to receive and process three-dimensional (3D) maps of at least the part of the body so as to extract the positions.

17. The product according to claim 16, wherein the instructions cause the computer to extract 3D positions from the 3D maps and to project the 3D positions onto a two-dimensional (2D) surface in the 3D space to create a 2D projected trajectory, and to analyze the projected trajectory in order to identify the characters.

18. The product according to claim 15, wherein the motion of the hand delineates the characters by writing on a virtual markerboard.

19. The product according to claim 15, wherein the motion of the hand comprises words written without breaks between at least some of the characters, and wherein the instructions cause the computer to extract the characters from the motion independently of any end-of-character indications between the characters in the motion of the hand.

20. The product according to claim 19, wherein the instructions cause the computer to apply a statistical language model to the words in order to identify the characters that are most likely to have been formed by the user.

21. The product according to claim 19, wherein each word is written in a continuous movement followed by an end-of-word gesture, and wherein the instructions cause the computer to extract the characters by processing the trajectory of the continuous movement.