Distinguishing gestures from handwriting in a pen based computer by stroke analysis

• US 5,864,635 A
• Filed: 06/14/1996
• Issued: 01/26/1999
• Est. Priority Date: 06/14/1996
• Status: Expired due to Term

• Current Assignee: Tomtom Global Assets BV (TomTom NV)
• Original Assignee: International Business Machines Corporation
• Inventors: Zetts, John Mark, Desrosiers, Maurice Roger
• Primary Examiner(s): SHALWALA, BIPIN H
• Application Number:
  US08/665,128
• Time in Patent Office: 956 Days
• Field of Search: 382/181, 382/182, 382/184, 382/185, 382/186, 382/187, 382/189, 382/190, 382/192, 382/194, 382/229, 382/202, 341/144, 341/155, 341/358
• US Class Current: 382/187
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction

Abstract

Pre-recognition analysis on stroke characteristics such as the count, size and position of each stroke in real-time as it is drawn improves recognition accuracy. After each stroke, the set of strokes is weighted toward handwriting or gesture. The system uses a gesture threshold size to distinguish between gestures and handwriting. The system also uses the stroke count to distinguish between the two inputs, relying on the knowledge of the allowable number of strokes in a gesture. The count information may also be used in conjunction with the stroke size information to weight the set of strokes between gestures handwriting. Once the stroke size crosses a gesture vs. text size threshold, the result is weighted toward gestures. By examining the `white space` between strokes and the juxtaposition of the strokes, a gesture vs. text determination can be made with high accuracy.

Claims

1. A method of distinguishing between handwriting and gesture input to a pen based computer system, comprising the steps of:

detecting a set of strokes input by a stylus to the computer system;

prior to recognition, sequentially analyzing the set of strokes for either a first set of characteristics indicative of a handwriting input or a second set of characteristics indicative of a gesture input;

prior to recognition, after analyzing each stroke, for either the first or the second set of characteristics of all analyzed strokes, categorizing the set of strokes as gesture or handwriting input; and

based on whether the set of strokes is categorized as gesture or handwriting input, Performing gesture recognition or handwriting recognition on the set of strokes to recognize the set of strokes as a particular gesture or handwriting.

2. The method as recited in claim 1 wherein categorizing step further comprises the step of categorizing the set of strokes as unknown input.

3. The method as recited in claim 1 wherein the categorizing step further comprises the step of after a predetermined number of strokes, categorizing the stroke as handwriting input.

4. The method as recited in claim 1 the categorizing step further comprises the step of responsive to a last analyzed stroke of a predetermined type, categorizing the set of strokes as gesture input.

5. The method as recited in claim 4 wherein the predetermined type of stroke is a tap stroke.

6. The method as recited in claim 1 wherein responsive to categorizing the set of strokes as handwriting input, the set of strokes is sent for handwriting recognition only.

7. The method as recited in claim 1 wherein responsive to categorizing the set of strokes as gesture input, the set of strokes is sent for gesture and handwriting recognition.

8. The method as recited in claim 7 further comprising the steps of:

building a results data structure containing a recognized gesture as a primary result and a recognized text character as a secondary result; and

sending the results data structure to a target application for further processing.

9. The method as recited in claim 1 wherein the categorizing step further comprises the step of testing for a window setting for a type of input accepted for a window in which the set of strokes were input for a type of input accepted setting.

10. The method as recited in claim 1 wherein the categorizing step further comprises the steps of:

analyzing a separation distance between strokes; and

responsive to a determination that the separation distance exceeds a predetermined distance, categorizing the set of strokes as handwriting input.

11. The method as recited in claim 10 wherein the separation distance between strokes is analyzed in the horizontal direction only.

12. The method as recited in claim 1 wherein the analyzed step further comprises the step of analyzing each stroke for size.

13. The method as recited in claim 12 wherein the categorizing step further comprises the step of categorizing the set of strokes as gesture input if all analyzed strokes are larger than a predetermined size.

14. The method as recited in claim 1 wherein the categorizing step further comprises the step of categorizing the set of strokes as handwriting or gesture input based on an elapsed time during which the set of strokes are input.

15. The method as recited in claim 1 wherein the categorizing step further comprises the steps of:

counting a total number of analyzed strokes; and

counting a number of analyzed strokes which exceed a predetermined size.

16. A system including processor, memory, display and, pen input device for distinguishing between handwriting and gesture input, comprising:

a handwriting recognition engine for recognizing pen input as handwriting input;

a gesture recognition engine for recognizing pen input as gesture input;

means for detecting a set of strokes input by a stylus to the computer system;

means for sequentially analyzing the set of strokes for a first or a second set of characteristics respectively indicative of a handwriting input or a gesture input prior to recognition;

means responsive to analyzing each stroke for either the first or the second set of characteristics of all analyzed strokes for categorizing the set of strokes as gesture or handwriting input prior to recognition; and

means for sending the set of strokes to one or both of the handwriting recognition engine or the gesture recognition engine for recognizing the set of strokes as a particular handwriting or gesture on the basis of the categorization.

17. The system as recited in claim 16 wherein the categorizing means further comprises means for categorizing the stroke as handwriting input after a predetermined number of strokes.

18. The system as recited in claim 16 wherein the categorizing means further comprises means responsive to a last analyzed stroke of a predetermined type for categorizing the set of strokes as gesture input.

19. The system as recited in claim 16 wherein responsive to categorizing the set of strokes as gesture input the set of strokes is sent to the gesture recognition engine.

20. The system as recited in claim 16 wherein the categorizing means further comprises means for testing for a window for a type of input accepted setting for a window in which the set of strokes were input.

21. The system as recited in claim 16 wherein the categorizing means further comprises:

means for analyzing a separation distance between strokes; and

means responsive to a determination that the separation distance exceeds a predetermined distance for categorizing the set of strokes as handwriting input.

22. The system as recited in claim 16 wherein the analyzed means further comprises means for analyzing each stroke for size.

23. The system as recited in claim 16 wherein the categorizing means further comprises means for categorizing the set of strokes as handwriting or gesture input based on an elapsed time during which the set of strokes are input.

24. A computer program product in a computer readable memory for distinguishing between handwriting and gesture input to a computer comprising:

means for detecting a set of strokes input by a stylus to the computer system;

means for sequentially analyzing each of the strokes either for a first set of characteristics indicative of a handwriting input or a second set of chracteristics indicative a gesture input prior to recognition;

means for categorizing the set of strokes as gesture or handwriting input for either the first or the second set of characteristics of all analyzed strokes and prior to recognition; and

means responsive to categorization of the set of strokes as gesture or handwriting input for performing gesture recognition or handwriting recognition on the set of strokes to recognize the set of strokes as a particular gesture or handwriting.

25. The product as recited in claim 24 wherein the categorizing means further comprises means for categorizing the stroke as handwriting after a predetermined number of strokes are input.

26. The product recited in claim 24 the categorizing means further comprises means responsive to a last analyzed stroke of a predetermined type for categorizing the set of strokes as gesture input.

27. The product as recited in claim 24 responsive to categorizing the set of strokes as gesture input the set of strokes is sent for gesture recognition.

28. The product as recited in claim 24 wherein the categorizing means further comprises means for testing for a window setting for a window for a type of input accepted in which the set of strokes were input.

29. The product as recited in claim 24 wherein the categorizing step further comprises:

means for analyzing a separation distance between strokes; and

means responsive to a determination that the separation distance exceeds a predetermined distance for categorizing the set of strokes as handwriting input.

30. The product as recited in claim 24 wherein the analyzed means further comprises means for analyzing each stroke for size.

31. The product as recited in claim 24 wherein the categorizing means further comprises means for categorizing the set of strokes as handwriting or gesture input, based on an elapsed time during which the set of strokes are input.