Systems and methods for interacting with a computer through handwriting to a screen

• US 20060092138A1
• Filed: 10/29/2004
• Published: 05/04/2006
• Est. Priority Date: 10/29/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kim, Jane T., Wang, Simin, Cupala, Shiraz J., Sommer, Peter D., Eason, Natalie M.
• Granted Patent: US 8,487,879 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/171 by use of digital ink
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields

Abstract

Systems and methods are described that enable a user to: select a control with a handwritten stroke at least part of which resides outside of a selectable area of the control; use a moving-input control without having to make a selection other than handwriting on, over, or near the control; and/or delete text displayed on an electronic form by handwriting over that text.

Claims

1. One or more computer-readable media embodying computer-readable instructions which, when executed, perform acts comprising:

receiving a handwritten stroke made to a screen that is displaying a control having a selectable area, at least part of the handwritten stroke residing outside of the selectable area of the control; and

selecting the control responsive to the stroke received.

2. The media of claim 1, further comprising displaying the control having the selectable area.

3. The media of claim 1, wherein the control comprises a check box.

4. The media of claim 3, wherein the selectable area of the control is bounded by the check box.

5. The media of claim 1, wherein the control comprises a radio button.

6. The media of claim 5, wherein the selectable area of the control is bounded by the radio button.

7. The media of claim 1, wherein the act of selecting comprises selecting the control if the handwritten stroke intersects the selectable area of the control.

8. The media of claim 1, wherein the act of selecting comprises selecting the control based on a geographic relation between where on the screen the handwritten stroke is made and where on the screen the control is displayed.

9. The media of claim 1, wherein the act of selecting comprises:

determining a bounded writing area for the handwritten stroke received;

determining overlapping areas of the bounded writing area and one or more selectable areas of controls displayed on the screen; and

selecting the control having a largest overlapping area.

10. The media of claim 1, wherein the screen comprises a tablet screen and the act of receiving comprises receiving handwriting made over the tablet screen.

11. The media of claim 1, wherein the act of receiving comprises receiving handwriting made over a non-screen surface for presentation on the screen.

12. The media of claim 1, further comprising indicating that the control is selected.

13. The media of claim 1, wherein the selectable area comprises an indication that the control is selected, and further comprising determining that the handwritten stroke is for deleting information, and wherein the act of selecting the control comprises de-selecting the control.

14. A system embodying the computer-readable media of claim 1.

15. A method comprising:

displaying, on a computing device, controls over which a user is enabled to write;

geometrically bounding at least a portion of the user'"'"'s writing to provide a bounded writing area; and

comparing the bounded writing area with control geometries and, based upon one or more geometric relationships therebetween, selecting at least one control as intended to be selected by the user.

16. The method of claim 15, wherein the user'"'"'s writing does not intersect any of the control geometries.

17. The method of claim 15, wherein the user'"'"'s writing comprises a handwritten stroke.

18. The method of claim 15, wherein the act of geometrically bounding comprises computing a bounding rectangle for the user'"'"'s writing.

19. The method of claim 15, wherein the control geometries comprise selectable areas of the controls.

20. The method of claim 19, wherein at least one of the selectable areas comprises a check box.

21. The method of claim 19, wherein at least one of the selectable areas comprises a radio button.

22. The method of claim 15, wherein the control geometries comprise areas having text that describe the controls.

23. The method of claim 15, wherein the act of comparing comprises determining an overlap of the bounded writing area with the control geometries.

24. The method of claim 23, wherein the act of selecting comprises selecting the control having the control geometry with the largest overlap with the bounded writing area.

25. The method of claim 15, further comprising determining that the user'"'"'s writing is for deletion of underlying information and wherein the act of selecting at least one control comprises de-selecting the control.

26. A system comprising means for performing the method recited in claim 15.

27. One or more computer-readable media having computer-readable instructions for performing the method recited in claim 15.

28. A method comprising:

determining a region of a screen into which handwriting interpretable as moving input is capable of being made;

receiving handwriting made to the region in a mode permitting the handwriting to be interpreted as text; and

interpreting the handwriting as moving input to a moving-input control associated with the region without user interaction other than the handwriting received.

29. The method of claim 28, wherein the act of determining comprises determining the region by mapping it to an area of the screen in which the moving-input control is displayed.

30. The method of claim 28, wherein the region is further capable of receiving handwriting that is interpretable as text.

31. The method of claim 28, wherein the act of interpreting the handwriting comprises interpreting the handwriting as moving input if the handwriting begins in the region.

32. The method of claim 28, further comprising interpreting the handwriting as text if the handwriting begins outside of the region.

33. The method of claim 28, wherein the moving-input control comprises a slider-bar control.

34. The method of claim 28, wherein the moving-input control comprises a drawing control.

35. The method of claim 28, wherein the act of receiving comprises receiving handwriting made over the screen with a stylus.

36. The method of claim 28, further comprising inputting the interpreted handwriting to the moving-input control.

37. The method of claim 28, further comprising, following the acts of receiving and interpreting the first handwriting, receiving second handwriting and interpreting the second handwriting as text without user interaction other than the first and second handwriting received.

38. A system comprising means for performing the method recited in claim 28.

39. One or more computer-readable media having computer-readable instructions for performing the method recited in claim 28.

40. A method comprising:

receiving, while in a mode permitting handwriting to be interpreted as text, a first portion of a handwriting stroke, the first portion made to a screen displaying a moving-input control;

selecting, responsive to the first portion received, the moving-input control; and

interpreting, responsive to the selection, a second portion of the handwriting stroke as input to the moving-input control.

41. The method of claim 40, wherein the first portion comprises indicia indicating a start point of the handwriting stroke.

42. The method of claim 40, wherein the mode generally interprets handwriting as text.

43. The method of claim 40, wherein the moving-input control comprises a slider-bar control.

44. The method of claim 40, wherein the moving-input control comprises a drawing control.

45. The method of claim 40, wherein the act of selecting is based on a geographic relation between the first portion and a region of the screen into which moving-input can be made.

46. The method of claim 45, wherein the geographic relation comprises an intersection of the first portion and the region.

47. The method of claim 45, wherein the geographic relation is based on a comparison of a bounded writing area of the first portion and the region.

48. The method of claim 40, wherein the second portion of the handwriting stroke is received after the act of selecting is performed.

49. The method of claim 40, wherein the act of interpreting comprises interpreting the first portion of the handwriting stroke as input to the moving-input control.

50. The method of claim 40, wherein the first portion and the second portion comprise substantially all of the handwriting stroke.

51. The method of claim 40, wherein the act of receiving comprises receiving handwriting made over the screen with a stylus.

52. A system comprising means for performing the method recited in claim 40.

53. One or more computer-readable media having computer-readable instructions for performing the method recited in claim 40.

54. A computing device having a screen, computer-readable media, and a processor capable of executing the computer-readable media, that is capable of:

presenting one or more moving-input controls on the screen;

receiving handwriting made to the screen;

selecting, without user interaction independent of the handwriting received and while in a mode permitting the handwriting to be interpreted as text, a moving-input control based on a geographic relation between the handwriting received and a region of the screen capable of receiving moving input intended for one of the moving-input controls; and

interpreting the handwriting as input to the selected moving-input control.

55. The device of claim 54, wherein the capability of selecting comprises selecting the moving-input control based on a geographic relation between a first-received portion of the handwriting and the region.

56. The device of claim 55, wherein the capability of interpreting comprises interpreting a second-received portion of the handwriting as input.

57. The device of claim 54, wherein at least one of the moving-input controls is a slider-bar control.

58. The device of claim 54, wherein at least one of the moving-input controls is a drawing control.

59. The device of claim 54 embodied as a hand-held computer.

60. The device of claim 54, wherein the screen comprises a tablet screen capable of receiving handwriting made over the tablet screen.

61. One or more computer-readable media embodying computer-readable instructions which, when executed, perform acts comprising:

receiving handwriting from a user at least part of which is made over text displayed on a screen;

selecting, without interaction with the user other than the handwriting received, at least part of the text; and

determining, without interaction with the user other than the handwriting received, whether the handwriting received is for deleting the selected text.

62. The media of claim 61, wherein the act of receiving is not predicated with a selection of a data-entry field in which the text is displayed.

63. The media of claim 61, wherein the acts of receiving, selecting, and determining are performed without interaction with the user other than the handwriting received.

64. The media of claim 63, wherein the text is displayed in an unselected data-entry field.

65. The media of claim 61, wherein the handwriting comprises a single stroke.

66. The media of claim 61, wherein the handwriting comprises multiple strokes.

67. The media of claim 61, wherein the selected text comprises multiple characters.

68. The media of claim 61, wherein the selected text comprises one or more words.

69. The media of claim 61, wherein the selected text comprises one or more sentences.

70. The media of claim 61, wherein the act of determining comprises:

determining that the handwriting received represents a continuous back-and-forth motion by the user.

71. The media of claim 61, wherein the act of determining comprises:

determining that a displayed representation of the handwriting received obscures a significant portion of the selected text.

72. The media of claim 71, wherein the significant portion comprises about twenty or more percent of the text.

73. The media of claim 71, wherein the handwriting received comprises a single handwritten stroke.

74. The media of claim 61, wherein the act of determining comprises:

determining that the handwriting received comprises two or more roughly parallel lines residing substantially over the selected text.

75. The media of claim 61, further comprising deleting the selected text responsive to determining that the handwriting received is for deleting the selected text.

76. The media of claim 61, wherein the screen comprises a tablet screen and the act of receiving comprises receiving handwriting made over the tablet screen.

77. The media of claim 61, wherein the act of receiving comprises receiving handwriting made over a non-screen surface for presentation on the screen.

78. A system embodying the computer-readable media of claim 61.

79. A computing device having a screen, computer-readable media, and a processor capable of executing the computer-readable media, that is capable of:

presenting non-handwritten text on the screen;

receiving a single handwritten stroke made to the screen, at least a portion of which resides over at least a portion of the non-handwritten text; and

deleting, without user interaction independent of the single handwritten stroke received, two or more characters of the non-handwritten text.

80. The device of claim 79, wherein the capability of presenting comprising is presenting the non-handwritten text in an unselected data-entry field of an electronic data-entry form.

81. The device of claim 79, wherein the capability of deleting comprises deleting a word of the non-handwritten text.

82. The device of claim 79, wherein the capability of deleting comprises determining that the single handwritten stroke resides at least in part over the two or more characters and represents a continuous back-and-forth motion.

83. The device of claim 79, wherein the capability of deleting comprises determining that a displayed representation of the single handwritten stroke obscures twenty or more percent of the two or more characters.

84. The device of claim 79 embodied as a hand-held computer.