System and method for injecting ink into an application

• US 8,508,489 B2
• Filed: 03/03/2009
• Issued: 08/13/2013
• Est. Priority Date: 06/07/2002
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Ieperen, Taco Van, Wilde, Keith Correy
• Primary Examiner(s): SHERMAN, STEPHEN G
• Application Number:
  US12/396,713
• Publication Number:
  US 20090160818A1
• Time in Patent Office: 1,624 Days
• Field of Search: 345173-179, 345/156, 178 1801- 1809, 178/18.11, 178 1901- 1906, 715/863
• US Class Current: 345/173
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A touch system includes a touch panel having a touch surface and a projector for projecting images onto the touch surface. A computer executing an applications program is coupled to the touch panel and the projector. The computer is responsive to contact output generated by the touch panel in response to proximity of a pointer to the touch surface and updates image data conveyed to the projector so that the images presented on the touch surface reflect the pointer activity. The computer executes a gesture responsive ink injection routine to detect when a selected gesture is made with the pointer on the touch surface. The gesture responsive ink injection routine performs recognition to convert ink drawn on the touch surface into a form suitable for the applications program when the selected gesture is recognized and enters the recognized text into the application.

Claims

1. A method comprising:

receiving handwritten input and displaying the handwritten input on a graphic surface presented on a display;

in response to a stationary pointer contact on a touch surface exceeding a first non-zero threshold duration, updating the graphic surface so as to provide a visual indication of the start of a potential input gesture entry;

determining when an input gesture has been entered following said updating, said input gesture indicating a position in an application;

in response to the input gesture, performing recognition on the handwritten input to convert the handwritten input into a different form; and

entering the converted handwritten input into said application substantially at the position indicated by said input gesture.

2. The method of claim 1 wherein said input gesture is determined when the stationary pointer contact on said touch surface exceeds a second non-zero threshold duration.

3. The method of claim 2 further comprising displaying said converted handwritten input on said graphic surface.

4. The method of claim 3 wherein said converted handwritten input is displayed on said graphic surface at a location generally corresponding to the location of said stationary pointer contact on said touch surface.

5. The method of claim 1 further comprising displaying visual feedback on said graphic surface to signify recognition of said gesture.

6. The method of claim 5 wherein said displayed visual feedback is text.

7. The method of claim 6 wherein said text comprises a list of selectable items, at least one selectable item in said list being a text representation of the converted handwritten input.

8. The method of claim 7 wherein said entering is performed upon selection of said at least one selectable item.

9. The method of claim 8 wherein said list of selectable items comprises a clear item, said method being terminated upon selection of said clear item.

10. The method of claim 7 wherein said list of selectable items comprises a plurality of possible text representations of the converted handwritten input.

11. The method of claim 10 wherein upon selection of one of the plurality of possible text representations of the converted handwritten input, the entering is performed using converted handwritten input corresponding to the selected possible text representation.

12. The method of claim 11 wherein said list of selectable items comprises a clear item, said method being terminated upon selection of said clear item.

13. The method of claim 6 wherein said text is a representation of the converted handwritten input when the percentage accuracy of recognition is over a threshold.

14. The method of claim 13 wherein said percentage accuracy threshold is 90%.

15. The method of claim 5 wherein said visual feedback is displayed prior to entering the converted handwritten input into said application.

16. The method of claim 1 wherein said graphic surface comprises a computer desktop image.

17. A method comprising:

receiving handwritten input entered via pointer interaction with a touch surface and displaying the handwritten input on a graphic surface presented on said touch surface;

in response to a stationary pointer contact on the touch surface exceeding a first non-zero threshold duration, updating the graphic surface so as to provide a visual indication of the start of a potential input gesture entry;

determining when a gesture has been input via pointer interaction with said touch surface following said updating, said gesture indicating a position in an application;

in response to said gesture, performing recognition on the displayed handwritten input to convert the handwritten input into a different form;

displaying the recognition result together with a clear result option;

entering the converted handwritten input into said application substantially at the position indicated by said gesture and displaying the converted handwritten input on said touch surface when the displayed recognition result is selected; and

clearing the recognition result when said clear result option is selected.

18. The method of claim 17 wherein said gesture is a stationary pointer contact on said touch surface that exceeds a threshold duration.

19. The method of claim 18 wherein said converted handwritten input is displayed on said touch surface at a location generally corresponding to the location of said stationary pointer contact on said touch surface.

20. The method of claim 17 wherein the displayed recognition result comprises a plurality of selectable possible text representations of said handwritten input.

21. A method comprising:

displaying a graphic surface on a touch surface to provide a surface for handwritten input;

receiving handwritten input entered via pen tool interaction with said touch surface and displaying the handwritten input on the graphic surface;

receiving mouse input entered via finger interaction with said touch surface and entering the mouse input into a computer application;

in response to a stationary pen tool contact on said touch surface exceeding a first non-zero threshold duration, updating the graphic surface so as to provide a visual indication of the start of a potential input gesture entry;

determining when a gesture has been input via pen tool interaction with said touch surface following said updating;

in response to said gesture, performing recognition on the displayed handwritten input to convert the handwritten input into a different form; and

entering the converted handwritten input into said computer application substantially at a position indicated by said gesture.

22. The method of claim 21 wherein said gesture is a stationary pointer contact on said touch surface that exceeds a threshold duration.

23. The method of claim 22 wherein said converted handwritten input is displayed on said touch surface at a location generally corresponding to the location of said stationary pointer contact on said touch surface.

24. The method of claim 21 further comprising displaying visual feedback to signify recognition of said gesture.

25. The method of claim 24 wherein said visual feedback is displayed prior to entering the converted handwritten input into said application.

26. The method of claim 21 wherein said displayed visual feedback is text.

27. The method of claim 26 wherein said text comprises a list of selectable items, at least one selectable item in said list being a text representation of the converted handwritten input.

28. The method of claim 27 wherein said entering is performed upon selection of said at least one selectable item.

29. The method of claim 28 wherein said list of selectable items comprises a clear item, said method being terminated upon selection of said clear item.

30. The method of claim 28 wherein said list of selectable items comprises a plurality of possible text representations of the converted handwritten input.

31. The method of claim 30 wherein upon selection of one of the plurality of possible text representations of the converted handwritten input, the entering is performed using converted handwritten input corresponding to the selected possible text representation.

32. The method of claim 31 wherein said list of selectable items comprises a clear item, said method being terminated upon selection of said clear item.

33. The method of claim 26 wherein said text is a representation of the recognized handwritten input when the percentage accuracy of recognition is over a threshold.

34. The method of claim 33 wherein said percentage accuracy threshold is 90%.

35. A touch system comprising:

a touch panel having a touch surface;

a device for generating an image that is presented on said touch surface; and

computing structure executing at least one program and being coupled to said touch panel and said device, said computing structure being responsive to output generated by said touch panel in response to proximity of a pointing device to said touch surface and updating image data conveyed to said device so that images presented on said touch surface reflect the activity of said pointing device, said computing structure executing a routine to detect when a stationary pointing device contact on said touch surface that exceeds a first non-zero threshold duration occurs and in response updating the image data to signify visually the start of a potential input gesture entry, to determine when a gesture is made via interaction of said pointing device with said touch surface following said updating, in response to said gesture, to convert handwritten input drawn on said touch surface into a form suitable for said at least one program, to display the result of the conversion together with a clear result option, to enter the handwritten input into said at least one program substantially at a position indicated by said gesture upon selection of the conversion result and to clear the conversion result upon selection of said clear result option.

36. A touch system according to claim 35 wherein the conversion result comprises one or more possible text representations of the handwritten input, and wherein the converted handwritten input corresponding to a selected text representation is entered into said at least one program.

37. A touch system comprising:

a touch panel having a touch surface;

a device for generating an image that is presented on said touch surface; and

computing structure executing at least one program and being coupled to said touch panel and said device, said computing structure being responsive to output generated by said touch panel in response to proximity of a pointing device to said touch surface and updating image data conveyed to said device so that images presented on said touch surface reflect the activity of said pointing device, said computing structure executing a routine to detect when a stationary pointing device contact on said touch surface that exceeds a first non-zero threshold duration occurs and in response updating the image data to signify visually the start of a potential input gesture entry, to determine when a gesture is made via interaction of said pointing device with said touch surface following said updating, to display visual feedback signifying recognition of said gesture, in response to said gesture to convert handwritten input drawn on said touch surface into a form suitable for said at least one program, and to enter the handwritten input into said at least one program substantially at a position indicated by said gesture upon selection of the result.

38. A touch system according to claim 37 wherein said visual feedback is at least one possible text representation of the handwritten input.

39. A computing system comprising:

computing structure executing an application; and

a display device communicating with said computing structure, said display device presenting an image responsive to computing structure image data, wherein said computing structure is configured to;

detect handwritten input;

in response to a stationary pointer contact on a touch surface exceeding a first non-zero threshold duration, update the graphic surface so as to provide a visual indication of the start of a potential input gesture entry;

determine when an input gesture has been entered following the update, said input gesture representing a command for converting the handwritten input and an indication of a position in the application;

in response to the input gesture, convert the handwritten input into a different form; and

enter the converted handwritten input into said application-substantially at the position indicated by said gesture.