Electronic ink as a software object

• US 20030053084A1
• Filed: 07/19/2001
• Published: 03/20/2003
• Est. Priority Date: 07/19/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geidl, Erik M., Karlov, Donald D., Dain, Robert A.
• Granted Patent: US 7,039,234 B2
• Time in Patent Office: Days
• US Class Current: 358/1.6
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

Described is electronic ink maintained as a software object, thereby associating ink functionality with electronic ink data. The ink may be separated into words or characters, with one object per word or character. By the associated functionality, applications that deal with embedded objects in general can automatically benefit from electronic ink, including having the object'"'"'s functionality render the ink data as part the application'"'"'s document. Further, because the ink data is maintained as an object, the data is automatically persisted in association with the document into which it is embedded. Ink-aware applications may call on methods of the electronic ink object to adjust formatting, search recognized ink along with text, and perform other functions. Via the electronic ink object, electronic ink substantially approaches much of the behavior normally available with text data, without requiring applications to interpret the ink data.

Claims

1. In a computing device, a system comprising:

an input device configured to receive handwriting input and convert the handwriting input to electronic ink information;

a mechanism configured to provide an object to maintain electronic ink data that corresponds to the electronic ink information, the object having at least one interface;

executable code associated with the object and comprising functionality arranged to interpret the electronic ink data and output a representation of the electronic ink data therefrom; and

a software module that invokes the functionality of the object via the at least one interface to cause the object to interpret the electronic ink data to output the representation.

2. The system of claim 1 wherein the input device comprises a tablet.

3. The system of claim 1 further comprising a display device, wherein the object outputs the representation of the electronic ink data as a visual image to the display device.

4. The system of claim 1 wherein the software module comprises an application program, and wherein the functionality is further arranged to enable the application program to control the output of the representation.

5. The system of claim 4 wherein the output comprises a displayed image, and wherein the application program controls the output by requesting a particular representation of the displayed image.

6. The system of claim 1 wherein the output comprises a displayed image, and wherein the object includes additional data and functionality for adjusting the image appearance to correspond to information displayed relative to the image.

7. The system of claim 1 further comprising a handwriting recognizer that converts electronic ink information to other information, and wherein the object maintains at least some results from the handwriting recognizer.

8. The system of claim 1 wherein the object is further associated with functionality comprising, a method directed to performing at least one of the following operations:

editing, formatting and searching.

9. The system of claim 1 wherein the object is further associated with functionality comprising, a method directed to performing at least one of the following operations:

combining multiple objects into a single object, and separating a single object into multiple objects.

10. The system of claim 1 further comprising, a handwriting recognizer that converts electronic ink information to words, and wherein the object represents a single word.

11. The system of claim 1 wherein the object includes additional data comprising at least one of the following types of data:

normalization data, previous object identification data, and next object identification data.

12. A method for providing electronic ink to an application for display thereof, comprising:

receiving electronic ink data;

creating an electronic ink object for maintaining information that corresponds to the electronic ink data, the object having ink-related functionality associated therewith including a method that when called interprets the electronic ink data and renders an image of the ink data therefrom;

embedding the object into a document that has data therein arranged in a format used by the application; and

providing an interface to the application to call the method, the application calling the method via the interface to render the image of the ink data.

13. The method of claim 12 further comprising, normalizing the received electronic ink data into the information that corresponds to the electronic ink data.

14. The method of claim 13 further comprising, maintaining normalizing information with the electronic ink object.

15. The method of claim 12 further comprising, providing the electronic ink data to a recognizer, and receiving a recognition result therefrom.

16. The method of claim 12 wherein the recognition result includes at least one text word alternate, and further comprising, maintaining the at least one text word alternate with the electronic ink object.

17. The method of claim 12 wherein the recognition result includes baseline information, and further comprising, maintaining the baseline information with the electronic ink object.

18. The method of claim 17 wherein the image is rendered based on the baseline information.

19. The method of claim 12 wherein the image is rendered at a location that is relative to text in the document.

20. The method of claim 19 wherein the electronic ink has a baseline associated therewith that is maintained in the object, and wherein the image is rendered at the location that is relative to the text in the document by aligning the image with the text via the baseline.

21. The method of claim 12 wherein the image is rendered with a size that is relative to text in the document.

22. The method of claim 21 wherein the electronic ink has a baseline and a midline associated therewith, and wherein the size is determined based on the baseline and midline.

23. The method of claim 21 wherein the electronic ink has a ceiling associated therewith, and further comprising, determining whether the electronic ink data includes an ascender that ascends above the ceiling, and if so, scaling the ascender to not ascend above the ceiling.

24. The method of claim 21 wherein the electronic ink has a floor associated therewith, and further comprising, determining whether the electronic ink data includes a descender that descends below the floor, and if so, scaling the descender to not descend below the floor.

25. The method of claim 12 wherein embedding the object comprises, pasting the object into the document.

26. The method of claim 12 further comprising, exposing an interface to the application including a method to enable the application to format the ink data, receiving a call to that method including formatting parameter data, and modifying the image based on the formatting parameter data.

27. The method of claim 12 further comprising, providing the electronic ink data to a recognizer, receiving a recognition result therefrom including a text word alternate, maintaining the text word alternate with the electronic ink object, and exposing an interface to the application, the interface having an associated method that enables the application to obtain information about the text word alternate.

28. A computer-readable medium having computer-executable instructions for performing the method of claim 12.

29. (Amended) A computer-readable medium having stored thereon a data structure, comprising,

a data field including electronic ink data;

at least one interface for invoking functionality associated with the data structure, including functionality for rendering a representation of the electronic ink data; and

the functionality invoked via the interface by executable code hosting a document containing the data structure, to render a representation of the electronic ink data relative to other information contained in the document.

30. The data structure of claim 29 further comprising, other functionality for matching the appearance of the representation to the other information contained in the document.

31. The data structure of claim 29 wherein the other information contained in the document comprises text data, and further comprising, at least one other data field in the data structure for maintaining adjustment information for adjusting the representation to match the text data, and other functionality associated with the data structure for adjusting the representation based on the adjustment information to match the text data.

32. The data structure of claim 31 wherein the at least one other field contains baseline data, and wherein the functionality adjusts the representation to match the text data by evaluating the baseline information.

33. The data structure of claim 31 wherein the at least one other field contains midline data, and wherein the functionality adjusts the representation to match the text data by evaluating the midline information.

34. The data structure of claim 29 further comprising, at least one other data field that maintains identifier information of another object.

35. The data structure of claim 29 further comprising, at least one other data field that maintains information related to normalization of the electronic ink data.

36. The data structure of claim 29 further comprising, at least one other data field that maintains information related to formatting the electronic ink data, and further comprising other functionality associated with the data structure to enable the executable code to modify the information related to formatting the electronic ink data.

37. The data structure of claim 29 further comprising, at least one other data field that maintains color information related to a color of the electronic ink data, and further comprising other functionality associated with the data structure to enable the executable code to modify the color information.

38. The data structure of claim 29 further comprising, at least one other data field that maintains size information related to a displayed size of the electronic ink data, and further comprising other functionality associated with the data structure to enable the executable code to modify the size information.

39. The data structure of claim 29 further comprising, at least one other data field that maintains recognition information related to recognition of the electronic ink data by a recognition engine.

40. The data structure of claim 39 wherein the at least one other field contains recognition state information.

41. The data structure of claim 39 wherein the at least one other field contains a result from the recognition engine.

42. The data structure of claim 41 wherein the result from the recognition engine includes at least one candidate corresponding to the electronic ink data.

43. The data structure of claim 42 further comprising other functionality associated with the data structure to enable the executable code to obtain information about the at least one candidate.