Universal file format for digital rich ink data

• US 6,850,228 B1
• Filed: 10/29/1999
• Issued: 02/01/2005
• Est. Priority Date: 10/29/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lui, Charlton E., Parks, Michael Jay
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): Sajous, Wesner
• Application Number:
  US09/430,150
• Time in Patent Office: 1,922 Days
• Field of Search: 345/169, 345/173, 345/179, 345/180, 345/181, 345/182, 345/183, 345/530, 345/531, 345/22, 345/24, 345/27, 345 87- 88, 345591-593, 345/597, 345/600, 345/603, 345/604, 345/605, 345543-544, 345/546, 345/555, 382174-177, 382/178, 382/179, 382187-189, 382/190, 382194-195, 382/162, 382/165, 382/166
• US Class Current: 345/173
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 2210/32 Image data format

Abstract

A universal file format for providing digital rich ink data documents. Digital rich ink data is hidden among the ink image data otherwise stored in a widely-accepted image file format. For example, depending on the type of file, data can be concealed in the least significant bit or bits of pixel information, or stored in-different color indexes of a palette that result in the same color output. In other file types, digital rich ink data may be appended after the end of file marker, or placed in a comment area. When such a file is accessed, computing devices capable of handling rich ink data recognize and extract the rich ink data from the file, while devices incapable of recognizing the format are still able to display the ordinary ink data. Two file formats for describing the arrangement of rich ink data are also provided.

Claims

1. A computer-readable medium having computer-executable instructions for performing steps comprising:

providing rich ink data having a first format, the rich ink data including movement information; and

preserving the rich ink data including concealing the rich ink data within a file having a second format, wherein an application or device that is not programmed to recognize the rich ink data processes the file normally, while an application or device that is programmed to recognize the rich ink data extracts the rich ink data from the file.

2. The computer-readable medium of claim 1, wherein the file having a second format comprises an image file format.

3. The computer-readable medium of claim 1, wherein the file includes a logical end of file, and wherein concealing the rich ink data into the file includes appending the rich ink data after the logical end of the file.

4. The computer-readable medium of claim 1, wherein the second format includes color palette indexes to a color palette, and wherein rich ink data is concealed in at least one least significant bit of at least some of the indexes to the color palette.

5. The computer-readable medium of claim 4, wherein each color palette index is eight bits in length, and wherein concealed rich ink data is stored in the least significant bit of selected color palette indexes.

6. The computer-readable medium of claim 1, wherein the second format comprises color data representing color values in a component color scheme, and wherein the rich ink data is concealed in at least one least significant bit of at least some of the color values.

7. The computer-readable medium of claim 6, wherein the component color scheme comprises 24-bit color.

8. The computer-readable medium of claim 6, wherein the component color scheme comprises three color component values for each pixel represented, and wherein the rich ink data is stored in the least significant bit of at least some of the color component values.

9. The computer-readable medium of claim 1, wherein the second format includes comments, and wherein the rich ink data is concealed in the comments.

10. The computer-readable medium of claim 1, wherein the second format allows extra data tags, and wherein the rich ink data is concealed within the extra data tags.

11. The computer-readable medium of claim 1, having further computer-executable instructions comprising concealing data having a third format into the file.

12. The computer-readable medium of claim 1, having further computer-executable instructions comprising arranging the rich ink data in a specialized format.

13. The computer-readable medium of claim 12, wherein arranging the rich ink data includes associating a data structure with the rich ink data.

14. The computer-readable medium of claim 13, wherein the data structure includes a header.

15. The computer-readable medium of claim 14, wherein the data structure includes version information.

16. The computer-readable medium of claim 15, wherein the version information is located within the header.

17. The computer-readable medium of claim 15, wherein the version information is located within the rich ink data.

18. The computer-readable medium of claim 1, wherein the file having the second format is created by forming an image file based on the rich ink data.

19. The computer-readable medium of claim 1, having further computer-executable instructions comprising concealing data having a third format into the file, and further comprising, associating a data structure with the rich ink data and with the data having the third format.

20. The computer-readable medium of claim 19, wherein the data structure comprises a directory with format information for both the first and third formats.

59. The computer-readable medium of claim 1 having further computer-executable instructions comprising, providing the file to another computing device, and extracting the rich ink data from the file at that other computing device.

60. The computer-readable medium of claim 1 having further computer-executable instructions comprising, providing the file to another computing device, determining at the other computing device whether a component is available to extract the rich ink data from the file, and if so, providing the file to that component, and if not, interpreting the data in the file with a program that understands the second file format.

61. The computer-readable medium of claim 60 wherein a component is available to the other computing device that can extract the rich ink data from the file, and having further computer-executable instructions comprising, extracting the rich ink data via the component, and passing the rich ink data to another component that can interpret the rich ink data.

21. A computer system comprising:

a renderer that creates an image file from rich ink data, the rich ink data including movement information; and

a combiner that conceals within the image file the rich ink data in a format that is different from the format of the image file.

62. The system of claim 21 further comprising, a second computer system that receives the file, the second computer system including an extractor that extracts the rich ink data from the file.

63. The system of claim 62 wherein the second computer system includes a component that interprets the rich ink data extracted from the file.

22. A computer-readable medium having computer-executable instructions comprising:

providing a file comprising rich ink data in a first format, the file having a second format wherein the first format is concealed within the second format, the rich ink data including movement information; and

extracting the rich ink data from the file.

23. The computer-readable medium of claim 22, having further computer-executable instructions for accessing the rich ink data with a rich ink-compatible reader.

24. The computer-readable medium of claim 23, having further computer-executable instructions for editing the rich ink data.

25. The computer-readable medium of claim 24, having further computer-executable instructions for saving the edited rich ink data.

26. The computer-readable medium of claim 22, wherein the file having the first format comprises an image file format.

27. The computer-readable medium of claim 22, wherein the rich ink data is associated with a data structure.

28. The computer-readable medium of claim 27, having further computer-executable instructions comprising accessing the rich ink data with a rich ink-compatible reader and reading the rich ink data according to the data structure.

29. The computer-readable medium of claim 27, wherein the data structure includes version information.

30. The computer-readable medium of claim 27, wherein the data structure includes a data type identifier.

31. The computer-readable medium of claim 27, wherein the rich ink data includes a header of the data structure.

32. The computer-readable medium of claim 31, wherein the header includes a checksum.

33. The computer-readable medium of claim 22, wherein the second format includes at least one comments section, and wherein the rich ink data is concealed in the at least one comments section.

34. The computer-readable medium of claim 22, wherein the second format allows extra data tags, and wherein the rich ink data is concealed within the extra data tags.

35. A computer system comprising:

an extractor that separates concealed rich ink data from a file having a first format, the rich ink data in a second format, the rich ink data including movement information; and

a rich ink compatible reader that interprets the rich ink data separated from the file by the extractor.

36. A computer-readable medium having computer-executable instructions for performing steps comprising,

providing a file comprising rich ink data in first and second formats, the file having a third format, wherein the first and second formats are concealed within the third format, the rich ink data including movement information;

and

extracting rich ink data corresponding to at least one of the first and second formats from the file.

37. The computer-readable medium of claim 36, having further computer-executable instructions comprising, rendering an image from the rich ink data corresponding to at least one of the formats.

38. The computer-readable medium of claim 37, having further computer-executable instructions comprising, enumerating through the first and second formats to find a compatible format for rendering, and selecting the compatible format for rendering the image.

39. The computer-readable medium of claim 38, having further computer-executable instructions comprising, receiving information indicative of a failure in rendering the image from the selected format, enumerating back through the first and second formats to find another compatible format for rendering, and selecting the other compatible format for rendering.

40. The computer-readable medium of claim 36, wherein the first and second formats are associated with data structures, and wherein the concealed rich ink data comprises a data structure directory.

41. A method of communicating between a sender and a remote recipient device, comprising:

organizing rich ink data into a first format at the sender, the rich ink data including movement information;

concealing at the sender the rich ink data in the first format within a file having a second format;

communicating the file to the remote recipient device; and

at the remote recipient device;

receiving the file, extracting the rich ink data therefrom, and recognizing the first format.

64. A computer-readable medium having computer-executable instructions for performing the method of claim 41.

42. A computer-readable medium having stored thereon a data structure, comprising,

rich ink data having a first format, the rich ink data including movement information;

other data in a second format, the rich ink data concealed among the other data such that a component accessing the other data need not be programmed to understand the first format in order to interpret the other data; and

extracting the rich ink data from the other data.

43. The computer-readable medium of claim 42, wherein the second format comprises an image file format.

44. The computer-readable medium of claim 42, wherein the data structure includes a logical end of file, and wherein the rich ink data is appended after the logical end of the file.

45. The computer-readable medium of claim 42, wherein the second format comprises color palette indexes, and wherein the rich ink data is concealed in at least one least significant bit of selected color palette indexes.

46. The computer-readable medium of claim 45, wherein the color palette indexes are eight bit values, and wherein the concealed rich ink data comprises the eighth bit of the selected color palette indexes.

47. The computer-readable medium of claim 42, wherein the second format comprises color data representing components in a color scheme, and wherein the rich ink data is concealed in the least significant bits of at least some of the color data.

48. The computer-readable medium of claim 47, wherein the color scheme comprises 24-bit color.

49. The computer-readable medium of claim 47, wherein the color scheme comprises a three-color model, and wherein the rich ink data is stored in the least significant bit of each color for selected data of the color scheme.

50. The computer-readable medium of claim 42, wherein the second format includes at least one comments section, and wherein the rich ink data is concealed in the at least one comments section.

51. The computer-readable medium of claim 42, wherein the second format allows extra data tags, and wherein the rich ink data is concealed within the extra data tags.

52. The computer-readable medium of claim 42, wherein data having a third format is concealed in the file.

53. The computer-readable medium of claim 42, wherein the rich ink data is arranged in a specialized format.

54. The computer-readable medium of claim 42, wherein when not concealed, the rich ink data is arranged into a second data structure.

55. The computer-readable medium of claim 54, wherein the second data structure includes a header.

56. The computer-readable medium of claim 55, wherein the second data structure includes a directory.

57. The computer-readable medium of claim 55, wherein the directory includes control data.

58. The computer-readable medium of claim 57, wherein data having a third format is concealed in the data structure, and wherein the directory includes an entry for the data in the first and third formats.

65. A computer-readable medium having computer-executable instructions for performing steps comprising:

providing rich ink data having a first format, the rich ink data including movement information;

preserving the rich ink data including concealing the rich ink data into a file having a second format, wherein the file includes a logical end of file, and wherein concealing the rich ink data into the file includes appending the rich ink data after the logical end of the file; and

extracting the rich ink data from the file.

66. A computer-readable medium having computer-executable instructions for performing steps comprising:

providing rich ink data having a first format, the rich ink data including movement information; and

preserving the rich ink data including concealing the rich ink data into a file having a second format, wherein the file includes a logical end of file, wherein the second format includes color palette indexes to a color palette, and wherein rich ink data is concealed in at least one least significant bit of at least some of the indexes to the color palette for extraction.

67. The computer-readable medium of claim 66, wherein each color palette index is eight bits in length, and wherein concealed rich ink data is stored in the least significant bit of selected color palette indexes.

68. A computer-readable medium having computer-executable instructions for performing steps comprising:

providing rich ink data having a first format, the rich ink data including movement information;

preserving the rich ink data including concealing the rich ink data into a file having a second format, wherein the second format comprises color data representing color values in a component color scheme, and wherein the rich ink data is concealed in at least one least significant bit of at least some of the color values; and

extracting the rich ink data.

69. The computer-readable medium of claim 68, wherein the component color scheme comprises 24 bit color.

70. The computer-readable medium of claim 68, wherein the component color scheme comprises three color component values for each pixel represented, and wherein the rich ink data is stored in the least significant bit of at least some of the color component values.

71. A computer-readable medium having computer-executable instructions comprising:

providing a file comprising concealed rich ink data in a first format, the file having a second format, the rich ink data including movement information; and

extracting the rich ink data from the file, wherein the file includes a logical end of file, the rich ink data is appended after the logical end of the file, and wherein extracting the rich ink data comprises reading the rich ink data from the end of the file.

72. A computer-readable medium having computer-executable instructions comprising:

providing a file comprising concealed rich ink data in a first format, the file having a second format, the rich ink data including movement information; and

extracting the rich ink data from the file, wherein the second format includes color palette indexes to a color palette, and wherein the rich ink data is concealed in selected bits of selected color palette indexes.

73. The computer-readable medium of claim 72, wherein the color palette indexes are eight-bit values, and wherein the rich ink data is concealed in at least one least significant bit of the selected color palette indexes.

74. A computer-readable medium having computer-executable instructions comprising:

providing a file comprising concealed rich ink data in a first format, the file having a second format, the rich ink data including movement information; and

extracting the rich ink data from the file, wherein the second format comprises color data representing components in a color scheme, and wherein the rich ink data is concealed in the least significant bits of selected color data.

75. The computer-readable medium of claim 74, wherein the color scheme comprises 24-bit color.

76. The computer-readable medium of claim 75, wherein the color scheme comprises pixel data having each pixel represented by three color components, and wherein the rich ink data is stored in least significant bits of selected color components.