Modifying electronic documents with recognized content or other associated data

• US 8,074,184 B2
• Filed: 11/07/2003
• Issued: 12/06/2011
• Est. Priority Date: 11/07/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Garside, Adrian James, Winkler, David Vaughn, Clow, Joshua
• Primary Examiner(s): Pesin, Boris
• Assistant Examiner(s): Heffington, John
• Application Number:
  US10/703,081
• Publication Number:
  US 20050099398A1
• Time in Patent Office: 2,951 Days
• Field of Search: 345/179, 715/810, 715/812, 715/813, 715/816, 715/817, 715/818, 715/819, 715/820, 715/825, 715/845, 715/854
• US Class Current: 715/854
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/171 by use of digital ink

Abstract

Systems and methods enhance editing capabilities associated with a wide variety of different types of electronic documents. Such systems and methods may include a processor that maintains an electronic document having a first portion (e.g., an individual word, character, character string, or the like) provided by a recognizer (e.g., by handwriting or speech recognition software), and they may provide access to potential alternative characters, words, or character strings generated by the recognizer during various user editing functions and operations. Other data associated with the first portion of the document also may be stored and made available to the user during various functions and operations. This invention further relates to computer-readable media including instructions for performing various methods and/or operating various systems for editing electronic documents, including systems and methods like those described above.

Claims

1. A computer-implemented method of modifying an electronic document using a computing system having processor, memory, and data storage subsystems, the computer-implemented method, comprising:

maintaining an electronic document, wherein at least a first portion of content in the electronic document includes content generated by a user via an input device and the content converted by a recognizer to recognized content as standard text;

obtaining data associated with the recognized content, wherein the data is stored in a data structure directly linked to the recognized content that includes information not included in the electronic document, the data structure including a plurality of different linked nodes, where each linked node stores additional information related to the data stored in the linked node including a location within the electronic document of the recognized content and a global unique identifier that identifies a source of the recognized content;

receiving from the user, an input selecting a segment of the first portion of the content or the recognized content;

providing to the user, at least one selectable alternative for the first portion of the content or recognized content selected by the user based at least in part on the data associated with the first portion where the user can modify the recognized content with the at least one selectable alternative;

wherein;

the data associated with the first portion of the content includes an expanded version and a supporting data structure of the electronic document separate from the electronic document, and a text injector strips alternative data from the expanded version of the electronic document and sends a stripped down text version of the data to the electronic document;

the expanded version and the supporting data structure are exclusively linked to the data structure of the electronic document, the expanded version and the supporting data structure being saved separate from the electronic document and made directly available to the user upon subsequent access of the electronic document; and

the data associated with the first portion of the content includes properties associated with the first portion of the content that are not included in the electronic document; and

maintaining synchronization between a received change in the content of the electronic document and an associated change in the content of the expanded version and the supporting data structure.

2. The computer-implemented method according to claim 1, further comprising:

changing the content of the electronic document when one of the selectable alternatives is selected.

3. The computer-implemented method according to claim 1, wherein the first portion of the content is provided by a handwriting recognizer.

4. The computer-implemented method according to claim 3, wherein the data associated with the first portion of the content includes one or more potential alternative characters, words, or character strings identified by the handwriting recognizer.

5. The computer-implemented method according to claim 1, wherein the first portion of the content includes a word, character, or character string generated by a handwriting recognizer, and the data associated with the first portion of the content includes one or more potential alternative words, characters, or character strings generated by the handwriting recognizer.

6. The computer-implemented method according to claim 1, further comprising:

receiving input changing the stored data associated with the first portion of the content of the electronic document; and

changing the electronic document, if necessary, based on the input changing the stored data associated with the first portion of the content of the electronic document.

7. A computer-storage medium including computer-executable instructions stored thereon for performing the computer-implemented method of claim 1.

31. The computer-implemented method of claim 1, wherein the input device comprises one of an electronic pen device and a touch-sensitive input device.

8. A computer-implemented method of editing an electronic document using a computing system having processor, memory, and data storage subsystems, the computer-implemented method, comprising:

receiving input in an electronic document via an input device, wherein the electronic document includes a data structure having a plurality of independent data sets, the input being converted by a recognizer to standard text data;

maintaining a supporting data structure that minors content of a user-visible document which is based on content in at least a portion of the electronic document and the converted standard text data, wherein the supporting data structure includes a plurality of supporting data sets such that at least some of the independent data sets in the electronic document and the converted standard text data are directly linked to an associated supporting data set in the supporting data structure, wherein;

the data structure includes a plurality of different linked nodes, where each linked node stores additional information related to data stored in the linked node including a location within the electronic document of recognized content and a global unique identifier that identifies a source of the recognized content, and

the supporting data structure is directly linked to the data structure of the electronic document, and the supporting data structure is saved and made available to a user upon subsequent access of the electronic document;

receiving input requesting display of or access to information contained in at least one supporting data set;

receiving input selecting information from the at least one supporting data set;

changing the electronic document based on the selected information from the at least one supporting data set;

wherein;

the supporting data structure includes an expanded version of the electronic document containing data that is not accepted or supported by the electronic document, the supporting data structure being maintained separate from and exclusively linked to the electronic document, and

at least some of the supporting data sets include one or more properties associated with its corresponding independent data set in the electronic document, and wherein the properties are not stored in the data structure of the electronic document; and

maintaining synchronization between the content of the electronic document and the content of the expanded version, wherein a change to the content of the expanded version changes the electronic document accordingly.

9. The computer-implemented method according to claim 8, wherein at least some of the independent data sets correspond to words, characters, or character strings in the electronic document.

10. The computer-implemented method according to claim 9, wherein at least some of the supporting data sets store potential alternative words, characters, or character strings corresponding to the words, characters, or character strings stored in the associated independent data sets.

11. The computer-implemented method according to claim 10, wherein the potential alternative words or characters stored in the supporting data sets are generated by a recognizer.

12. The computer-implemented method according to claim 8, wherein at least some of the supporting data sets include one or more properties associated with its corresponding independent data set in the electronic document, wherein the properties are not stored in the data structure of the electronic document.

13. The computer-implemented method according to claim 12, wherein the one or more properties includes at least one member selected from the group of:

handwriting recognizer generated alternative words, characters, or character strings;

speech recognizer generated alternative words, characters, or character strings;

language of a word, character, or character string;

identifier of a word, character, or character string source;

original electronic ink data associated with a word, character, or character string;

time of input of a word, character, or character string; and

spacing data associated with a word, character, or character string.

14. The computer-implemented method according to claim 8, further comprising:

receiving input changing the supporting data structure; and

changing the electronic document based on the input changing the supporting data structure.

15. A computer-storage medium including computer-executable instructions stored thereon for performing the computer-implemented method of claim 8.

32. The computer-implemented method of claim 8, wherein the input device comprises one of an electronic pen device and a touch-sensitive input device.

16. A computer-implemented electronic document modification system having processor, memory, and data storage subsystems, the computer-implemented system, comprising:

a processor programmed and adapted to;

(a) maintain an electronic document, wherein at least a first portion of content in the electronic document includes content generated by a user via an input device and the content converted by a recognizer to recognized content as standard text, and (b) obtain data associated with the recognized content; and

a text injector system to strip additional data from incoming input that is not supported in a data structure of the electronic document and move the stripped additional data to a supporting data structure, wherein;

the data is stored in the data structure directly linked to the recognized content that includes information not included in the electronic document, the data structure including a plurality of different linked nodes, where each linked node stores additional information related to the data stored in the linked node including a location within the electronic document of the recognized content and a global unique identifier that identifies a source of the recognized content;

the input device receives by the user, a selection of a segment that optionally includes the first portion of the content or the recognized content,

the processor further is programmed and adapted to provide to the user, at least one selectable alternative for the first portion of the content or the recognized content selected by the user based at least in part on the data associated with the first portion where the user can modify the recognized content with the at least one selectable alternative;

the data associated with the first portion of the content includes an expanded version of the electronic document separate from the electronic document;

the expanded version is directly and exclusively linked to the electronic document, and the expanded version is saved and made available to the user upon subsequent access of the electronic document;

the data associated with the first portion of the content includes properties associated with the first portion of the content that are not included in the electronic document; and

the processor further is programmed and adapted to maintain synchronization between changes to either the content of the electronic document or the expanded version.

17. The computer-implemented system according to claim 16, wherein the processor further is programmed and adapted to change the content of the electronic document when one of the selectable alternatives is selected.

18. The computer-implemented system according to claim 16, wherein the first portion of the content of the electronic document is provided by a handwriting recognizer.

19. The computer-implemented system according to claim 18, wherein the data associated with the first portion of the content of the electronic document includes one or more potential alternative words, characters, or character strings identified by the handwriting recognizer.

20. The computer-implemented system according to claim 16, wherein the first portion of the content of the electronic document includes a word, character, or character string generated by a handwriting recognizer, and the data associated with the first portion of the content of the electronic document includes one or more potential alternative words, characters, or character strings generated by the handwriting recognizer.

21. The computer-implemented system according to claim 16, wherein the processor further is programmed and adapted to receive input including a change to the content of the electronic document, and to change the stored data associated with the first portion of the content of the electronic document, if necessary, based on the change to the content of the electronic document.

22. The computer-implemented system according to claim 16, wherein the processor further is programmed and adapted to receive input changing the stored data associated with the first portion of the content of the electronic document, and to change the electronic document, if necessary, based on the input changing the stored data associated with the first portion of the content of the electronic document.

33. The computer-implemented system of claim 16, wherein the input device comprises one of an electronic pen device and a touch-sensitive input device.

23. A computer-implemented modification system having processor, memory, and data storage subsystems, the computer-implemented system comprising:

a processor programmed and adapted to;

(a) receive input in an electronic document via an input device, wherein the electronic document includes a data structure having a plurality of independent data sets, the input being converted by a recognizer to standard text data, and

(b) maintain a supporting data structure based on recognized content in at least a portion of the electronic document and the converted standard text data, wherein the supporting data structure and the converted standard text data includes a plurality of supporting data sets such that at least some of the independent data sets and/or converted standard text data in the electronic document are directly linked to an associated supporting data set in the supporting data structure, wherein the data structure includes a plurality of different linked nodes, where each linked node stores additional information related to data stored in the linked node including a location within the electronic document of the recognized content and a global unique identifier that identifies a source of the recognized content;

an input device adapted to receive input requesting display of or access to information contained in at least one supporting data set;

input of selected information from the at least one supporting data set; and

a revised electronic document based on the selected information from the at least one supporting data set;

wherein;

the supporting data structure includes an expanded version of the electronic document maintained separate from the electronic document;

the supporting data structure is exclusively linked to the electronic document, and the expanded version is saved and made available to a user upon subsequent access of the electronic document;

at least some of the supporting data sets include one or more properties associated with its corresponding independent data set in the electronic document;

the properties are not stored in the data structure of the electronic document; and

the processor further is programmed and adapted to maintain synchronization between changes to either the content of the electronic document or the expanded version, wherein ambiguous input changes are synchronized over smaller portions to check for potential available alternatives.

24. The computer-implemented system according to claim 23, wherein the processor further is programmed and adapted to receive input selecting information from the at least one supporting data set, and change the electronic document based on the selected information from the at least one supporting data set.

25. The computer-implemented system according to claim 23, wherein at least some of the independent data sets correspond to words, characters, or character strings in the electronic document.

26. The computer-implemented system according to claim 25, wherein at least some of the supporting data sets store potential alternative words, characters, or character strings corresponding to the words, characters, or character strings stored in the associated independent data sets.

27. The computer-implemented system according to claim 26, wherein the potential alternative words, characters, or character strings stored in the supporting data sets are generated by a recognizer.

28. The computer-implemented system according to claim 23, wherein at least some of the supporting data sets include one or more properties associated with its corresponding independent data set in the electronic document, wherein the properties are not stored in the data structure of the electronic document.

29. The computer-implemented system according to claim 28, wherein the one or more properties includes at least one member selected from the group of:

handwriting recognizer generated alternative words, characters, or character strings;

language of a word, character or character string;

identifier of a word, character, or character string source;

original electronic ink data associated with a word, character, or character string;

time of input of a word, character, or character string; and

spacing data associated with a word, character, or character string.

30. The computer-implemented system according to claim 23, wherein the processor further is programmed and adapted to receive input changing the supporting data structure, and to change the electronic document based on the input changing the supporting data structure.

34. The computer-implemented system of claim 23, wherein the input device comprises one of an electronic pen device and a touch-sensitive input device.