Leveraging markup language data for semantically labeling text strings and data and for providing actions based on semantically labeled text strings and data

• US 20040268237A1
• Filed: 06/27/2003
• Published: 12/30/2004
• Est. Priority Date: 06/27/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Ziyi, Jones, Brian, Reynar, Jeff
• Granted Patent: US 7,739,588 B2
• Time in Patent Office: Days
• US Class Current: 715/205
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

Methods and systems are provided for leveraging markup language data applied to text or data for providing helpful actions on certain types of text or data such as names, addresses, dates, stock symbols, book titles, etc. As text or data is entered into a document, selected portions of the text or data, such as sentences, paragraphs or spreadsheet cell ranges, along with any associated markup language data (e.g., XML) are passed to an action dynamically linked library for obtaining one or more actions associated with markup language elements applied to the text or data. The text or data may be passed to a recognizer dynamically linked library (DLL) for recognition of certain data types. The recognizer DLL utilizes markup language data associated with the text or data to assist recognition and labeling of text or data. The recognition process may continue iteratively for a given text or data selection until no new text or data is recognized and labeled. After all applicable text and/or data is recognized and labeled, an action DLL is called for actions associated with the labeled text or data. Markup language data associated with recognized and labeled text or data is passed to the action DLL and individual action plug-ins to locate additional actions associated with markup language data that has been designated as equivalent to the markup language data associated with the labeled text or data.

Claims

1. In an electronic system for creating, editing and/or viewing an electronic document, a method for providing actions on a string of text or data in the electronic document, the method comprising the steps of:

receiving a text string annotated with markup language data in an action dynamic link library (DLL);

transmitting the text string and the associated markup language data to a plurality of action plug-ins;

determining, in the action plug-ins, one or more actions based on the associated markup language data;

passing the one or more actions to an application program module for displaying the one or more actions in association with the text string; and

displaying the one or more actions in association with the text string.

2. The method of claim 1, whereby determining, in the action plug-ins, one or more actions based on the associated markup language data comprises:

for each markup language element of the associated markup language data, parsing a namespace library for equivalent markup language elements;

obtaining one or more actions associated with the equivalent markup language elements for displaying with the plurality of actions received from the plurality of action plug-ins.

3. The method of claim 1, whereby for any portion of the text string not annotated with markup language data, further comprising:

receiving the text string in a recognizer dynamically linked library (DLL);

receiving markup language data associated with the text string in the recognizer dynamically linked library;

parsing the associated markup language data to assist the recognizer DLL to determine one or more labels for the text string;

transmitting the one or more labels and the associated markup language data to the application program module for passing to the action (DLL).

4. The method of claim 3, further comprising:

prior to transmitting the one or more labels and the associated markup language data to the application program module for passing to the action (DLL), transmitting the text string, the associated markup language data and the one or more labels back to the recognizer DLL; and

parsing the string of text, the associated markup language data and the one or more labels to determine a one or more labels for the text string not previously determined for the text string.

5. The method of claim 3, wherein the step of parsing the text string to determine one or more labels comprises the steps of:

comparing the text string with a plurality of stored text string with an associated stored label to determine a match;

if a the text string matches a stored text string with an associated label, then labeling the text string with the associated stored label of the matched stored text string;

comparing the elements of the markup language data associated with the text string with a plurality of stored markup language elements associated with associated stored labels to determine a match; and

if a one or more markup language elements associated with the text string matches one or more stored markup language elements with associated stored labels, then labeling the text string with the associated stored label of the matched one or more markup language elements.

6. A computer-readable medium having computer-executable instructions for performing the steps recited in claim 1.

7. The method of claim 3, further comprising the step of modifying the content of an electronic document to reflect the one or more labels.

8. The method of claim 7, further comprising the steps of:

causing the application program module to fire an event within an object model of the application program module;

causing software instructions associated with the event to be executed when at least one of the plurality of labels is determined.

9. The method of claim 3, further comprising the steps of examining the content of the electronic document surrounding the text string to aid in the steps of parsing the text string to determine a plurality of labels.

10. The method of claim 1, whereby the markup language is the Extensible Markup Language (XML).

11. A method for labeling a string of text in an electronic document as the electronic document is created in an application program module, the method comprising the steps of:

as a string of text having an associated one or more Extensible Markup Language (XML) elements is entered into the electronic document, determining whether the string of text matches one of a plurality of stored strings;

if so, then designating a label associated with the matched stored string for application to the entered string of text;

if the string of text does not match one of a plurality of stored strings, determining whether the one or more XML elements associated with the string of text is associated with a label for use with the entered string of text; and

if so, then designating a label associated with the one or more XML elements for application to the entered string of text.

12. The method of claim 11, further comprising:

if a label associated with the matched stored string is designated for application to the entered string of text, determining a set of actions associated with the label associated with the matched stored string; and

if a label associated with the one or more XML elements is designated for application to the entered string of text, determining a set of actions associated with the label associated with the one or more XML elements.

13. The method of claim 12, whereby determining a set of actions associated with the label associated with the one or more XML elements, further comprises:

for each label associated with the one or more XML elements, parsing a namespace library for equivalent markup language elements;

obtaining zero or more actions associated with the equivalent XML elements for combining with the set of actions associated with the label associated with the matched stored string.

14. The method of claim 11, further comprising displaying an indication indicating that the label has been found for the string of text.

15. The method of claim 13, further comprising the steps of:

determining that a user has selected the string of text; and

in response, displaying the combined set of actions to the user.

16. The method of claim 15, further comprising the steps of:

receiving an indication that one of the plurality of actions has been selected; and

in response to receiving an indication that one of the plurality of actions has been selected, then causing the application program module to execute the selected action.

17. A computer-readable medium having computer-executable instructions for performing the steps received in claim 16.

18. The method recited in claim 16, wherein the application program module executes the selected action by determining whether an action plug-in dynamically linked library assigned to the action is available;

and

if so, then receiving instructions from the action dynamically linked library assigned to the selected action.

19. The method recited in claim 18, further comprising the steps of:

if an action plug-in dynamic link library is not available, then using a Uniform Resource Locator assigned to the action to navigate to a Web site and download the action plug-in dynamic link library.

20. A system for providing helpful actions on a string of text in an electronic document as the string is entered into the electronic document, the system comprising:

an application program module for creating the electronic document;

an action dynamically linked library connected to the application program module operative to provide one or more actions associated with one or more markup language elements applied to the string of text;

a namespace library associated with the application program module for providing one or more equivalent markup language elements that have been designated as equivalent to the one or more markup language elements applied to the string of text in the electronic document; and

whereby the action dynamically linked library is further operative to provide additional one or more actions associated with the one or more equivalent markup language elements.

21. The system of claim 20, further comprising at least one recognizer dynamically linked library for providing semantic labeling to one or more portions of the string of text based on the one or more markup language elements applied to the string of text.

22. The system of claim 21, wherein

the recognizer dynamically linked library is operative

to receive the string of text;

to receive the one or more markup language elements applied to the string of text in the recognizer dynamically linked library;

to transmit the string of text and associated markup language elements to a plurality of recognizer plug-ins;

the plurality of recognizer plug-ins being operative

to parse the string of text to determine a plurality of labels;

to parse the associated markup language elements to assist each of the plurality of recognizer plug-ins to determine a plurality of labels for the string of text;

to transmit the plurality of labels to the recognizer dynamically linked library; and

the recognizer dynamically linked library being further operative to transmit the plurality of labels and the associated markup language data to the application program module.

23. The system of claim 22,

whereby recognizer dynamically linked library is further operative prior to transmitting the plurality of labels from the recognizer plug-ins to the recognizer dynamically linked library, to transmit the string of text, the associated markup language elements and the plurality of labels back to the plurality of recognizer plug-ins;

and

the plurality of recognizer plug-ins being further operative to parse the string of text, the associated markup language elements and the plurality of labels to determine a plurality of labels for the string of text not previously determined for the string of text.