TEXT ANALYSIS USING PHRASE DEFINITIONS AND CONTAINERS

• US 20100250235A1
• Filed: 03/24/2009
• Published: 09/30/2010
• Est. Priority Date: 03/24/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Madan, Umesh
• Granted Patent: US 8,433,559 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

In one example, a phrase analyzer may analyze a text input stream to identify phrases contained in the text input stream. The phrase analyzer may receive a specification, which includes dictionaries of phrases and synonyms, and a specification of the phrases, or sequences of phrases to be matched. The phrase analyzer may compare the input stream to the specification and may produce, as output, an identification of which phrases appear in the input stream, and where in the input stream those phrases occur.

Claims

1. One or more computer-readable storage media that store executable instructions that, when executed by a computer, cause the computer to perform acts comprising:

receiving a specification of a text pattern to be matched in an input stream, said specification comprising one or more data expressions and one or more code expressions, the data expressions defining one or more phrase sets, a first one of the code expressions comprising a container that comprises a first phrase set and a second phrase set;

comparing words in said input stream with said container;

determining that a context in said input stream matches said container; and

generating a report indicating that said container matches a sequence of words in said input stream.

2. The one or more computer-readable storage media of claim 1, wherein said generating of said report comprises:

creating a file that identifies one or more ordinal positions in said input stream in which a match for said container is found, and a label that identifies said match.

3. The one or more computer-readable storage media of claim 1, wherein container comprises either:

an AND container that specifies that said context matches said container when said context comprises an instance of said first phrase set and an instance of said second phrase set;

or

an OR container that specifies that said context matches said container when said context comprises an instance of said first phrase set or an instance of said second phrase set.

4. The one or more computer-readable storage media of claim 1, wherein said container comprises an ALL container that specifies that said context matches said container when (a) said context comprises an instance of said first phrase set and an instance of said second phrase set, and (b) there are no more words to be consumed in said context after words corresponding to all elements in said container have been consumed.

5. The one or more computer-readable storage media of claim 1, wherein said container comprises a REPEAT container, wherein said context starts at a first ordinal position in said input stream, and wherein said acts further comprise:

determining whether said input stream contains instances of elements in said container starting at said first ordinal position; and

determining whether said input stream contains instances of elements in said container starting at successive ordinal positions following said first ordinal position.

6. The one or more computer-readable storage media of claim 1, wherein said container further comprises a second container, and wherein said determining of whether said context matches said container comprises determining whether said context comprises an instance of said first phrase set, an instance of said second phrase set, and an instance of said second container.

7. The one or more computer-readable storage media of claim 1, wherein said container comprises a LIST container, and wherein said determining comprises:

determining that said context comprises an instance of said first phrase set; and

after determining that said context comprises an instance of said first phrase set, determining whether said context comprises instances of all other elements in said container, said context matching said container as long as said context comprises an instance of at least one of the elements in said container, wherein the elements in the container include said first phrase set and said other elements.

8. A system comprising:

one or more processors;

one or more data remembrance components;

a specification that is stored in said one or more data remembrance components, said specification comprising;

a data expression that defines a plurality of phrase sets;

a code expression that defines a container that specifies a combination that comprises a first one of said phrase sets and a second one of said phrase sets;

a phrase analyzer that executes on said one or more processors, that receives said specification and an input stream, that compares words in said input stream to said container, and that generates a report that identifies where, in said input stream, a match between said container and said input stream is found.

9. The system of claim 8, wherein said specification further comprises:

a map set that maps a plurality of phrases to a first phrase, wherein said phrase analyzer encounters one of said plurality of phrases in said input stream and, based on said maps set, converts said one of said plurality of phrases to said first phrase when determining whether said input stream matches said container.

10. The system of claim 8, wherein said specification further comprises:

an equivalence set that comprises a plurality of phrases, said phrase analyzer finding that all of said plurality of phrases in said equivalence set are found when any one of said phrases is found in said input stream.

11. The system of claim 8, wherein container comprises either:

an AND container that specifies that said input stream matches said container when a context in said input stream comprises an instance of said first one of said phrase sets and an instance of said second one of said phrase sets;

or

an OR container that specifies that said input stream matches said container when a context in said input stream comprises an instance of said first one of said phrase sets or an instance of said second one of said phrase sets.

12. The system of claim 8, wherein said container comprises an ALL container that specifies that said input stream matches said container when (a) a context in said input stream comprises an instance of said first one of said phrase sets and an instance of said second one of said phrase sets, and (b) there are no more words to be consumed in said context after words corresponding to all elements in said ALL container have been consumed.

13. The system of claim 8, wherein said container comprises a REPEAT container, and wherein said phrase analyzer determines that a sequence of a plurality of words starting at a first ordinal position in said input stream matches said container, and subsequently compares said container to a sequence of words that starts one ordinal position ahead of said first ordinal position, and after a word that appears one ordinal position ahead of said first ordinal position has already been consumed.

14. The system of claim 8, wherein said container further comprises a second container, and wherein said phrase analyzer determines whether said input stream matches said container by determining whether a context in said input stream comprises an instance of said first one of said phrase sets, an instance of said second one of said phrase sets, and an instance of said second container.

15. The system of claim 8, wherein said container comprises a LIST container, wherein said phrase determines that input stream comprises a context that comprises an instance of said first one of said phrase sets and subsequently determines whether said context comprises instances of all other elements in said container, said context matching said container as long as said context comprises an instance of at least one of the elements in said container, wherein the elements in the container include said first phrase set and said other elements.

16. A method of performing an action based on input text, the method comprising:

using a processor to perform acts comprising;

comparing words in an input text to a container that describes a combination that comprises a first phrase set and a second phrase set, said first phrase set comprising a first plurality of phrases, said second phrase set comprising a second plurality of phrases;

determining that a sequence of words in said input text matches said container;

generating a report that indicates where, in said input text, said sequence of words occurs; and

taking a tangible action based on said report.

17. The method of claim 16, further comprising:

using a processor to perform acts comprising;

creating a text file that identifies one or more ordinal positions in said input text in which a match for said container is found, and a label that identifies said match.

18. The method of claim 16, wherein said container comprises either:

an AND container that specifies that a context in said input text matches said container when said context comprises an instance of said first phrase set and an instance of said second phrase set;

or

an OR container that specifies that a context in said input set matches said container when said context comprises an instance of said first phrase set or an instance of said second phrase set.

19. The method of claim 16, further comprising:

using a processor to perform acts comprising;

receiving a specification that comprises;

data expressions that comprise said first phrase set and said second phrase set; and

code expressions that comprise said container.

20. The method of claim 19, wherein said specification further comprises:

a map set that maps a first plurality of phrases to a first phrase; and

an equivalence set that comprises a second plurality of phrases; and

wherein the method further comprises;

using a processor to perform acts comprising;

converting occurrence of any one of said first plurality of phrases in said input text to said first phrase; and

indicating that all of said second plurality of phrases have occurred in said input text when any one of said second plurality of phrases occurs in said input text.