EXTRACTION OF ATTRIBUTES AND VALUES FROM NATURAL LANGUAGE DOCUMENTS

• US 20070282892A1
• Filed: 04/30/2007
• Published: 12/06/2007
• Est. Priority Date: 06/05/2006
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture (Accenture PLC)
• Inventors: Probst, Katharina, Ghani, Rayid, Fano, Andrew E., Krema, Marko, Liu, Yan
• Granted Patent: US 7,996,440 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/258 Heading extraction; Automatic titling; Numbering

Abstract

One or more classification algorithms are applied to at least one natural language document in order to extract both attributes and values of a given product. Supervised classification algorithms, semi-supervised classification algorithms, unsupervised classification algorithms or combinations of such classification algorithms may be employed for this purpose. The at least one natural language document may be obtained via a public communication network. Two or more attributes (or two or more values) thus identified may be merged to form one or more attribute phrases or value phrases. Once attributes and values have been extracted in this manner, association or linking operations may be performed to establish attribute-value pairs that are descriptive of the product. In a presently preferred embodiment, an (unsupervised) algorithm is used to generate seed attributes and values which can then support a supervised or semi-supervised classification algorithm.

Claims

1. A method for identifying at least one attribute and at least one value for a product based on at least one natural language document, the method comprising:

identifying the at least one attribute and the at least one value for the product via a classification algorithm operating upon the at least one natural language document; and

storing the at least one attribute and the at least one value.

2. The method of claim 1, wherein the classification algorithm is a supervised classification algorithm.

3. The method of claim 1, wherein the classification algorithm is a semi-supervised classification algorithm.

4. The method of claim 1, wherein the classification algorithm is an unsupervised classification algorithm.

5. The method of claim 1, further comprising:

obtaining the at least one natural language document via a public communication network.

6. The method of claim 1, further comprising:

associating an attribute of the at least one attribute with a value of the at least one value to provide an attribute-value pair; and

storing the attribute-value pair.

7. The method of claim 6, wherein the at least one attribute comprises at least two attributes and the at least one value comprises at least two values, and wherein associating the attribute with the value to provide the attribute-value pair further comprises:

calculating correlation values between each of the at least two attributes;

calculating correlation values between each of the at least two values;

merging attributes of the at least two attributes having correlation values above a correlation threshold; and

merging values of the at least two values having correlation values above the correlation threshold.

8. The method of claim 6, further comprising associating the attribute with the value based on selection criteria to provide the attribute-value pair.

9. The method of claim 1, further comprising:

generating seed attributes and corresponding seed values,

wherein identifying the at least one attribute and the least one value further comprises identifying the at least one attribute and the at least one value via the classification algorithm operating upon the at least one nature language document based on the seed attributes and the corresponding seed values.

10. The method claim 9, wherein generating the seed attributes and the seed values further comprises:

identifying the seed attributes and the corresponding seed values via an unsupervised classification algorithm as applied to the at least one natural language document.

11. An apparatus for identifying at least one attribute and at least one value for a product based on at least one natural language document, comprising:

a classification algorithm module operative to identify the at least one attribute and the at least one value for the product based on the at least one natural language document; and

a machine readable store operative to store the at least one attribute and the at least one value.

12. The apparatus of claim 11, wherein the classification algorithm module comprises a supervised classification algorithm.

13. The apparatus of claim 11, wherein the classification algorithm module comprises a semi-supervised classification algorithm.

14. The apparatus of claim 11, wherein the classification algorithm module comprises an unsupervised classification algorithm.

15. The apparatus of claim 11, further comprising:

a network interface operative to obtain the at least one natural language document via a public communication network.

16. The apparatus of claim 11, further comprising:

a linking module operative to associate an attribute of the at least one attribute with a value of the at least one value to provide an attribute-value pair,

wherein the machine readable store is further operative to store the attribute-value pair.

17. The apparatus of claim 16, wherein the at least one attribute comprises at least two attributes and the at least one value comprises at least two values, and wherein the linking module is further operative to:

calculate correlation values between each of the at least two attributes;

calculate correlation values between each of the at least two values;

merge attributes of the at least two attributes having correlation values above a correlation threshold; and

merge values of the at least two values having correlation values above the correlation threshold.

18. The apparatus of claim 16, wherein the linking module is further operative to associate the attribute with the value based on selection criteria to provide the attribute-value pair.

19. The apparatus of claim 11, further comprising:

a seed generation module operative to provide seed attributes and corresponding seed values,

wherein the classification algorithm module is further operative to identify the at least one attribute and the at least one value via the classification algorithm operating upon the at least one nature language document based on the seed attributes and the corresponding seed values.

20. The apparatus claim 19, wherein the seed generation module is further operative to identify the seed attributes and the corresponding seed values via an unsupervised classification algorithm as applied to the at least one natural language document.

21. A computer-readable medium having stored thereon executable instructions that, when executed, cause a computer to:

identify at least one attribute and at least one value for a product via a classification algorithm operating upon at least one natural language document; and

store the at least one attribute and the at least one value.

22. The computer-readable medium of claim 21, wherein the executable instructions that, when executed, cause the computer to identify the at least one attribute and the at least one value via the classification algorithm further cause the computer to identify the at least one attribute and the at least one value via a supervised classification algorithm.

23. The computer-readable medium of claim 21, wherein the executable instructions that, when executed, cause the computer to identify the at least one attribute and the at least one value via the classification algorithm further cause the computer to identify the at least one attribute and the at least one value via a semi-supervised classification algorithm.

24. The computer-readable medium of claim 21, wherein the executable instructions that, when executed, cause the computer to identify the at least one attribute and the at least one value via the classification algorithm further cause the computer to identify the at least one attribute and the at least one value via an unsupervised classification algorithm.

25. The computer-readable medium of claim 21, further comprising executable instructions that, when executed, cause the computer to:

obtain the at least one natural language document via a public communication network.

26. The computer-readable medium of claim 21, further comprising executable instructions that, when executed, cause the computer to:

associate an attribute of the at least one attribute with a value of the at least one value to provide an attribute-value pair; and

store the attribute-value pair.

27. The computer-readable medium claim 26, wherein the at least one attribute comprises at least two attributes and the at least one value comprises at least two values, and wherein the executable instructions that, when executed, cause the computer to associate the attribute with the value further cause the computer to:

calculate correlation values between each of the at least two attributes;

calculate correlation values between each of the at least two values;

merge attributes of the at least two attributes having correlation values above a correlation threshold; and

merge values of the at least two values having correlation values above the correlation threshold.

28. The computer-readable medium claim 26, wherein the executable instructions that, when executed, cause the computer to associate the attribute with the value further cause the computer to associate the attribute with the value based on selection criteria to provide the attribute-value pair.

29. The computer-readable medium of claim 21, further comprising executable instructions that, when executed, cause the computer to:

generate seed attributes and corresponding seed values,

wherein the executable instructions that, when executed, cause the computer to identify the at least one attribute and the least one value further cause the computer to identify the at least one attribute and the at least one value via the classification algorithm operating upon the at least one nature language document based on the seed attributes and the corresponding seed values.

30. The computer-readable medium of claim 29, wherein the executable instructions that, when executed, cause the computer to generate the seed attributes and the seed values further cause the computer to identify the seed attributes and the corresponding seed values via an unsupervised classification algorithm as applied to the at least one natural language document.

31. A method for identifying at least one attribute and at least one value of a product based on at least one natural language document, the method comprising:

identifying a first set of attributes and a first set of values of the product via a supervised algorithm as applied to the at least one natural language document;

identifying a second set of attributes and a second set of values of the product via a semi-supervised algorithm as applied to the at least one natural language document based at least in part upon the first set of attributes and the first set of values;

providing the first set of attributes and the second set of attributes as the at least one attribute;

providing the first set of values and the second set of values as the at least one value; and

storing the at least one attribute and the at least one value.

32. The method of claim 31, further comprising:

providing seed attributes and corresponding seed values identified by an unsupervised classification algorithm as applied to the at least one natural language document,

wherein the supervised algorithm identifies the first set of attributes and the first set of values based on the seed attributes and the corresponding seed values.

33. The method of claim 32, further comprising:

obtaining the at least one natural language document via a public communication network.

34. An apparatus for identifying at least one attribute and at least one value of a product based on at least one natural language document, comprising:

a supervised classification algorithm module operative to identify a first set of attributes and a first set of values of the product based on the at least one natural language document;

a semi-supervised classification algorithm module operative to identify a second set of attributes and a second set of values of the product based at least in part upon the at least one natural language document and the first set of attributes and the first set of values; and

a machine readable store operative to store the first set of attributes and the second set of attributes as the at least one attribute, and to store the first set of values and the second set of values as the at least one value.

35. The apparatus of claim 34, further comprising:

an unsupervised classification algorithm module operative to identify seed attributes and corresponding seed values based on the at least one natural language document,

wherein the supervised classification algorithm module is further operative to identify the first set of attributes and the first set of values based on the seed attributes and the corresponding seed values.

36. The apparatus of claim 34, further comprising:

a network interlace operative to obtain the at least one natural language document via a public communication network.

37. A computer-readable medium having stored thereon executable instructions that, when executed, cause the computer to:

identify a first set of attributes and a first set of values of a product via a supervised classification algorithm as applied to at least one natural language document;

identify a second set of attributes and a second set of values of the product via a semi-supervised classification algorithm as applied to the at least one natural language document based at least in part upon the first set of attributes and the first set of values;

provide the first set of attributes and the second set of attributes as at least one attribute;

provide the first set of values and the second set of values as at least one value; and

store the at least one attribute and the at least one value.

38. The computer-readable medium of claim 37, further comprising executable instructions that, when executed, cause the computer to:

provide seed attributes and corresponding seed values identified by an unsupervised classification algorithm as applied to the at least one natural language document,

wherein the executable instructions that, when executed, cause the computer to identify the first set of attributes and the first set of values further cause the computer to identify the first set of attributes and the first set of values via the supervised classification algorithm based on the seed attributes and the corresponding seed values.

39. The computer-readable medium of claim 37, further comprising executable instructions that, when executed, cause the computer to:

obtain the at least one natural language document via a public communication network.