Method and system for analyzing text

• US 8,788,261 B2
• Filed: 11/04/2009
• Issued: 07/22/2014
• Est. Priority Date: 11/04/2008
• Status: Active Grant

• Current Assignee: Strossle International AB
• Original Assignee: Saplo AB
• Inventors: Sikstrom, Sverker, Tyrberg, Mattias, Hall, Anders, Horte, Fredrik, Stenberg, Joakim
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US13/126,583
• Publication Number:
  US 20110208511A1
• Time in Patent Office: 1,721 Days
• Field of Search: 704 1- 10, 704/251, 704/255, 704/257, 704/270, 704/270.1, 707/739, 1/1
• US Class Current: 704/9
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202

Abstract

An apparatus for providing a control input signal for an industrial process or technical system having one or more controllable elements includes elements for generating a semantic space for a text corpus, and elements for generating a norm from one or more reference words or texts, the or each reference word or text being associated with a defined respective value on a scale, and the norm being calculated as a reference point or set of reference points in the semantic space for the or each reference word or text with its associated respective scale value. Elements for reading at least one target word included in the text corpus, elements for predicting a value of a variable associated with the target word based on the semantic space and the norm, and elements for providing the predicted value in a control input signal to the industrial process or technical system. A method for predicting a value of a variable associated with a target word is also disclosed together with an associated system and computer readable medium.

Claims

1. A method for predicting a value of a variable associated with a target word or set of words, performed on at least one computer and comprising the steps of:

collecting a text corpus comprising a set of words that include the target word,

generating a representation of the text corpus,

creating a semantic space for the set of words, based on the representation of the text corpus,

defining, for a location in the semantic space, a value of the variable,

estimating, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculating, by the computer, a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word,

wherein the estimating of the target word variable value comprises performing regression analysis having the target word variable value as a dependent variable.

2. A method according to claim 1, wherein the creating of the semantic space comprises performing a singular value decomposition on the representation of the text corpus.

3. A method according to claim 1, further comprising the step of statistically testing the predicted value of the variable, by comparing the predicted value with known values.

4. A method according to claim 1, further comprising the step of statistically testing if two sets of words or two sets of documents of the text corpora differ in semantic representation.

5. A method according to claim 1, wherein the collecting of the text corpus comprises collecting time information associated with text in the text corpus.

6. A method according to claim 5, wherein the predicting of the value of the variable comprises associating the predicted value with the time information of the text corpus.

7. A method according to claim 1, wherein the collecting of the text corpus comprises collecting a relevance indicating measure associated with text in the text corpus.

8. A method according to claim 7, wherein the predicting of the value of the variable comprises numerically weighting the value with the relevance indicating measure.

9. An apparatus (300) for providing a control input signal (412) for an industrial process or technical system (400) having one or more controllable elements (421-42n), the apparatus comprising:

means (320;

909) for generating a semantic space (892) for a text corpus (302;

908);

means (330;

914) for generating a norm (896) from one or more reference words or texts (332;

893), the or each reference word or text being associated with a defined respective value on a scale, and the norm being calculated as a reference point or set of reference points in the semantic space for the or each reference word or text with its associated respective scale value;

means (340) for reading at least one target word;

means (340) for predicting a value (350) of a variable associated with the target word based on the semantic space and the norm; and

means (340) for providing the predicted value in a control input signal (412) to said industrial process or technical system (400),

wherein the predicting of the target word variable value comprises performing regression analysis having the target word variable value as a dependent variable.

10. An apparatus as defined in claim 9, further comprising means (310) for collecting said digital text corpus from a plurality of distributed text sources (314-316) accessible via one or more networks (312).

11. An apparatus as defined in claim 9, further comprising a data processing unit, wherein said data processing unit is configured to perform a method for predicting a value of a variable associated with a target word or set of words, performed on at least one computer and comprising the steps of:

collecting a text corpus comprising a set of words that include the target word,

generating a representation of the text corpus,

creating a semantic space for the set of words, based on the representation of the text corpus,

defining, for a location in the semantic space, a value of the variable,

estimating, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculating a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word.

12. A system for predicting a value of a variable associated with a target word or set of words, comprising at least one computer and configured to:

collect a text corpus comprising a set of words that include the target word,

generate a representation of the text corpus,

create a semantic space for the set of words, based on the representation of the text corpus,

define, for a location in the semantic space, of a subset of the words, a value of the variable,

estimate, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculate a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word,

wherein the estimate of the target word variable value comprises performing regression analysis having the target word variable value as a dependent variable.

13. A system according to claim 12, wherein the creating of the semantic space comprises performing a singular value decomposition on the representation of the text corpus.

14. A system according to claim 12, further configured for statistically testing the predicted value of the variable, by comparing the predicted value with known values.

15. A system according to claim 12, further configured for statistically testing if two sets of words or two sets of documents of the text corpora differ in semantic representation.

16. A system according to claim 12, wherein the collecting of the text corpus comprises collecting time information associated with text in the text corpus.

17. A system according to claim 16, wherein the predicting of the value of the variable comprises associating the predicted value with the time information of the text corpus.

18. A system according to claim 12, wherein the collecting of the text corpus comprises collecting a relevance indicating measure associated with text in the text corpus.

19. A system according to claim 18, wherein the predicting of the value of the variable comprises numerically weighting the value with the relevance indicating measure.

20. A non-transitory computer readable medium having stored thereon a computer program having software instructions which when run on a computer cause the computer to perform the steps of:

collecting a text corpus comprising a set of words that include the target word,

generating a representation of the text corpus,

creating a semantic space for the set of words, based on the representation of the text corpus,

defining, for a location in the semantic space, a value of the variable,

estimating, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculating a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word,

wherein the estimating of the target word variable value comprises performing regression analysis having the target word variable value as a dependent variable.

21. A method for predicting a value of a variable associated with a target word or set of words, performed on at least one computer and comprising the steps of:

collecting a text corpus comprising a set of words that include the target word,

generating a representation of the text corpus,

creating a semantic space for the set of words, based on the representation of the text corpus,

defining, for a location in the semantic space, a value of the variable,

estimating, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculating, by the computer, a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word,

wherein the collecting of the text corpus comprises collecting time information associated with text in the text corpus, and

wherein the predicting of the value of the variable comprises associating the predicted value with the time information of the text corpus.

22. A method according to claim 21, wherein the creating of the semantic space comprises performing a singular value decomposition on the representation of the text corpus.

23. A method according to claim 21, further comprising the step of statistically testing the predicted value of the variable, by comparing the predicted value with known values.

24. A method according to claim 21, further comprising the step of statistically testing if two sets of words or two sets of documents of the text corpora differ in semantic representation.

25. A method according to claim 21, wherein wherein the collecting of the text corpus comprises collecting a relevance indicating measure associated with text in the text corpus.

26. A method according to claim 25, wherein the predicting of the value of the variable comprises numerically weighting the value with the relevance indicating measure.

27. An apparatus (300) for providing a control input signal (412) for an industrial process or technical system (400) having one or more controllable elements (421-42n), the apparatus comprising:

means (320;

909) for generating a semantic space (892) for a text corpus (302;

908);

means (330;

914) for generating a norm (896) from one or more reference words or texts (332;

893), the or each reference word or text being associated with a defined respective value on a scale, and the norm being calculated as a reference point or set of reference points in the semantic space for the or each reference word or text with its associated respective scale value;

means (340) for reading at least one target word;

means (340) for predicting a value (350) of a variable associated with the target word based on the semantic space and the norm; and

means (340) for providing the predicted value in a control input signal (412) to said industrial process or technical system (400),

wherein the means (320;

909) for generating the semantic space (892) for the text corpus (302;

908) includes collecting time information associated with text in the text corpus, and

wherein the predicting of the value of the variable comprises associating the predicted value with the time information of the text corpus.

28. An apparatus as defined in claim 27, further comprising means (310) for collecting said digital text corpus from a plurality of distributed text sources (314-316) accessible via one or more networks (312).

29. An apparatus as defined in claim 27, further comprising a data processing unit, wherein said data processing unit is configured to perform a method for predicting a value of a variable associated with a target word or set of words, performed on at least one computer and comprising the steps of:

collecting a text corpus comprising a set of words that include the target word,

generating a representation of the text corpus,

creating a semantic space for the set of words, based on the representation of the text corpus,

defining, for a location in the semantic space, a value of the variable,

estimating, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculating a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word.

30. A system for predicting a value of a variable associated with a target word or set of words, comprising at least one computer and configured to:

collect a text corpus comprising a set of words that include the target word,

generate a representation of the text corpus,

create a semantic space for the set of words, based on the representation of the text corpus,

define, for a location in the semantic space, of a subset of the words, a value of the variable,

estimate, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculate a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word,

wherein the collecting of the text corpus comprises collecting time information associated with text in the text corpus, and

wherein the predicting of the value of the variable comprises associating the predicted value with the time information of the text corpus.

31. A system according to claim 30, wherein the creating of the semantic space comprises performing a singular value decomposition on the representation of the text corpus.

32. A system according to claim 30, further configured for statistically testing the predicted value of the variable, by comparing the predicted value with known values.

33. A system according to claim 30, further configured for statistically testing if two sets of words or two sets of documents of the text corpora differ in semantic representation.

34. A system according to claim 30, wherein the collecting of the text corpus comprises collecting a relevance indicating measure associated with text in the text corpus.

35. A system according to claim 34, wherein the predicting of the value of the variable comprises numerically weighting the value with the relevance indicating measure.

36. A non-transitory computer readable medium having stored thereon a computer program having software instructions which when run on a computer cause the computer to perform the steps of:

collecting a text corpus comprising a set of words that include the target word,

generating a representation of the text corpus,

creating a semantic space for the set of words, based on the representation of the text corpus,

defining, for a location in the semantic space, a value of the variable,

estimating, for the target word, a value of the variable, based on the semantic space and the defined variable value of the location in the semantic space, and

calculating a predicted value of the target word, on basis of the semantic space, the defined variable value of the location in the semantic space and the estimated variable value of the target word,

wherein the collecting of the text corpus comprises collecting time information associated with text in the text corpus, and

wherein the predicting of the value of the variable comprises associating the predicted value with the time information of the text corpus.