SELF-LEARNING DATA LENSES

• US 20090281792A1
• Filed: 01/05/2009
• Published: 11/12/2009
• Est. Priority Date: 05/23/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Silver Creek Systems, Inc. (Oracle Corporation)
• Inventors: MARKEY, KEVIN L., GREEN, EDWARD A.
• Granted Patent: US 9,037,613 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/24526 Internal representations for queries
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

A semantic conversion system (1900) includes a self-learning tool (1902). The self-learning tool (1902) receives input files from legacy data systems (1904). The self-learning tool (1902) includes a conversion processor (1914) that can calculate probabilities associated with candidate conversion terms so as to select an appropriate conversion term. The self-learning tool (1902) provides a fully attributed and normalized data set (1908).

Claims

1. A method for use in converting semantic information from a source form to a target form, comprising the steps of:

providing a computer-based analysis tool for executing a conversion analysis having conversion logic that is independent of a conversion environment under consideration;

obtaining a first set of data reflecting a conversion environment under consideration;

first operating said analysis tool on said first set of data to develop a conversion model specific to said conversion environment under consideration; and

second operating said analysis tool to convert a second set of data, reflecting said conversion environment, from said source form to said target form by applying said developed conversion model to said second set of data.

2. A method as set forth in claim 1, wherein said first set of data includes structured data having structural elements defining attributes of the data and content elements defining attribute values of specific items of data, and said step of first operating comprises obtaining a set of attributes by analyzing said structural elements and obtaining at least one definition of acceptable attribute values by analyzing said content elements.

3. A method as set forth in claim 2, wherein said first set of data is provided in table form and at least some of said structural elements define said table structure and at least some of said content elements populate addresses of said table.

4. A method as set forth in claim 2, wherein said first set of data includes logged data having content element defining attribute values of specific items of data and tag elements describing said content elements, and said step of first operating comprises obtaining a set of attributes by analyzing said tag elements and obtaining at least one definition of acceptable attribute values by analyzing said content elements.

5. A method as set forth in claim 1, wherein said step of first operating comprises performing a statistical analysis on said first set of data to identify attributes and attribute values and to associate probability information with said attributes and attribute values.

6. A method as set forth in claim 5, wherein said statistical analysis is applied with respect to substantially unstructured semantic data.

10. A method as set forth in claim 1, wherein said step of obtaining comprises receiving a search string entered into a search tool.

7. A method for use in converting semantic information from a source form to a target form, comprising the steps of:

obtaining an input string for conversion from said source form to said target form;

first identifying a source term of said string for conversion;

second identifying a subject matter context of said string;

based on said subject matter context, establishing a set of at least two candidate conversion terms for said source term;

performing an analysis on said candidate conversion terms to select said candidate conversion terms for use in converting said source term; and

using said selected one of said conversion terms to convert said source term from said source form to said target form.

8. A method as set forth in claim 7, wherein said input string comprises unstructured semantic information.

9. A method as set forth in claim 7, wherein said input string comprises data structured in accordance with a source data structure.

11. A method as set forth in claim 7, wherein said search string is substantially in a free speech form.

12. A method as set forth in claim 7, wherein said step of obtaining comprises accessing stored data of a legacy data system.

13. A method as set forth in claim 7, wherein said step of first identifying comprises applying logic to parse said input string into cognizable units and selecting one or more of said units as said source term.

14. A method as set forth in claim 13, wherein said logic selects said source term based on identifying a frame of said input string and identifying a missing attribute or attribute value in relation to said input string.

15. A method as set forth in claim 7, wherein said step of second identifying is executed before said step of first identifying.

16. A method as set forth in claim 7, wherein said step of second identifying comprises inferring said subject matter context from analysis of said input string.

17. A method as set forth in claim 7, wherein said step of second identifying comprises determining said subject matter context from analyzing metadata associated with said input string describing a semantic information content of said input string.

18. A method as set forth in claim 7, wherein said step of second identifying comprises performing an analysis of other data from an input source of said input string.

19. A method as set forth in claim 7, wherein said step of second identifying comprises receiving a user input indicating said subject matter context.

20. A method as set forth in claim 7, wherein said step of establishing comprises accessing a master set of conversion terms and using a linguistic comparison tool to select first candidate conversion terms from said master set of conversion terms.

21. A method as set forth in claim 20, wherein said master set of conversion terms is obtained at least in part from a standardized listing compiled for said subject matter context.

22. A method as set forth in claim 20, wherein said master set of conversion terms is obtained based on analysis of other data from an input source of said input string.

23. A method as set forth in claim 20, wherein said step of establishing comprises eliminating at least one of said first candidate conversion terms to establish said set of candidate conversion terms.

24. A method as set forth in claim 23, wherein said step of eliminating comprise determining that at least one eliminated candidate conversion term is an incorrect conversion term based on a contextual cue.

25. A method as set forth in claim 24, wherein said contextual cue is based on a classification taxonomy structure of said subject matter context.

26. A method as set forth in claim 18, wherein said contextual cue comprises a rule of a frame-slot structure.

27. A method as set forth in claim 26, wherein said rule defines acceptable values of an attribute associated with said input string.

28. A method as set forth in claim 7, wherein said step of performing an analysis comprises establishing a statistical probability for each of said candidate conversion terms and selecting said one of said conversion terms having a highest probability.

29. A method as set forth in claim 28, wherein said establishing a probability comprises determining said probability based on an analysis of successive pieces of evidence for each candidate term.

30. A method as set forth in claim 29, wherein at least one of said successive pieces of evidence relates to, for a given candidate conversion term, an occurrence of an associated term in said input string.

31. A method as set forth in claim 28, wherein said establishing a statistical probability involves, for a given candidate term, determining a frequency of occurrence of said given candidate term in a set of sample data.

32. A method as set forth in claim 7, wherein said target form is different than said selected candidate term.

33. A method for use in converting semantic information from a source form to a target form, comprising the steps of:

obtaining a set of at least two candidate conversion terms for use in converting a given source term from said source form to a target form;

performing an analysis of each of said candidate conversion terms in relation to a series of pieces of evidence, where each piece of evidence, when applied to one of said conversion terms, has an impact on the likelihood that said one conversion term is a correct conversion term, so as to progressively determine which of said candidate conversion terms is most likely to be said correct conversion term; and

based on said analysis, using a selected one of said candidate conversion terms to convert said given source term from said source form to said target form.

34. A method for use in operating a system for converting information from a source form to a target form, comprising the steps of:

providing a self-learning statistical tool for analyzing multiple candidate conversion terms for a given source term so as to select a correct one of said conversion terms for use in converting said given source term, said self-learning statistical tool being operative to determine a probability, for each one of said candidate terms, that said one candidate term is said correct candidate term;

training said self-learning statistical tool by providing a set of training data and operating said self-learning statistical tool on said training data to develop statistical parameters relative to conversion of said information from said source form to said target form; and

using said trained self-learning statistical tool to convert a first source term from said source form to said target form by applying at least one of said statistical parameters with respect to said first source term.

35. An apparatus for use in converting semantic information from a source form to a target form, comprising:

an input port for receiving an input string for conversion from said source form to said target form, said input string including a source term;

a matching module for identifying a set of at least two candidate conversion terms potentially corresponding to said source term; and

a statistical tool for analyzing said candidate conversion terms to select one of said candidate conversion terms for use in converting said source term from said source form to said target form, said statistical tool being operative for establishing a statistical probability for each of said candidate conversion terms and selecting said one conversion term based on said statistical probability of said one conversion term.

36. A method for use in converting semantic information from a source form to a target form, comprising the steps of:

accessing a set of structured data reflecting information of a source repository, said structured data including indications of a number of attributes;

analyzing said set of structured data to define a set of attributes; and

using said identified, set of attributes to convert an item of source data from said source form to said target form.

37. A method as set forth in claim 36, wherein said structured data includes multiple data entries having one or more common data fields and said indications identify said data fields.