Data analyzing method and system

• US 5,761,389 A
• Filed: 06/06/1995
• Issued: 06/02/1998
• Est. Priority Date: 09/07/1994
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Taniguchi, Yoji, Maeda, Akira, Ashida, Hitoshi, Ito, Yukiyasu, Takahashi, Yori
• Primary Examiner(s): Hafiz, Tariq R.
• Application Number:
  US08/470,217
• Time in Patent Office: 1,092 Days
• Field of Search: 395/11, 395/3, 395/61, 395/67, 395/22, 395/66, 395/77, 395/1, 395/21, 395/2, 395/50, 395/75
• US Class Current: 706/59
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A data analyzing method and system for generating a rule based on data items in a data base, wherein the rule expresses relational features of the data items. The invention includes a user interface and a rule generation module. The rule generation module, in response to an input from the user via the user interface, selects data items for use in a conditional clause and a conclusion clause of a rule from the data items stored in the data base, converts, when the selected data items have numerical values, the numerical values into symbolic values and creates plural candidate rules each expressing a correlation between selected data items in a rule form having one or plural sets of item names and symbolic values. The rule generation module further calculates a criterion for evaluating strength of correlation between data items in each of the candidate rules, determines one or plural candidate rules having highest calculated criterion from the candidate rules, and outputs to the user via the user interface the one or plural candidate rules.

Claims

1. A data analyzing method, utilized in an information processing system including a user interface and a rule generation module, of generating a rule based on a plurality of data items stored in a data base, said rule expressing relational features of said data items, said method comprising the steps, executed by said information processing system, of:

selecting data items for use in an IF clause and a THEN clause of an IF, THEN rule from said data items stored in said data base;

when said selected data items have numerical values, converting said numerical values into symbolic values;

creating plural candidate rules in said rule generation module, each rule expressing a correlation between selected data items in a rule form having one or plural sets of item names and symbolic values;

calculating a criterion for evaluating strength of correlation between data items in each of said candidate rules;

determining one or plural candidate rules having highest calculated criterion from said candidate rules; and

outputting said one or plural candidate rules having the highest calculated criterion.

2. A data analyzing method according to claim 1, wherein said selecting step is carried out such that a degree of correlation between data items is determined and data item selection is performed automatically based on said degree of correlation.

3. A data analyzing method according to claim 1, wherein said converting step is carried out by determining several symbolic values and a membership function for each symbolic value, said membership function indicating the degree of conformity of the numerical values to the determined symbolic values.

4. A data analyzing method according to claim 3, wherein a weight of each input record is calculated based on values of said membership functions, and the strength of correlation between selected data items is evaluated by using information of said weight.

5. A data analyzing method according to claim 4, further comprising the step, executed by said information processing system, of:

adjusting the value of a parameter which designates the shape of each said membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

6. A data analyzing method according to claim 3, further comprising the step, executed by said information processing system, of:

adjusting the value of a parameter which designates the shape of each said membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

7. A data analyzing method according to claim 1, further comprising the steps, executed by said information processing system, of:

when the value of a certain data item present within the records to be analyzed includes a deficit value, adding a symbolic value indicative of the deficit value as a symbolic value of said data item; and

instructing, for each data item, whether or not the symbolic value indicating the deficit value is to be used in the rule to be created.

8. A data analyzing method according to claim 7, further comprising the step, executed by said information processing system, of:

deciding whether the symbolic value indicative of the deficit value is to be used automatically based on a dependence relation between said data item and said conclusion item.

9. A data analyzing method according to claim 1, wherein said candidate rule criterion is calculated from both a cover ratio expressed by the number of records related to the candidate rules and an accuracy ratio expressed by a correct answer ratio of the candidate rules, said method further comprises the steps, executed by said information processing system, of:

designating, as parameters a degree of importance which is to be attached to the cover ratio and the accuracy ratio.

10. A data analyzing method according to claim 1, further comprising the step, executed by said information processing system, of:

determining a functional relation present between the data items of each input record, and wherein, based on said functional relation, there are not performed generation and display of a redundant rule.

11. A data analyzing method according to claim 3, further comprising the steps of:

calculating a weight of each input record as a multiple of said membership function values of the numerical values included in the record; and

calculating a number of records as a sum of said weights of the records when calculating a criterion for evaluating the strength of the correlation between the selected data items.

12. A data analyzing method, utilized in an information processing system including a user interface and a rule generation module, of generating a rule based on a plurality of data items stored in a data base, said rule expressing relational features of said data items, said method comprising the steps, executed by said information processing system, of:

selecting one conclusion item as a data item for use in a conclusion clause of a rule from said data items stored in said data base;

selecting condition items as data items for use in a condition clause of said rule from said data items stored in said data base;

when said selected data items have numerical values, converting said numerical values into symbolic values;

combining one or plural sets of item names and symbolic values thereof in the condition items to create a condition clause, combining an item name and a symbolic value thereof in the conclusion items to create a conclusion clause, and combining the thus-created condition and conclusion clauses to create plural candidate rules;

calculating a criterion for evaluating said candidate rules;

determining one or plural candidate rules out of said candidate rules having a high evaluation level; and

outputting said one or plural candidate rules determined by said determining step as having the high evaluation level.

13. A data analyzing method according to claim 12, wherein said selecting condition items step is carried out such that a degree of correlation between data items is determined and data item selection is performed automatically based on said degree of correlation.

14. A data analyzing method according to claim 12, wherein said step of selecting condition items step includes the sub-steps of:

calculating a similarity barometer indicating a degree of correlation between items with respect to all combinations of data items other than said conclusion item among said data items;

collecting data items having a high degree of correlation other than said conclusion item in accordance with said similarity barometer and then clustering the collected data items into clusters;

calculating a dependency barometer indicating a degree of correlation between items with respect to all combinations of said conclusion item with data items other than said conclusion item; and

selecting a data item having the highest degree of correlation with the conclusion item among said clusters, and adopting the selected data item as said condition item, in accordance with the calculated dependency barometer.

15. A data analyzing method according to claim 12, wherein said converting step is carried out by determining several symbolic values and a membership function for each symbolic value, said membership function indicating the degree of conformity of the numerical values to the determined symbolic values.

16. A data analyzing method according to claim 15, wherein a weight of each input record is calculated based the values of said membership functions, and the strength of correlation between selected data items is evaluated by using information of said weight.

17. A data analyzing method according to claim 12, further comprising the step, executed by said information processing system, of:

adjusting the value of a parameter which designates the shape of each membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

18. A data analyzing method according to claim 15, further comprising the step, executed by said information processing system, of:

adjusting the value of a parameter which designates the shape of each membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

19. A data analyzing method according to claim 12, further comprising the steps, executed by said information processing system, of:

when the value of a certain data item present within the records to be analyzed includes a deficit value, adding a symbolic value indicative of the deficit value as a symbolic value of said data item; and

instructing for each data item whether the symbolic value indicating the deficit value is to be used or not in the rule to be created.

20. A data analyzing method according to claim 19, further comprising the step, executed by said information processing system, of:

deciding whether the symbolic value indicative of the deficit value is to be used automatically based on a dependence relation between said data item and said conclusion item.

21. A data analyzing method according to claim 12, wherein said candidate rule criterion is calculated from both a cover ratio expressed by the number of records related to the candidate rules and an accuracy ratio expressed by a correct answer ratio of the candidate rules, and further comprising the step, executed by said information processing system, of:

designating, as parameters, a degree importance which is to be attached to the cover ratio and the accuracy ratio.

22. A data analyzing method according to claim 12, further comprising the step, executed by said information processing system, of:

determining a functional relation present between the data items of each input record, wherein based on said functional relation there are not performed generation and display of a redundant rule.

23. A data analyzing system which groups data of numerical or symbolic values as values of plural data items into one record, then inputs and analyzes data including a plurality of said records, and creates and outputs a rule expressing features of said data, said system comprising:

means for selecting data items for use in an IF clause and a THEN clause of an IF, THEN rule out of said plural data items;

means for converting the numerical values into symbolic values, when the selected data items have numerical values;

means for creating plural candidate rules expressing a correlation between selected data items in a rule form having one or plural sets of item names and symbolic values;

means for calculating a criterion for evaluating the strength of the correlation expressed by said candidate rules;

means for determining one or plural candidate rules having the highest of said criterion out of the created plural candidate rules; and

means for outputting the thus-determined candidate rules having the highest criterion.

24. A data analyzing system comprising:

a user interface for interfacing with a user;

a memory for storing a data base including a plurality of data items; and

a rule generation module for generating a rule based on said data items in said data base, said rule expressing relational features of said data items;

wherein said rule generation module, in response to an input from said user via said user interface, selects data items for use in an IF clause and a THEN clause of an IF, THEN rule from said data items stored in said data base;

converts and numerical values into symbolic values, when said selected data items have numerical values;

creates plural candidate rules each expressing a correlation between selected data items in a rule form having one or plural sets of item names and symbolic values;

calculates a criterion for evaluating strength of correlation between data items in each of said candidate rules;

determines one or plural candidate rules having highest calculated criterion from said candidate rules, and outputs said one or plural candidate rules to said user via said user interface.

25. A data analyzing system according to claim 24, wherein said rule generation module selects said data item such that a degree of correlation between data items is determined and data item selection is performed automatically based on said degree of correlation degree.

26. A data analyzing system according to claim 24, wherein said rule generation module converts said numerical values by determining several symbolic values and a membership function for each symbolic value, said membership function indicates the degree of conformity of the numerical values to the symbolic values determined.

27. A data analyzing system according to claim 26, wherein said rule generation module calculates weight of each input record based on values of said membership functions, and evaluates the strength of correlation between selected data items by using information of said weight.

28. A data analyzing system according to claim 27, wherein said rule generation modules comprises:

means for adjusting the value of a parameter which designates the shape of each said membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

29. A data analyzing system according to claim 26, wherein said rule generation modules comprises:

means for adjusting the value of a parameter which designates the shape of each said membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

30. A data analyzing system according to claim 24, wherein said rule generation modules comprises:

means for adding a symbolic value indicative of the deficit value as a symbolic value of said data items and instructing, for each data item, whether the symbolic value indicating the deficit value is to be used in the rule to be created, when the value of a certain data item present within the records to be analyzed includes a deficit value.

31. A data analyzing system according to claim 30, wherein said rule generation modules comprises:

means for deciding whether the symbolic value indicative of the deficit value is to be used automatically based on a dependence relation between said data item and said conclusion item.

32. A data analyzing system according to claim 24, wherein said rule generation modules calculates said candidate rule criterion from both a cover ratio expressed by the number of records related to the candidate rules and an accuracy ratio expressed by a correct answer ratio of the candidate rules, and designates as parameters to what degree importance is to be attached to the cover ratio and the accuracy ratio.

33. A data analyzing system according to claim 24, wherein said rule generation modules comprises:

means for determining a functional relation present between the data items of each input record, and wherein there are not performed generation and display of a redundant rule based on said functional relation.

34. A data analyzing system comprising:

a user interface for interfacing with a user;

a memory for storing a data base including a plurality of data items; and

a rule generation module for generating a rule based on said data items in said data base, said rule expressing relational features of said data items;

wherein said rule generation module, in response to an input from said user via said interface, selects one conclusion item as a data item for use in a conclusion clause of a rule from said data items stored in said data base;

selects condition items as data items for use in a condition clause of said rule from said data items stored in said data base;

converts said numerical values into symbolic values, when said selected data items have numerical values;

combines one or plural sets of item names and symbolic values thereof in the condition items to create a condition clause;

combines an item name and a symbolic value thereof in the conclusion items to create a conclusion clause;

combines the thus-created condition and conclusion clauses to create plural candidate rules;

calculates a criterion for evaluating said candidate rules;

determines one or plural candidate rules out of said candidate rules having a highest evaluation level; and

outputs to said user via said user interface said one or plural candidate rules having the highest evaluation level.

35. A data analyzing system according to claim 34, wherein said rule generation module selects condition items in such a manner that a degree of correlation between data items is determined and data item selection is performed automatically based on said degree of correlation degree.

36. A data analyzing system according to claim 34, wherein said rule generation module selects condition items by calculating a similarity barometer indicating a degree of correlation between items with respect to all combinations of data items other than said conclusion item among said data items, collecting data items having a high correlativity other than said conclusion item in accordance with said similarity barometer and then clustering the collected data items into clusters, calculating a dependency barometer indicating a degree of correlation between items with respect to all combinations of said conclusion item with data items other than said conclusion item, and selecting a data item having the highest degree of correlation with the conclusion item among said clusters, and adopting the selected data item as said condition item, in accordance with the calculated dependency barometer.

37. A data analyzing system according to claim 34, wherein said rule generation module converts numerical values by determining several symbolic values and a membership function for each symbolic value, said membership function indicating the degree of conformity of the numerical values to the symbolic values determined.

38. A data analyzing system according to claim 37, wherein said rule generation module calculates a weight of each input record based on values of said membership functions, and evaluates the strength of correlation between selected data items by using information of said weight.

39. A data analyzing system according to claim 38, wherein said rule generation modules comprises:

means for adjusting the value of a parameter which designates the shape of each membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

40. A data analyzing system according to claim 37, wherein said rule generation modules comprises:

means for adjusting the value of a parameter which designates the shape of each membership function so as to maximize the criterion of said rule, by using the whole or part of the output rule and also using the input data.

41. A data analyzing system according to claim 34, wherein said rule generation modules comprises:

means for adding a symbolic value indicative of the deficit value as a symbolic value of said data item, when the value of a certain data item present within the records to be analyzed includes a deficit value and instructing for each data item whether the symbolic value indicating the deficit value is to be used in the rule to be created.

42. A data analyzing system according to claim 41, wherein said rule generation modules comprises:

means for deciding whether the symbolic value indicative of the deficit value is to be used automatically based on a dependence relation between said data item and said conclusion item.

43. A data analyzing system according to claim 34 wherein said rule generation module calculates said candidate rule criterion from both a cover ratio expressed by the number of records related to the candidate rules and an accuracy expressed by a correct answer ratio of the candidate rules, and designates, as parameters, a degree of importance which is to be attached to the cover ratio and the accuracy.

44. A data analyzing system according to claim 34, wherein said rule generation modules comprises:

means for determining a functional relation present between the data items of each input record, and wherein based on said functional relation there are not performed generation and display of a redundant rule.