Discretization of dimension attributes using data mining techniques

• US 20060005121A1
• Filed: 06/30/2004
• Published: 01/05/2006
• Est. Priority Date: 06/30/2004
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Berger, Alexander, Melomed, Edward, Iyer, Raman, Krishnamurthi Anand, Thulusalamatom
• Application Number:
  US10/881,262
• Publication Number:
  US 20060005121A1
• Time in Patent Office: Days
• US Class Current: 715/230
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 2216/03 Data mining

Abstract

In order to allow the use of data in dimension attributes for grouping members of a dimension, dimension attribute data is analyzed so it can be used as if it were data for a categorical attribute with a manageable number of states. The values possible for the dimension attribute are divided into groups. This is done by determining the distribution of data. An approximate distribution may be determined (by sampling some data) or an actual distribution may be determined (by sampling all data). The distribution is then used to determine the groups into which the range of data values will be divided. Each group is then treated as if it were a state for a categorical-type dimension attribute. A state can be determined for a member by determining which subrange contains the value for the dimension attribute for the member. The number of groups can be determined by a user or determined dynamically, e.g. to best fit the distribution found. The group data may be stored in order to allow further conversion of future cases.

Claims

1. A method for grouping members of a dimension for OLAP data, each of said members comprising a corresponding value for at least one dimension attribute, comprising:

determining a distribution of said corresponding values;

using said distribution to divide said corresponding values into at least two groups; and

determining a specific group from among said at least two groups to assign for a given one of said members, where said group contains said corresponding value for said given member.

2. The method of claim 1, further comprising:

displaying selected cases from said dimension to a user based on said determination of a specific group.

3. The method of claim 2 where said step of displaying selected cases comprises accepting browsing commands from a user.

4. The method of claim 1, where determination of a distribution comprises determining an approximation of the distribution of said corresponding values in said dimension using a sample set of members selected from said dimension.

5. The method of claim 1, where the number of groups is determined by a user.

6. The method of claim 1, where the number of groups is determined dynamically.

7. The method of claim 1, where said use of said distribution comprises using a K-means algorithm to create said groups.

8. The method of claim 1, where said use of said distribution comprises using an equal areas algorithm to create said groups such that, for each group, there are an approximately equal number of members for which the corresponding value falls in said group.

9. The method of claim 1, where said use of said distribution comprises determining at least one point where the gradient of said distribution changes from positive to negative and using said at least one point to determine said at least two groups.

10. The method of claim 1, where said use of said distribution comprises using an agglomeration clustering algorithm to create said groups.

11. The method of claim 1, where further members are added to said dimension, said method further comprising:

storing data regarding said groups;

determining a specific group from among said at least two groups for a given one of said further members.

12. A computer-readable medium having computer-executable instructions for grouping members of a dimension for OLAP data, each of said members comprising a corresponding value for at least one dimension attribute, said instructions for performing steps comprising:

determining a distribution of said corresponding values;

using said distribution to divide said corresponding values into at least two groups; and

determining a specific group from among said at least two groups to assign for a given one of said members, where said group contains said corresponding value for said given member.

13. The computer-readable medium of claim 12, said steps further comprising:

displaying selected cases from said dimension to a user based on said determination of a specific group.

14. The computer-readable medium of claim 13 where said step of displaying selected cases comprises accepting browsing commands from a user.

15. The computer-readable medium of claim 12, where determination of a distribution comprises determining an approximation of the distribution of said corresponding values in said dimension using a sample set of members selected from said dimension.

16. The computer-readable medium of claim 12, where the number of groups is determined by a user.

17. The computer-readable medium of claim 12, where the number of groups is determined dynamically.

18. The computer-readable medium of claim 12, where said use of said distribution comprises using a K-means algorithm to create said groups.

19. The computer-readable medium of claim 12, where said use of said distribution comprises using an equal areas algorithm to create said groups such that, for each group, there are an approximately equal number of members for which the corresponding value falls in said group.

20. The computer-readable medium of claim 12, where said use of said distribution comprises determining at least one point where the gradient of said distribution changes from positive to negative and using said at least one point to determine said at least two groups.

21. The computer-readable medium of claim 12, where said use of said distribution comprises using an agglomeration clustering algorithm to create said groups.

22. The computer-readable medium of claim 12, where further members are added to said dimension, said steps further comprising:

storing data regarding said groups;

determining a specific group from among said at least two groups for a given one of said further members.

23. A data converter for grouping members of a dimension for OLAP data, each of said members comprising a corresponding value for at least one dimension attribute, comprising:

a distribution determiner for determining a distribution of said corresponding values;

a range divider for using said distribution to divide said corresponding values into at least two groups; and

a group assigner for determining a specific group from among said at least two groups to assign for a given one of said members, where said group contains said corresponding value for said given member

24. The data converter of claim 23, further comprising:

a display for displaying selected cases from said dimension to a user based on said determination of a specific group.

25. The data converter of claim 24 further comprising:

a command accepter for accepting browsing commands from a user.

26. The data converter of claim 23, where determination of a distribution comprises determining an approximation of the distribution of said corresponding values in said dimension using a sample set of members selected from said dimension.

27. The data converter of claim 23, where the number of groups is determined by a user.

28. The data converter of claim 23, where the number of groups is determined dynamically.

29. The data converter of claim 23, where said use of said distribution comprises using a K-means algorithm to create said groups.

30. The data converter of claim 23, where said use of said distribution comprises using an equal areas algorithm to create said groups such that, for each group, there are an approximately equal number of members for which the corresponding value falls in said group.

31. The data converter of claim 23, where said use of said distribution comprises determining at least one point where the gradient of said distribution changes from positive to negative and using said at least one point to determine said at least two groups.

32. The data converter of claim 23, where said use of said distribution comprises using an agglomeration clustering algorithm to create said groups.