Architecture for automated data analysis

• US 6,330,563 B1
• Filed: 04/23/1999
• Issued: 12/11/2001
• Est. Priority Date: 04/23/1999
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rounthwaite, Robert L., Meek, Christopher A., Heckerman, David E., Chickering, D. Maxwell
• Primary Examiner(s): Alam, Hosain T.
• Assistant Examiner(s): HWANG, JOON H
• Application Number:
  US09/298,717
• Time in Patent Office: 963 Days
• Field of Search: 707/100, 707/5, 709/7, 709/10
• US Class Current: 1/1
• CPC Class Codes: G06F 17/10 Complex mathematical operations function generation by table look-up G06F1/03; evaluation of elementary functions by calculation G06F7/544
• CPC Class Codes: G06F 2216/03 Data mining
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

An architecture for automated data analysis. In one embodiment, a computerized system comprising an automated problem formulation layer, a first learning engine, and a second learning engine. The automated problem formulation layer receives a data set. The data set has a plurality of records, where each record has a value for each of a plurality of raw transactional variables. The layer abstracts the raw transactional variables into cooked transactional variables. The first learning engine generates a model for the cooked transactional variables, while the second learning engine generates a model for the raw transactional variables.

Claims

1. A computerized system comprising:

an automated problem formulation layer to receive a data set comprising a plurality of records, each record having a value for each of a plurality of raw transactional variables, and to abstract the raw transactional variables organized into a hierarchy of nodes over which a concept is defined as one of a root node of the hierarchy, a node in the hierarchy, and a set of nodes in the hierarchy having a same parent node into a plurality of cooked transactional variables, each cooked transactional variable corresponding to an unrefined concept after the hierarchy has been successively refined a number of times starting with an initial concept equal to the root node,

where a refined concept is defined as a first concept refined into two second concepts such that one of the two second concepts is a most populous node of the hierarchy as measured by the plurality of records having non-zero values for a raw transactional variable corresponding to the most populous node and the two second concepts cover a same set of nodes of the hierarchy as the first concept does;

a first learning engine to generate a model for the plurality of cooked transactional variables based on the cooked transactional variables, the model having a type; and

,

a second learning engine to generate a model for the plurality of raw transactional variables based on the model for the plurality of cooked transactional variables and the type of the model for the plurality of cooked transactional variables.

2. The system of claim 1, further comprising a tool to analyze at least one of the model for the plurality of raw transactional variables and the model for the plurality of cooked transactional variables.

3. The system of claim 2, wherein the tool comprises a visualization tool.

4. The system of claim 2, wherein the tool comprises an explanation tool.

5. The system of claim 1, wherein the layer persists transformation of the plurality of raw transactional variables to the plurality of cooked transactional variables as a transformation model.

6. The system of claim 1, wherein the layer comprises a software object.

7. The system of claim 1, wherein the layer comprises a Component Object Model (COM) object.

8. The system of claim 1, wherein at least one of the layer and the first and the second learning engine comprises a computer program executable by a processor from a computer-readable medium.

9. A computer-implemented method comprising:

inputting a data set comprising a plurality of records, each record having a value for each of a plurality of raw transactional variables organized into a hierarchy of nodes over which a concept is defined as one of a root node of the hierarchy, a node in the hierarchy, and a set of nodes in the hierarchy having a same parent node;

abstracting the raw transactional variables into a plurality of cooked transactional variables, each cooked transactional variable corresponding to an unrefined concept after the hierarchy has been successively refined a number of times starting with an initial concept equal to the root node,

where a refined concept is defined as a first concept refined into two second concepts such that one of the two second concepts is a most populous node of the hierarchy as measured by the plurality of records having non-zero values for a raw transactional variable corresponding to the most populous node and the two second concepts cover a same set of nodes of the hierarchy as the first concept does;

generating a model for the plurality of cooked transactional variables based on the cooked transactional variables, the model having a type;

generating a model for the plurality of raw transactional variables based on the model for the plurality of cooked transactional variables and the type of the model for the plurality of cooked transactional variables; and

,

outputting at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables.

10. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables for an application selected from the group essentially consisting of:

pattern recognition, data modeling, and data mining.

11. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict what products a user will want to buy given what is already in a shopping basket of the user.

12. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict what ads a user will click on based on at least one of what other ads the user has clicked on, what web pages the user has read, and demographics regarding the user.

13. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict what television shows a user will want to watch based on at least one of what other television shows the user has watched and demographics regarding the user.

14. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict a desired song of a user based on songs the user has already listened to.

15. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict a desired music video of a user, based on music videos the user has already viewed.

16. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict desired music to be purchased by a user, based on music the user has already purchased.

17. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to customize an electronic newspaper for a user based on types of stories the user has already read.

18. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict at least one of:

a desired Internet page of a user;

a desired Encarta article of a user;

a desired graphical layout of a user for a graphical presentation program;

a desired email alias of a user;

a desired movie of a user;

a desired wine of a user;

a desired restaurant of a user;

a desired real estate of a user;

a desired advertisement of a user; and

, a desired user interface of a user.

19. The method of claim 9, wherein outputting comprises utilizing at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables to predict inappropriate material that is deemed unsuitable for viewing by children.

20. The method of claim 9, wherein outputting the at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables comprises providing a tool to analyze at least one of the model for the plurality of raw transactional variables and the model for the plurality of cooked transactional variables.

21. The method of claim 20, wherein the tool comprises a visualization tool.

22. The method of claim 20, wherein the tool comprises an explanation tool.

23. The method of claim 9, wherein abstracting the raw transactional variables into a plurality of cooked transactional variables comprises persisting transformation of the plurality of raw transactional variables to the plurality of cooked transactional variables as a transformation model.

24. A computer-readable medium having instructions stored thereon for execution by a processor to perform a method comprising:

inputting a data set comprising a plurality of records, each record having a value for each of a plurality of raw transactional variables organized into a hierarchy of nodes over which a concept is defined as one of a root node of the hierarchy, a node in the hierarchy, and a set of nodes in the hierarchy having a same parent node;

abstracting the raw transactional variables into a plurality of cooked transactional variables, each cooked transactional variable corresponding to an unrefined concept after the hierarchy has been successively refined a number of times starting with an initial concept equal to the root node,

where a refined concept is defined as a first concept refined into two second concepts such that one of the two second concepts is a most populous node of the hierarchy as measured by the plurality of records having non-zero values for a raw transactional variable corresponding to the most populous node and the two second concepts cover a same set of nodes of the hierarchy as the first concept does;

generating a model for the plurality of cooked transactional variables based on the cooked transactional variables, the model having a type;

generating a model for the plurality of raw transactional variables based on the model for the plurality of cooked transactional variables and the type of the model for the plurality of cooked transactional variables; and

,

outputting at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables.

25. The medium of claim 24, wherein outputting the at least one of the model for the plurality of cooked transactional variables and the model for the plurality of raw transactional variables comprises providing a tool to analyze at least one of the model for the plurality of raw transactional variables and the model for the plurality of cooked transactional variables.

26. The medium of claim 25, wherein the tool comprises a visualization tool.

27. The medium of claim 25, wherein the tool comprises an explanation tool.

28. The medium of claim 24, wherein abstracting the raw transactional variables into a plurality of cooked transactional variables comprises persisting transformation of the plurality of raw transactional variables to the plurality of cooked transactional variables as a transformation model.