Computer systems and methods for automatic generation of models for a dataset

• US 7,999,809 B2
• Filed: 04/17/2007
• Issued: 08/16/2011
• Est. Priority Date: 04/19/2006
• Status: Active Grant

• Current Assignee: Tableau Software Incorporated (Salesforce.com, Inc.)
• Original Assignee: Tableau Software Incorporated (Salesforce.com, Inc.)
• Inventors: Eldridge, Matthew W., Beers, Andrew C., Taylor, Jonathan E., Hanrahan, Patrick M.
• Primary Examiner(s): Vy; Hung T
• Application Number:
  US11/787,761
• Publication Number:
  US 20070250523A1
• Time in Patent Office: 1,582 Days
• Field of Search: 345/440, 707/100, 707/104.1, 707/102
• US Class Current: 345/440
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G16B 40/00 ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding

Abstract

A method of automatically generating models from a dataset includes multiple steps. First, a description of a view of a dataset is provided. The description includes multiple fields associated with the dataset. Next, a set of properties is determined for each of the multiple fields. Finally, the description is automatically translated into one or more models based on the respective properties of the multiple fields and a set of predefined heuristics.

Claims

1. A method of automatically generating statistical models for a dataset, comprising:

at a computer system having one or more processors and memory storing programs executed by the one or more processors;

receiving a user description of a graphical view of a dataset, wherein the user description specifies a graphical relationship between a first field and a second field of the dataset in the graphical view;

determining a set of data properties for each of the first and second fields;

with no further user input, generating one or more statistical models of the dataset in accordance with the data properties of the first and second fields and data values associated with the first and second fields in the dataset, wherein the one or more statistical models are derived from the data values and are expressed in one or more mathematical equations for describing and predicting a respective trend of one of the first and second fields that varies as a function of the other of the first and second fields; and

displaying at least one graphical representation of the one or more statistical models in the graphical view of the dataset.

2. The method of claim 1, wherein the first and second fields are partitioned into a plurality of sets, each set associated with a respective region of the view including row, column, page, and an encoding area.

3. The method of claim 2, wherein the data properties of each field include a field type selected from the group consisting of categorical and quantitative and a data type selected from the group consisting of integer, real, Boolean, and time.

4. The method of claim 3, further comprising generating the one or more statistical models in accordance with predefined heuristics for modeling a relationship between the first field and the second field wherein the predefined heuristics include:

selecting one of the two fields from the user description as an independent variable and the other of the two fields as a response variable of the one or more statistical models, and converting categorical fields in the user description into factors of the one or more statistical models.

5. The method of claim 4, wherein the predefined heuristics further comprise:

defining a first quantitative field on the row (or the column) as a response variable and a second field on the column (or the row) as an independent variable if the first quantitative field is the only quantitative field on the row or the column of the view and the second field is either a quantitative field or a categorical field having a quantitative interpretation; and

generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

6. The method of claim 4, wherein the predefined heuristics further comprise:

defining a first quantitative field on the row (or the column) as a response variable and a second quantitative field on the column (or the row) as an independent variable if the second quantitative field is a quantitative field from a dimension class; and

generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

7. The method of claim 4, wherein the predefined heuristics further comprise:

defining a first quantitative field on the row as a response variable and a second quantitative field on the column as an independent variable; and

generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

8. The method of claim 7, wherein the predefined heuristics further comprise:

repeating said defining and generating steps for each unique pair of first and second quantitative fields on the column and row.

9. The method of claim 4, wherein the predefined heuristics further comprise:

defining each categorical field on the row, the column and the page of the view as a categorical factor of the one or more statistical models.

10. The method of claim 4, wherein the predefined heuristics further comprise:

defining each categorical field in the encoding area as a categorical factor of the one or more statistical models if there are more than one data points corresponding to each value in the categorical field.

11. The method of claim 4, wherein the predefined heuristics further comprise:

disregarding each quantitative field in the encoding area of the view from the factors of the one or more statistical models.

12. The method of claim 1, wherein each of the one or more statistical models has a model type including linear, polynomial, exponential, logarithmic, and transcendental.

13. The method of claim 1, wherein the user description is generated by a user dragging and dropping the first and second fields into respective field containers of a graphical user interface.

14. The method of claim 1, wherein the user description is generated by a user selecting the first and second fields as fields of interest through a graphical user interface.

15. The method of claim 1, wherein the user description is generated by a user entering a text string including the first and second fields as fields of interest through a user interface.

16. The method of claim 1, wherein the one or more statistical models are derived in accordance with one or more mathematical theories selected from the group consisting of regression, clustering, summarization, dependency modeling, and classification.

17. A method of automatically generating models for a dataset, comprising:

at a computer system having one or more processors and memory storing programs executed by the one or more processors;

allowing a user to provide a user description by dragging and dropping fields into field containers of a graphical user interface to generate a graphical view of the dataset, the user description including at least two sets of fields associated with the graphical view of the dataset;

determining a set of data properties for each of the two sets of fields;

with no further user input, generating one or more statistical models based on the properties of the two sets of fields, data values associated with the two sets of fields, and a set of predefined heuristics for defining a relationship between a first field from one of the two sets of fields and a second field from the other of the two sets of fields, wherein the one or more statistical models are derived from the data values and are expressed in one or more mathematical equations for describing and predicting a respective trend of one of the first and second fields that varies as a function of the other of the first and second fields; and

displaying at least one graphical representation of the one or more statistical models in a specific region of the graphical user interface.

18. The method of claim 17, wherein each set of fields is associated with a respective region of the view including row, column, page, and an encoding area.

19. The method of claim 18, wherein each field includes a field type selected from the group consisting of categorical and quantitative and a data type selected from the group consisting of integer, real, Boolean, and time.

20. The method of claim 19, wherein the set of predefined heuristics includes selecting one of the two fields from the user description as an independent variable and the other of the two fields as a response variable of the one or more statistical models, and converting categorical fields in the user description into factors of the one or more statistical models.

21. The method of claim 20, wherein the set of predefined heuristics further comprises:

defining a first quantitative field on the row (or the column) as a response variable and a second field on the column (or the row) as an independent variable if the first quantitative field is the only quantitative field on the row or the column of the view and the second field is either a quantitative field or a categorical field having a quantitative interpretation; and

generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

22. The method of claim 20, wherein the set of predefined heuristics further comprises:

defining a first quantitative field on the row (or the column) as a response variable and a second quantitative field on the column (or the row) as an independent variable if the second quantitative field is a quantitative field from a dimension class; and

generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

23. The method of claim 20, wherein the set of predefined heuristics further comprises:

defining a first quantitative field on the row as a response variable and a second quantitative field on the column as an independent variable; and

generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

24. The method of claim 23, wherein the set of predefined heuristics further comprises:

repeating said defining and generating steps for each unique pair of first and second quantitative fields on the column and row.

25. The method of claim 20, wherein the set of predefined heuristics further comprises:

defining each categorical field on the row, the column and the page of the view as a categorical factor of the one or more statistical models.

26. The method of claim 20, wherein the set of predefined heuristics further comprises:

defining each categorical field in the encoding area as a categorical factor of the one or more statistical models if there are more than one data points corresponding to each value in the categorical field.

27. The method of claim 20, wherein the set of predefined heuristics further comprises:

disregarding each quantitative field in the encoding area of the view from the factors of the one or more statistical models.

28. The method of claim 17, wherein each of the one or more statistical models has a model type including linear, polynomial, exponential, logarithmic, and transcendental.

29. The method of claim 17, wherein the user description is generated by a user dragging and dropping a subset of the two sets of fields into respective field containers of a graphical user interface.

30. The method of claim 17, wherein the user description is generated by a user selecting a subset of the two sets of fields as fields of interest through a graphical user interface.

31. The method of claim 17, wherein the user description is generated by a user entering a text string including a subset of the two sets of fields as fields of interest through a user interface.

32. The method of claim 17, wherein the one or more statistical models are derived in accordance with one or more mathematical theories selected from the group consisting of regression, clustering, summarization, dependency modeling, and classification.

33. A computer system, comprising:

a main memory;

a processor; and

at least one program, stored in the main memory and executed by the processor, the at least one program further including;

instructions for receiving a user description to generate a graphical view of a dataset, wherein the user description specifies a graphical relationship between a first field and a second field of the dataset in the graphical view;

instructions for determining a set of data properties for each of the first and second fields;

instructions for generating one or more statistical models in accordance with the respective data properties of the first and second fields and data values that are associated with the first and second fields in the dataset with no further user input, wherein the one or more statistical models are derived from the data values and are expressed in one or more mathematical equations for describing and predicting a respective trend of one of the first and second fields that varies as a function of the other of the first and second fields; and

instructions for displaying at least one graphical representation of the one or more statistical models in the graphical view of the dataset.

34. The computer system of claim 33, wherein the first and second fields are partitioned into a plurality of sets, each set associated with a respective region of the view including row, column, page, and an encoding area.

35. The computer system of claim 34, wherein the data properties of each field include a field type selected from the group consisting of categorical and quantitative and a data type selected from the group consisting of integer, real, Boolean, and time.

36. The computer system of claim 35, further comprising instructions for generating the one or more statistical models in accordance with predefined heuristics for modeling a relationship between the first field and the second field wherein the predefined heuristics include instructions for selecting one of the two fields from the user description as an independent variable and the other of the two fields as a response variable of the one or more statistical models, and instructions for converting categorical fields in the user description into factors of the one or more statistical models.

37. The computer system of claim 36, wherein the predefined heuristics further comprise:

instructions for defining a first quantitative field on the row (or the column) as a response variable and a second field on the column (or the row) as an independent variable if the first quantitative field is the only quantitative field on the row or the column of the view and the second field is either a quantitative field or a categorical field having a quantitative interpretation; and

instructions for generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

38. The computer system of claim 36, wherein the predefined heuristics further comprise:

instructions for defining a first quantitative field on the row (or the column) as a response variable and a second quantitative field on the column (or the row) as an independent variable if the second quantitative field is a quantitative field from a dimension class; and

instructions for generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

39. The computer system of claim 36, wherein the predefined heuristics further comprise:

instructions for defining a first quantitative field on the row as a response variable and a second quantitative field on the column as an independent variable; and

instructions for generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

40. The computer system of claim 39, wherein the predefined heuristics further comprise:

instructions for repeating said defining and generating instructions for each unique pair of first and second quantitative fields on the column and row.

41. The computer system of claim 36, wherein the predefined heuristics further comprise:

instructions for defining each categorical field on the row, the column and the page of the view as a categorical factor of the one or more statistical models.

42. The computer system of claim 36, wherein the predefined heuristics further comprise:

instructions for defining each categorical field in the encoding area as a categorical factor of the one or more statistical models if there are more than one data points corresponding to each value in the categorical field.

43. The computer system of claim 36, further comprising:

instructions for disregarding each quantitative field in the encoding area of the view from the factors of the one or more statistical models.

44. The computer system of claim 33, wherein each of the one or more statistical models has a model type including linear, polynomial, exponential, logarithmic, and transcendental.

45. The computer system of claim 33, wherein the user description is generated by a user dragging and dropping the first and second fields into respective field containers of a graphical user interface.

46. The computer system of claim 33, wherein the user description is generated by a user selecting the first and second fields as fields of interest through a graphical user interface.

47. The computer system of claim 33, wherein the user description is generated by a user entering a text string including the first and second fields as fields of interest through a user interface.

48. The computer system of claim 33, wherein the one or more statistical models are derived in accordance with one or more mathematical theories selected from the group consisting of regression, clustering, summarization, dependency modeling, and classification.

49. A computer program product for use in conjunction with a computer system that includes one or more processors and memory, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising instructions that, if executed by the computer system, cause the computer system to:

receive a user description of a graphical view of a dataset, wherein the user description specifies a graphical relationship between a first field and a second field of the dataset in the graphical view;

determine a set of data properties for each of the first and second fields;

generate one or more statistical models of the dataset in accordance with the data properties of the first and second fields and data values associated with the first and second fields in the dataset with no further user input, wherein the one or more statistical models are derived from the data values and are expressed in one or more mathematical equations for describing and predicting a respective trend of one of the first and second fields that varies as a function of the other of the first and second fields; and

display at least one graphical representation of the one or more statistical models in the graphical view of the dataset.

50. The computer program product of claim 49, wherein the first and second fields are partitioned into a plurality of sets, each set associated with a respective region of the view including row, column, page, and an encoding area.

51. The computer program product of claim 50, wherein the data properties of each field include a field type selected from the group consisting of categorical and quantitative and a data type selected from the group consisting of integer, real, Boolean, and time.

52. The computer program product of claim 51, further comprising instructions for generating the one or more statistical models in accordance with predefined heuristics for modeling a relationship between the first field and the second field wherein the predefined heuristics include instructions for selecting one of the two fields from the user description as an independent variable and the other of the two fields as a response variable of the one or more statistical models, and instructions for converting categorical fields in the user description into factors of the one or more statistical models.

53. The computer program product of claim 52, wherein the predefined heuristics further comprise:

instructions for defining a first quantitative field on the row (or the column) as a response variable and a second field on the column (or the row) as an independent variable if the first quantitative field is the only quantitative field on the row or the column of the view and the second field is either a quantitative field or a categorical field having a quantitative interpretation; and

instructions for generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

54. The computer program product of claim 52, wherein the predefined heuristics further comprise:

instructions for defining a first quantitative field on the row (or the column) as a response variable and a second quantitative field on the column (or the row) as an independent variable if the second quantitative field is a quantitative field from a dimension class; and

instructions for generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

55. The computer program product of claim 52, wherein the predefined heuristics further comprise:

instructions for defining a first quantitative field on the row as a response variable and a second quantitative field on the column as an independent variable; and

instructions for generating the one or more statistical models using data in the dataset associated with the response variable and the independent variable.

56. The computer program product of claim 55, wherein the predefined heuristics further comprise:

instructions for repeating said defining and generating instructions for each unique pair of first and second quantitative fields on the column and row.

57. The computer program product of claim 52, wherein the predefined heuristics further comprise:

instructions for defining each categorical field on the row, the column and the page of the view as a categorical factor of the one or more statistical models.

58. The computer program product of claim 52, wherein the predefined heuristics further comprise:

instructions for defining each categorical field in the encoding area as a categorical factor of the one or more statistical models if there are more than one data points corresponding to each value in the categorical field.

59. The computer program product of claim 52, wherein the predefined heuristics further comprise:

instructions for disregarding each quantitative field in the encoding area of the view from the factors of the one or more statistical models.

60. The computer program product of claim 49, wherein each of the one or more statistical models has a model type including linear, polynomial, exponential, logarithmic, and transcendental.

61. The computer program product of claim 49, wherein the user description is generated by a user dragging and dropping the first and second fields into respective field containers of a graphical user interface.

62. The computer program product of claim 49, wherein the user description is generated by a user selecting the first and second fields as fields of interest through a graphical user interface.

63. The computer program product of claim 49, wherein the user description is generated by a user entering a text string including the first and second fields as fields of interest through a user interface.

64. The computer program product of claim 49, wherein the one or more statistical models are derived in accordance with one or more mathematical theories selected from the group consisting of regression, clustering, summarization, dependency modeling, and classification.

65. A computer system comprising:

a main memory;

a processor; and

at least one program, stored in the main memory and executed by the processor, the at least one program further including;

means for receiving a user description to generate a graphical view of a dataset, wherein the user description specifies a graphical relationship between a first field and a second field of the dataset in the graphical view

means for determining a set of data properties for each of the first and second fields; and

means for generating one or more statistical models in accordance with the respective data properties of the first and second fields and data values that are associated with the first and second fields in the dataset with no further user input, wherein the one or more statistical models are derived from the data values and are expressed in one or more mathematical equations for describing and predicting a respective trend of one of the first and second fields that varies as a function of the other of the first and second fields; and

means for displaying at least one graphical representation of the one or more statistical models in the graphical view of the dataset.

66. The computer system of claim 65, wherein the one or more statistical models are derived in accordance with one or more mathematical theories selected from the group consisting of regression, clustering, summarization, dependency modeling, and classification.