Guided data exploration

• US 10,552,484 B2
• Filed: 05/08/2015
• Issued: 02/04/2020
• Est. Priority Date: 09/24/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Sheffer, Uri, Pocock, Adam Craig, Stevens, Brook, Ishaque, Mashhood, Zelevinsky, Vladimir, Spaulding, Tristan R.
• Primary Examiner(s): Mahmoudi, Tony
• Assistant Examiner(s): Dang, Thai V
• Application Number:
  US14/707,283
• Publication Number:
  US 20160085880A1
• Time in Patent Office: 1,733 Days
• Field of Search: 707741-744
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/901 Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957

Abstract

A system for exploring data receives the data from a database and indexes the data in a server. The system displays one or more selectable datasets from the indexed data, where the selectable datasets include a plurality of attributes. The system receives a selection of one of the plurality of attributes. The system then sorts the one or more attributes by level of interestingness relative to the selected attribute, and displays the sorted attributes.

Claims

1. A method of exploring data, the method comprising:

receiving the data from a database;

indexing the data in a server to generate a plurality of database datasets;

displaying one or more selectable database datasets of the plurality database datasets from the indexed data, each of the selectable database datasets comprising a plurality of database data records and a plurality of attributes that correspond to columns of the database dataset, wherein data records of a given database dataset comprise values for one or more attributes of the given database dataset;

receiving a selection of one of the selectable database datasets that corresponds to a first plurality of attributes;

receiving a selection of one of the first plurality of attributes;

in response to a selection of the attribute, determining bivariate entropy metrics for a remaining set of the first plurality of attributes relative to the selected attribute, wherein the bivariate entropy metric for a given one of the remaining set of the first plurality of attributes is a function of joint probabilities calculated over different outcomes for the selected attribute and different outcomes for the (liven attribute within the data records of the selected data set, given attribute probabilities calculated over different outcomes of the given attribute within the data records of the selected data set, and selected attribute probabilities calculated over different outcomes of the selected attribute within the data records of the selected data set;

sorting the remaining set of the first plurality of attributes that correspond to the selected dataset based on the determined bivariate entropy metrics; and

displaying the sorted attributes of the selected dataset as selectable attributes.

2. The method of claim 1, further comprising receiving a dynamic modification of the sorted attributes using user interface elements to exclude or promote sorted attributes.

3. The method of claim 1, wherein at least some of the data in the database is unstructured data.

4. The method of claim 1, wherein the displaying the sorted attributes comprises displaying a tile for each sorted attribute, the tile comprising a visualization of the attribute.

5. The method of claim 1, wherein the determining the bivariate entropy metric for the remaining set of the first plurality of attributes relative to the selected attribute comprises determining mutual information between each of the remaining set of the first plurality of attributes and the selected attribute.

6. The method of claim 1, wherein the determining the bivariate entropy metric for the remaining set of the first plurality of attributes relative to the selected attribute comprises determining conditional mutual information between each of the remaining set of the first plurality of attributes and the selected attribute.

7. The method of claim 1, wherein each of the attributes consists of a database column and each of the database datasets corresponds to a database table.

8. The method of claim 1, wherein

at least one of the remaining set of the first plurality of attributes comprises a numerical attribute with numerical values within the data records of the selected data set, and

when determining the bivariate entropy metric for the one of the remaining set of the first plurality of attributes, the numerical values are discretized into bins to covert the numerical attribute into a categorical attribute.

9. The method of claim 1, further comprising:

mapping the determined bivariate entropy metrics for each of the remaining set of the first plurality of attributes to a plurality of sorting metrics by applying a high value cutoff and a low value cutoff to the determined bivariate entropy metrics, the sorting metrics comprising values between the high value cutoff and the low value cutoff, wherein the sorting of the remaining set of the first plurality attributes is based on the mapped sorting metrics.

10. The method of claim 1, wherein

the function of the joint probabilities, given attribute probabilities, and selected attribute probabilities that is used to determine the bivariate entropy metric for the given attribute comprises a summation function that includes the joint probabilities, given attribute probabilities, and selected attribute probabilities, and

the summation function comprises a double summation over possible outcomes for the given attribute and the selected attribute within the data records of the selected data set.

11. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to provided guided data exploration, the providing comprising:

receiving the data from a database;

indexing the data in a server to generate a plurality of database datasets;

displaying one or more selectable database datasets of the plurality database datasets from the indexed data, each of the selectable database datasets comprising a plurality of database data records and a plurality of attributes that correspond to columns of the database dataset, wherein data records of a given database dataset comprise values for one or more attributes of the given database dataset;

receiving a selection of one of the selectable database datasets that corresponds to a first plurality of attributes;

receiving a selection of one of the first plurality of attributes;

in response to a selection of the attribute, determining bivariate entropy metrics for the first plurality of attributes relative to the selected attribute, wherein the bivariate entropy metric for a given one of the remaining set of the first plurality of attributes is a function of joint probabilities calculated over different outcomes for the selected attribute and different outcomes for the given attribute within the data records of the selected data set, given attribute probabilities calculated over different outcomes of the given attribute within the data records of the selected data set, and selected attribute probabilities calculated over different outcomes of the selected attribute within the data records of the selected data set;

sorting the remaining set of the first plurality of attributes that correspond to the selected dataset based on the determined bivariate entropy metrics; and

displaying the sorted attributes of the selected dataset as selectable attributes.

12. The computer readable medium of claim 11, the providing further comprising receiving a dynamic modification of the sorted attributes using user interface elements to exclude or promote sorted attributes.

13. The computer readable medium of claim 11, wherein at least some of the data in the database is unstructured data.

14. The computer readable medium of claim 11, wherein the displaying the sorted attributes comprises displaying a tile for each sorted attribute, the tile comprising a visualization of the attribute.

15. The computer readable medium of claim 11, wherein the determining the bivariate entropy metric for the remaining set of the first plurality of attributes relative to the selected attribute comprises determining mutual information between each of the remaining set of the first plurality of attributes and the selected attribute.

16. The computer readable medium of claim 11, wherein the determining the bivariate entropy metric for the remaining set of the first plurality of attributes relative to the selected attribute comprises determining conditional mutual information between each of the remaining set of the first plurality of attributes and the selected attribute.

17. The computer readable medium of claim 11, wherein each of the attributes consists of a database column and each of the database datasets corresponds to a database table.

18. A guided data exploration system comprising:

a processor coupled to a storage device that stores instructions, the processor executing instructions to generate modules comprising;

an indexing module that receives data from a database and indexes the data in a server to generate a plurality of database datasets;

a display module that displays one or more selectable database datasets of the plurality database datasets from the indexed data, each of the selectable database datasets comprising a plurality of database data records and a plurality of attributes that correspond to columns of the database dataset, that receives a selection of one of the selectable database datasets that corresponds to a first plurality of attributes and that receives a selection of one of the first plurality of attributes, wherein data records of a given database dataset comprise values for one or more attributes of the given database dataset; and

a determining module that, in response to a selection of the attribute, determines bivariate entropy metrics for a remaining set of the first plurality of attributes relative to the selected attribute, wherein the bivariate entropy metric for a given one of the remaining set of the first plurality of attributes is a function of joint probabilities calculated over different outcomes for the selected attribute and different outcomes for the given attribute within the data records of the selected data set, given attribute probabilities calculated over different outcomes of the given attribute within the data records of the selected data set, and selected attribute probabilities calculated over different outcomes of the selected attribute within the data records of the selected data set;

a sorting module that sorts the remaining set of the first plurality of attributes that correspond to the selected dataset based on the determined bivariate entropy metrics;

wherein the display module further displays the sorted attributes of the selected dataset as selectable attributes.

19. The system of claim 18, wherein the determining the bivariate entropy metric for the remaining set of the first plurality of attributes relative to the selected attribute comprises determining mutual information between each of the remaining set of the first plurality of attributes and the selected attribute.

20. The system of claim 18, wherein the determining the bivariate entropy metric for the remaining set of the first plurality of attributes relative to the selected attribute comprises determining conditional mutual information between each of the remaining set of the first plurality of attributes and the selected attribute.