SYSTEM AND METHOD FOR PROVIDING BIG DATA ANALYTICS ON DYNAMICALLY-CHANGING DATA MODELS

• US 20150213109A1
• Filed: 04/02/2015
• Published: 07/30/2015
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Datorama Technologies Ltd. (Salesforce.com, Inc.)
• Original Assignee: Datorama Technologies Ltd. (Salesforce.com, Inc.)
• Inventors: Kassko, Efraeim Cohen, Raz, Raanan
• Granted Patent: US 10,754,877 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof
• CPC Class Codes: G06F 16/24535 of sub-queries or views
• CPC Class Codes: G06F 16/2456 Join operations
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06Q 30/0243 Comparative campaigns

Abstract

A system and method for providing big data analytics responsive to dynamically-changing data models are provided. The method includes parsing, based on a logical data schema, a user query into a plurality of sub-queries; sending the plurality of sub-queries to a plurality of data stores, wherein each data store is selected based on a physical data schema of a dynamic data schema; receiving a plurality of sub-result datasets, wherein each sub-result dataset corresponds to a sub-query; and combining the plurality of sub-result datasets into a single resulting data set based on a logical schema of the dynamic data schema, wherein the combining includes at least any of: an aggregation operation and a join operation.

Claims

1. A method for providing big data analytics responsive to dynamically-changing data models, comprising:

parsing, based on a logical data schema, a user query into a plurality of sub-queries;

sending the plurality of sub-queries to a plurality of data stores, wherein each data store is selected based on a physical data schema of a dynamic data schema;

receiving a plurality of sub-result datasets, wherein each sub-result dataset corresponds to a sub-query; and

combining the plurality of sub-result datasets into a single resulting data set based on a logical schema of the dynamic data schema, wherein the combining includes at least any of;

an aggregation operation and a join operation.

2. The method of claim 1, further comprising:

calculating, based on the single resulting data set, at least one metric dimension; and

determining, based on the at least one metric dimension, at least one data dimension.

3. The method of claim 1, wherein combining the plurality of sub-result datasets further comprises:

determining if any of the sub-result datasets are associated with the same dimension; and

for each group of at least two sub-result datasets that are determined to be associated with the same dimension, aggregating the at least two sub-result datasets into an aggregate dataset.

4. The method of claim 3, wherein combining the plurality of sub-result datasets further comprises:

determining whether each sub-result dataset is aggregated; and

joining the aggregate datasets and the sub-result datasets that are not aggregated into a joint dataset.

5. The method of claim 1, further comprising:

determining if data values in each sub-result dataset are common data dimensions; and

normalizing each common data dimension into a unified notation, further comprising transforming data values in each normalized dataset based on the logical schema.

6. The method of claim 5, wherein transforming data values in each normalized dataset based on the logical schema is based on a plurality of transformation rules.

7. The method of claim 6, wherein each transformation rule defines an action to be performed on the data values, wherein each action is selected from the group consisting of:

altering the data values, classifying the data values, and segmenting the data values.

8. The method of claim 1, wherein the dynamic data schema is user-specific.

9. The method of claim 1, wherein parsing, based on the logical data schema, the user query into the plurality of sub-queries further comprises:

adding, to each sub-query, at least one dimension or at least one granularity.

10. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 1.

11. A system for providing big data analytics responsive to dynamically-changing data models, comprising:

a data mart having an analytical processor;

a processor; and

a memory, the memory containing instructions that, when executed by the processor, configure the system to;

parse, based on a logical data schema, a user query into a plurality of sub-queries;

send the plurality of sub-queries to a plurality of data stores, wherein each data store is selected based on a physical data schema of a dynamic data schema;

receive a plurality of sub-result datasets, wherein each sub-result dataset corresponds to a sub-query; and

combine the plurality of sub-result datasets into a single resulting data set based on a logical schema of the dynamic data schema, wherein the combining includes at least any of;

an aggregation operation, and a join operation.

12. The system of claim 11, wherein the system is further configured to:

calculate, based on the single resulting data set, at least one metric dimension; and

determine, based on the at least one metric dimension, at least one data dimension.

13. The system of claim 12, wherein the system is further configured to:

determine if any of the sub-result datasets are associated with the same dimension; and

for each group of at least two sub-result datasets that are determined to be associated with the same dimension, aggregate the at least two sub-result datasets into an aggregate dataset.

14. The system of claim 13, wherein the system is further configured to:

determine whether each sub-result dataset is aggregated; and

join the aggregate datasets and the sub-result datasets that are not aggregated into a joint dataset.

15. The system of claim 11, wherein the system is further configured to:

determine if data values in each sub-result dataset are common data dimensions; and

normalize each common data dimension into a unified notation, further comprising transforming data values in each normalized dataset based on the logical schema.

16. The system of claim 15, wherein the transformation of data values in each normalized dataset based on the logical schema is based on a plurality of transformation rules.

17. The system of claim 16, wherein each transformation rule defines an action to be performed on the data values, wherein each action is selected from the group consisting of:

altering the data values, classifying the data values, and segmenting the data values.

18. The system of claim 11, wherein the dynamic data schema is user-specific.

19. The system of claim 11, wherein the system is further configured to:

add, to each sub-query, at least one dimension or at least one granularity.