Systems, methods, and apparatuses for implementing cross-organizational processing of business intelligence metrics

• US 10,460,270 B2
• Filed: 09/11/2013
• Issued: 10/29/2019
• Est. Priority Date: 09/12/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Jain, Samarpan, Tamm, Steven
• Primary Examiner(s): Stamber, Eric W
• Assistant Examiner(s): Torrico-Lopez, Alan
• Application Number:
  US14/023,934
• Publication Number:
  US 20140074558A1
• Time in Patent Office: 2,239 Days
• Field of Search: None
• CPC Class Codes: G06Q 10/0637 Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals

Abstract

In accordance with disclosed embodiments, there are provided methods, systems, and apparatuses for implementing cross-organizational processing of business intelligence metrics including, for example, means for scheduling a time for metrics collection from databases of a host organization; generating one or more messages and associated parameters to implement the metrics collection at the host organization; enqueuing the one or more messages and associated parameters for processing by the host organization; dequeuing the one or more messages and associated parameters into the host organization for processing, wherein the processing is to output the metrics collection into an intermediate table; capturing the metrics collection output via the processing at the intermediate table; and exposing the intermediate table to metrics queries. Other related embodiments are disclosed.

Claims

1. A method comprising:

executing a multi-tenant database system within a host organization in support of multiple distinct customer organizations which utilize on-demand cloud based services provided by the host organization, wherein the multi-tenant database system includes processors and memory which are shared amongst the multiple distinct customer organizations;

receiving, at a request interface, incoming requests from any of the multiple distinct customer organizations, each operating as a tenant of the multi-tenant database system, and returning responses to the multiple distinct customer organizations responsive to the incoming requests received;

storing protected data of the multiple distinct customer organizations within one or more database systems, wherein the one or more database systems having the protected data stored therein are accessible via a database query interface of the host organization;

wherein each one of the multiple distinct customer organizations are prohibited from submitting database queries to the one or more database systems which exceed the scope of one single customer organization'"'"'s data corresponding to the respective customer organization'"'"'s tenancy within the multi-tenant database system;

generating, by the processors, a plurality of messages for each of a first and subsequent distinct customer organizations and processing against the one or more database systems to perform metrics collection for each distinct customer organization separately and releasing the plurality of messages for processing against the one or more database systems specifying a common repository for the metrics collection output, wherein the plurality of messages are generated based on an inference of scope and granularity required to perform the metrics collection;

wherein the metrics collection output is stored within the common repository as aggregation data in a common pool, with each distinct customer organization'"'"'s pooled data derived from the multi-tenant database and organized via a customer organization ID column identifying each respective customer organization;

wherein the aggregation data is abstracted from the protected data to eliminate exposure of any of the protected data of any of the multiple distinct customer organizations various customer organizations'"'"' underlying data records as stored within the one or more database systems; and

receiving, via a metrics API, a cross-organizational query metrics query requesting results to be returned with data having a scope greater than a single one of the multiple distinct customer organization despite the prohibition against cross-organizational database queries, wherein the one or more database systems having the multiple distinct customer organizations'"'"' protected data stored therein are not accessible via the metrics API;

processing the cross-organizational query metrics query against the common repository and returning metrics query results from the common repository having metrics associated with multiple of the distinct customer organizations according to the customer organization ID column; and

monitoring a workload of the host organization to prioritize real-time transaction requests over messages and associated parameters utilized to implement the metrics collection by releasing one or more of the plurality of messages for immediate processing when the workload of the host organization is below a threshold and processing the real-time transaction requests at the host organization concurrently with the released one or more messages and any associated parameters to implement the metrics collection.

2. The method of claim 1, further comprising:

scheduling a time for metrics collection from the one or more database systems;

releasing the plurality of messages for processing against the one or more database systems to perform metrics collection according to the time scheduled for metrics collection; and

capturing the metrics collection output as the aggregation data of the common repository.

3. The method of claim 1, wherein generating the plurality of messages comprises generating one or more messages and associated parameters to implement the metrics collection at the host organization, the metrics collection based on protected customer data stored within the one or more database systems.

4. The method of claim 1, further comprising:

triggering the metrics collection at a time previously scheduled, wherein triggering the metrics collection comprises at least (i) generating one or more messages and associated parameters to implement the metrics collection (ii) enqueuing the one or more messages and associated parameters for processing by the one or more database systems and releasing the generated one or more messages for processing against the one or more database systems to perform metrics collection for each tenant.

5. The method of claim 1, further comprising:

implementing metrics collection and generating the output based on selecting a granularity from one of plurality of granularities such as a per-user, a per-tenant, a per-organization, a per-node, per-partition, or a per-database granularity; and

scheduling the metrics collection utilizing the selected granularity and performing the metrics collection; and

processing metrics collection in parallel with any of the plurality of the granularities.

6. The method of claim 1:

wherein the output captured via the common repository is associated with the customer organization ID column identifying which one among a plurality of customer organizations of the host organization is associated with any metric, statistic, or metadata present within the metrics collection output captured by the common repository; and

wherein returning the metrics query results from the common repository comprises returning the metrics query results from the common repository having metrics, statistics, and/or metadata associated with multiple customer organizations organized according to the customer organization ID column.

7. The method of claim 1:

wherein users of the host organization are restricted to metrics queries having a query scope that is limited to a single individual customer organization to which the user belongs; and

wherein system administrators of the host organization are enabled to perform metrics queries having a query scope that exceeds a single individual customer organization via a restricted administrative interface for the system administrators which submits metrics queries against the common repository via the metrics API.

8. The method of claim 1:

wherein the protected data include at least one of HIPAA (Health Insurance Portability and Accountability Act) protected data;

Sarbanes-Oxley Act (SOX) protected data;

proprietary sales data;

proprietary financial data;

proprietary trade-secret data; and

government classified data; and

wherein the aggregation data is void of the protected data and includes metadata, metrics, and statistics.

9. The method of claim 1, further comprising:

scheduling a time for metrics collection by scheduling a metrics collection job at a specified periodicity; and

wherein processing the metrics collection against the one or more database systems comprises iterating the metrics collection at the specified periodicity selected from one of a daily periodicity, a weekly periodicity, a bi-weekly periodicity, a monthly periodicity, a quarterly periodicity, and a yearly periodicity.

10. Non-transitory computer readable storage media having instructions stored thereon that, when executed by a processor of a system, the instructions cause the system to perform operations comprising:

executing a multi-tenant database system within a host organization in support of multiple distinct customer organizations which utilize on-demand cloud based services provided by the host organization, wherein the multi-tenant database system includes processors and memory which are shared amongst the multiple distinct customer organizations;

receiving, at a request interface, incoming requests from any of the multiple distinct customer organizations, each operating as a tenant of the multi-tenant database system, and returning responses to the multiple distinct customer organizations responsive to the incoming requests received;

storing protected data of the multiple distinct customer organizations within one or more database systems, wherein the one or more database systems having the protected data stored therein are accessible via a database query interface of the host organization;

wherein each one of the multiple distinct customer organizations are prohibited from submitting database queries to the one or more database systems which exceed the scope of one single customer organization'"'"'s data corresponding to the respective customer organization'"'"'s tenancy within the multi-tenant database system;

generating, by the processors, a plurality of messages for each of a first and subsequent distinct customer organizations and processing against the one or more database systems to perform metrics collection for each distinct customer organization separately and releasing the plurality of messages for processing against the one or more database systems specifying a common repository for the metrics collection output, wherein the plurality of messages are generated based on an inference of scope and granularity required to perform the metrics collection;

wherein the metrics collection output is stored within the common repository as aggregation data in a common pool, with each distinct customer organization'"'"'s pooled data derived from the multi-tenant database and organized via a customer organization ID column identifying each respective customer organization;

wherein the aggregation data is abstracted from the protected data to eliminate exposure of any of the protected data of any of the multiple distinct customer organizations various customer organizations'"'"' underlying data records as stored within the one or more database systems; and

receiving, via a metrics API, a cross-organizational query metrics query requesting results to be returned with data having a scope greater than a single one of the multiple distinct customer organization despite the prohibition against cross-organizational database queries, wherein the one or more database systems having the multiple distinct customer organizations'"'"' protected data stored therein are not accessible via the metrics API;

processing the cross-organizational query metrics query against the common repository and returning metrics query results from the common repository having metrics associated with multiple of the distinct customer organizations according to the customer organization ID column; and

monitoring a workload of the host organization to prioritize real-time transaction requests over messages and associated parameters utilized to implement the metrics collection by releasing one or more of the plurality of messages for immediate processing when the workload of the host organization is below a threshold and processing the real-time transaction requests at the host organization concurrently with the released one or more messages and any associated parameters to implement the metrics collection.

11. The non-transitory computer readable storage media of claim 10:

wherein the output captured via the common repository is associated with the customer organization ID column identifying which one among a plurality of customer organizations of the host organization is associated with any metric, statistic, or metadata present within the metrics collection output captured by the common repository; and

wherein returning the metrics query results from the common repository comprises returning the metrics query results from the common repository having metrics, statistics, and/or metadata associated with multiple customer organizations organized according to the customer organization ID column.

12. The non-transitory computer readable storage media of claim 10, wherein the instructions cause the system to perform operations further comprising:

scheduling a time for metrics collection by scheduling a metrics collection job at a specified periodicity; and

wherein processing the metrics collection against the one or more database systems comprises iterating the metrics collection at the specified periodicity selected from one of a daily periodicity, a weekly periodicity, a bi-weekly periodicity, a monthly periodicity, a quarterly periodicity, and a yearly periodicity.

13. The non-transitory computer readable storage media of claim 10:

wherein users of the host organization are restricted to metrics queries having a query scope that is limited to a single individual customer organization to which the user belongs; and

wherein system administrators of the host organization are enabled to perform metrics queries having a query scope that exceeds a single individual customer organization via a restricted administrative interface for the system administrators which submits metrics queries against the common repository via the metrics API.

14. The non-transitory computer readable storage media of claim 10, wherein the instructions cause the system to perform operations further comprising:

scheduling one or more messages and associated parameters to implement metrics collection via a bulk processing database call; and

capturing the metrics collection output into the common repository by specifying a BULK COLLECT INTO call for the processing and specifying the common repository as a target repository for the metrics collection output.

15. The non-transitory computer readable storage media of claim 10:

wherein the protected data include at least one of HIPAA (Health Insurance Portability and Accountability Act) protected data;

Sarbanes-Oxley Act (SOX) protected data;

proprietary sales data;

proprietary financial data;

proprietary trade-secret data; and

government classified data; and

wherein the aggregation data is void of the protected data and includes metadata, metrics, and statistics.

16. A system comprising:

a multi-tenant database system to execute within a host organization in support of multiple distinct customer organizations which utilize on-demand cloud based services provided by the host organization, wherein the multi-tenant database system includes processors and memory which are shared amongst the multiple distinct customer organizations;

a request interface to receive incoming requests from any of the multiple distinct customer organizations, each operating as a tenant of the multi-tenant database system, and returning responses to the multiple distinct customer organizations responsive to the incoming requests received;

one or more databases for storing protected data of the multiple distinct customer organizations within one or more database systems, wherein the one or more database systems having the protected data stored therein are accessible via a database query interface of the host organization;

wherein each one of the multiple distinct customer organizations are prohibited from submitting database queries to the one or more database systems which exceed the scope of one single customer organization'"'"'s data corresponding to the respective customer organization'"'"'s tenancy within the multi-tenant database system;

a scheduler module for generating a plurality of messages for each of a first and subsequent distinct customer organizations and processing against the one or more database systems to perform metrics collection for each distinct customer organization separately and releasing the plurality of messages for processing against the one or more database systems specifying a common repository for the metrics collection output, wherein the plurality of messages are generated based on an inference of scope and granularity required to perform the metrics collection;

wherein the metrics collection output is stored within the common repository as aggregation data in a common pool, with each distinct customer organization'"'"'s pooled data derived from the multi-tenant database and organized via a customer organization ID column identifying each respective customer organization;

wherein the aggregation data is abstracted from the protected data to eliminate exposure of any of the protected data of any of the multiple distinct customer organizations various customer organizations'"'"' underlying data records as stored within the one or more database systems; and

a metrics Application Programming Interface (API) for receiving, via a metrics API, a cross-organizational query metrics query requesting results to be returned with data having a scope greater than a single one of the multiple distinct customer organization despite the prohibition against cross-organizational database queries, wherein the one or more database systems having the multiple distinct customer organizations'"'"' protected data stored therein are not accessible via the metrics API;

a message handler for processing the cross-organizational query metrics query against the common repository and returning metrics query results from the common repository having metrics associated with multiple of the distinct customer organizations according to the customer organization ID column; and

wherein the system is to monitor a workload of the host organization to prioritize real-time transaction requests over messages and associated parameters utilized to implement the metrics collection by releasing one or more of the plurality of messages for immediate processing when the workload of the host organization is below a threshold and processing the real-time transaction requests at the host organization concurrently with the released one or more messages and any associated parameters to implement the metrics collection.

17. The system of claim 16:

wherein the scheduler is to further schedule a time for metrics collection from the one or more database systems;

wherein the scheduler is to further release the plurality of messages for processing against the one or more database systems to perform metrics collection according to the time scheduled for metrics collection; and

wherein the metrics API is to further capture the metrics collection output as the aggregation data of the common repository.

18. The system of claim 16:

wherein the scheduler is to further generate one or more messages and associated parameters to implement the metrics collection at the host organization, the metrics collection based on protected customer data stored within the one or more database systems.

19. The system of claim 16:

wherein the scheduler is to further trigger the metrics collection at a time previously scheduled, by (i) generating one or more messages and associated parameters to implement the metrics collection (ii) enqueuing the one or more messages and associated parameters for processing by the one or more database systems and releasing the generated one or more messages for processing against the one or more database systems to perform metrics collection for each tenant.

20. The system of claim 16:

wherein the metrics API is to further determine a granularity and scope required to implement metrics collection and generate the output, the granularity being selected from one of a plurality of granularities such as a per-user, a per-tenant, a per-organization, a per-node, per-partition, or a per-database granularity; and

wherein the scheduler is to further process the metrics collection in parallel with any of the plurality of the granularities.

21. The system of claim 16:

wherein the output captured via the common repository is associated with the customer organization ID column identifying which one among a plurality of customer organizations of the host organization is associated with any metric, statistic, or metadata present within the metrics collection output captured by the common repository; and

wherein the message handler is to further return the metrics query results from the common repository having metrics, statistics, and/or metadata associated with multiple customer organizations organized according to the customer organization ID column.

22. The system of claim 16:

wherein users of the host organization are restricted to metrics queries having a query scope that is limited to a single individual customer organization to which the user belongs; and

wherein system administrators of the host organization are enabled to perform metrics queries having a query scope that exceeds a single individual customer organization via a restricted administrative interface for the system administrators which submits metrics queries against the common repository via the metrics API.

23. The system of claim 16:

wherein the scheduler is to further schedule a metrics collection job at a specified periodicity and iterate the metrics collection at the specified periodicity selected from one of a daily periodicity, a weekly periodicity, a bi-weekly periodicity, a monthly periodicity, a quarterly periodicity, and a yearly periodicity.

24. The system of claim 16:

wherein the scheduler is to further implement metrics collection via a bulk processing database call and capture the metrics collection output into the common repository by specifying a BULK COLLECT INTO call for the processing and specifying the common repository as a target repository for the metrics collection output.

25. The system of claim 16:

wherein the protected data include at least one of HIPAA (Health Insurance Portability and Accountability Act) protected data;

Sarbanes-Oxley Act (SOX) protected data;

proprietary sales data;

proprietary financial data;

proprietary trade-secret data; and

government classified data; and

wherein the aggregation data is void of the protected data and includes metadata, metrics, and statistics.