DATA SURFACING CONTROL FRAMEWORK

• US 20160321320A1
• Filed: 04/28/2015
• Published: 11/03/2016
• Est. Priority Date: 04/28/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Gunderson, Karl N.
• Granted Patent: US 10,229,159 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution

Abstract

A parallel processing framework comprises, in one example, a data gathering system configured to receive a set of data requests, each corresponding to at least one structured data representation (SDR). The data gathering system is configured to obtain a data set for each of the data requests. The parallel processing framework comprises an SDR generation system configured to receive indications of the plurality of data requests and, for each data request, an indication as to whether the data set corresponding to the data request has been obtained by the data gathering system. The SDR generation system is configured to generate the plurality of SDRs based on the data sets obtained by the data gathering system.

Claims

1. A parallel processing framework for generating structured data representations (SDRs) in a computing system, the parallel processing framework comprising:

a data gathering system configured to receive a set of data requests, each corresponding to at least one structured data representation (SDR), wherein the data gathering system is configured to obtain a data set for each of the data requests; and

an SDR generation system configured to receive indications of the plurality of data requests and, for each data request, an indication as to whether the data set corresponding to the data request has been obtained by the data gathering system, wherein the SDR generation system is configured to generate the plurality of SDRs based on the data sets obtained by the data gathering system.

2. The parallel processing framework of claim 1, wherein the SDR generation system performs an SDR generation process to generate an SDR for a one of the data requests while the data gathering system performs a data gathering process to obtain the data set for another one of the data requests.

3. The parallel processing framework of claim 1, wherein the plurality of SDRs comprises a plurality of reports that are generated to include the data sets.

4. The parallel processing framework of claim 1, wherein the data gathering system receives the set of data requests from a controller component, each data request including data request parameters that are used by the data gathering system to obtain the data set from a data store.

5. The parallel processing framework of claim 1, wherein the indications received by the SDR generation system comprise a plurality of SDR generation requests, each SDR generation request identifying a destination for a corresponding SDR and a location of the data set for the corresponding SDR.

6. The parallel processing framework of claim 1, wherein the data gathering system stores the set of data requests in a queue.

7. The parallel processing framework of claim 6, wherein the data gathering system is configured to perform a batch processing task on a plurality of the data requests in the queue.

8. The parallel processing framework of claim 1, wherein the data gathering system is configured to send a signal to the SDR generation system based on obtaining a data set for a data request.

9. The parallel processing framework of claim 8, wherein the data sets comprise a plurality of data tables, each data table being generated for one or more of the SDRs.

10. The parallel processing framework of claim 1, and further comprising a request status store that stores a plurality of request entries, each entry representing a given one of the data requests and identifying whether the data set for the given data request has been obtained by the data gathering system.

11. The parallel processing framework of claim 10, wherein the SDR generation system updates the request status store based on the signal received from the data gathering system.

12. The parallel processing framework of claim 11, wherein the request status store comprises a request table that is generated by the SDR generation system.

13. The parallel processing framework of claim 10, wherein the request status store indicates a sequential order of the set of data requests, and each entry in the request status store identifies a destination for the SDR corresponding to the given data request.

14. The parallel processing framework of claim 13, wherein the SDR generation system is configured to generate the plurality of SDRs based on the sequential order of the set of data requests and a destination type for each SDR.

15. The parallel processing framework of claim 14, wherein the SDR generation system is configured to generate the plurality of SDRs based on whether the destinations are order dependent destinations.

16. A computing system for generating structured data representations (SDRs), the computing system comprising:

a controller component configured to receive an SDR generation input for generating a set of SDRs and to obtain, based on the SDR generation input, a set of SDR generation requests, each SDR generation request having at least one corresponding data request with parameters that define data to be obtained for an SDR;

a data gathering system configured to obtain a data set for each data request; and

an SDR generation system configured to perform an SDR generation process for one of the SDR generation requests while the data gathering system performs a data gathering process for another one of the SDR generation requests.

17. The computer system of claim 16, comprising a request status store that stores a plurality of request entries, each entry representing at least one of the SDR generation requests and identifying whether a data set for the at least one SDR generation request has been obtained by the data gathering system.

18. The computer system of claim 17, wherein the SDR generation system performs the SDR generation process based on the request status store.

19. A computer-implemented method comprising:

receiving a set of structured data representation (SDR) requests, each request identifying data parameters for a corresponding SDR;

generating a set of SDRs using a parallel processing framework that simultaneously performs a data gathering process for the set of SDR requests and an SDR generation process for the set of SDR requests; and

outputting the set of SDRs

20. The computer-implemented method of claim 19, wherein the set of SDR requests define an order for the corresponding SDRs, and wherein generating the set of SDRs comprises:

obtaining data sets for the set of SDR requests in an order that is different than the defined order for the corresponding SDRs; and

performing the SDR generation process to generate the corresponding SDRs based on the defined order.