AGGREGATING DATA FROM A WORK QUEUE

• US 20110179028A1
• Filed: 01/15/2010
• Published: 07/21/2011
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Viswanathan, Rajesh, Copeland, Bruce, Mallet, Alexander, Rickman, Christopher, Ness, Andrew
• Granted Patent: US 8,645,377 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

One or more techniques and/or systems are disclosed herein for aggregating web-based data stored in a distributed data store so that it can be retrieved in a first-in, first-out (FIFO) manner. A unique aggregation key is generated for respective one or more data generated from a web-based event, where the one or more data are added to the distributed data store, and the aggregation key corresponds merely to the data generated from the web-based event. The one or more data from the web based event is aggregated in a FIFO queue and stored in a same partition of the distributed data store, based on the aggregation key.

Claims

1. A computer-based method for aggregating web-based data stored in a distributed data store so that it can be retrieved in a first-in, first-out (FIFO) manner, comprising:

generating an aggregation key for respective one or more data generated from a web-based event, using a microprocessor, where the one or more data are added to the distributed data store, the aggregation key corresponding merely to the data generated from the web-based event; and

aggregating the one or more data from the web based event in a FIFO queue in a same partition of the distributed data store based on the aggregation key.

2. The method of claim 1, comprising:

designating a first partition of the distributed data store to store data associated with a first aggregation key; and

designating a second partition of the distributed data store to store data associated with a second aggregation key

3. The method of claim 2, the generating an aggregation key comprising:

generating a first aggregation key for one or more first data generated from a first web-based event, using a microprocessor, where the first aggregation key is merely associated with the first data, and the first data are added to the distributed data store; and

generating a second aggregation key for one or more second data generated from a second web-based event, using a microprocessor, where the second aggregation key is merely associated with the second data, and the second data are added to the distributed data store.

4. The method of claim 1, comprising attaching the aggregation key to the data.

5. The method of claim 1, providing for data having a same aggregation key to be retrieved upon request from the distributed data store in a FIFO manner.

6. The method of claim 1, the distributed data store comprising a plurality of data storage devices that provide for related web-event generated data to be stored on different devices.

7. The method of claim 1, the aggregation key comprising one of:

a randomly generated string;

a hash generated by hashing the data; and

a sequentially generated string.

8. The method of claim 1, generating the aggregation key and aggregating the data using a service-oriented application, running on the one or more computers, built from an application programming interface (API) in a software development framework.

9. The method of claim 8, comprising:

a first web-based service remotely connected to the service-oriented application providing the data to be stored from the web-based event; and

a second web-based service remotely connected to the service-oriented application retrieving the stored data in a FIFO manner.

10. The method of claim 8, the service-oriented application running on a first set of one or more devices and the distributed data store disposed on a second set of one or more devices.

11. The method of claim 1, comprising performing operations on the distributed data store, comprising one or more of:

an add operation comprising storing data to the partition of the distributed operation in the FIFO queue; and

a take operation comprising retrieving data from the FIFO queue of the partition in the distributed data store.

12. A system for aggregating web-based data stored in a distributed data store for retrieval in a first-in, first-out (FIFO) manner, comprising:

a web-based work queue service component operably coupled with a distributed data store comprising a plurality of data storage devices, and configured to store and retrieve, in a FIFO manner, web-based data on the distributed data store, the work queue service component comprising;

an aggregation key generation component configured to generate an aggregation key corresponding merely to data generated from a same web-based event, where the data is added to the distributed data store; and

a queue storing component configured to aggregate the data in a FIFO queue in a same partition of the distributed data store based on the aggregation key.

13. The system of claim 12, the web-based work queue service component remotely coupled with one or more services configured to run a web-based event that performs one or more of:

generates web-based data for storage in a FIFO queue of a partition in the distributed data store; and

retrieves stored data from the FIFO queue in a FIFO manner.

14. The system of claim 13, the web-based work queue service using a SOAP interface to communicate with the one or more services.

15. The system of claim 12, the web-based work queue service component comprising an aggregation key attachment component configured to attach the aggregation key to respective web-based data from a same web-based event.

16. The method of claim 12, the aggregation key comprising one of:

a string generated for the web-based event that corresponds merely to the web-based event; and

a hash generated by hashing the data from the web-based event.

17. The system of claim 12, the web-based work queue service component comprising a data retrieval component configured to provide data to a web-based service upon request in a FIFO manner from the partitioned queue on the distributed data store.

18. A computer-based method for aggregating web-based data over a distributed data store so that it can be retrieved in a first-in, first-out (FIFO) manner, comprising:

designating a partition of the distributed data store to store web event generated data associated with a first aggregation key;

designating a partition of the distributed data store to store web event generated data associated with a second aggregation key;

generating the first aggregation key for one or more first data provided by a remotely connected first service running a web-based event, using a microprocessor, where the first aggregation key is merely associated with the first data, and the first data are added to the distributed data store;

generating the second aggregation key for one or more second data provided by a remotely connected second web-based service running a second web-based event, using a microprocessor, where the second aggregation key is merely associated with the second data, and the second data are added to the distributed data store.

attaching the first aggregation key to the respective one or more first data;

attaching the second aggregation key to the respective one or more second data;

storing the first data having the first aggregation key in a first FIFO queue in the first partition of the distributed web-based data store, where the distributed data store comprises a plurality of data storage devices that provide for related data to be stored on different devices;

storing the second data having the second aggregation key in a second FIFO queue in the second partition of the distributed web-based data store; and

providing for data having a same aggregation key to be retrieved, upon request by a remotely connected third service, from the distributed data store at a same time in a FIFO manner.

19. The method of claim 18, performed using a SOAP interface for a service-oriented application, running on the one or more computers, built from an application programming interface (API) in a software development framework utilized in a web-based environment.

20. The method of claim 18, comprising performing operations on the distributed data store, comprising one or more of:

an add operation comprising storing data to the partition of the distributed operation in the FIFO queue; and

a take operation comprising retrieving data from the FIFO queue of the partition in the distributed data store.