Scalable data feed system

• US 9,596,127 B2
• Filed: 02/20/2012
• Issued: 03/14/2017
• Est. Priority Date: 02/20/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bheemarajaiah, Anand, Garg, Manas, Prabhu, Sandeep, Srinivasan, Krishnan
• Primary Examiner(s): LAZARO, DAVID R
• Application Number:
  US13/400,555
• Publication Number:
  US 20130219009A1
• Time in Patent Office: 1,849 Days
• Field of Search: 709/202, 709/203, 709/217, 709/218
• US Class Current: 1/1
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A scalable data feed system is disclosed. The data feed system may be offered as a cloud service that can serve many enterprises or tenants that require data to be pulled from information sources such as FTP, POP3, databases, line of business systems, a topic subscription, or an RSS feed, and pushed the data to information sinks, such as SMTP, email, FTP, mobile phones, and other devices and services. A pull agent pumps data from pull sources and pushes the data out to push agent counterparts. The push agent transforms and sends the data in messages to push sink, such as FTP, SMTP, or a mobile device. Both the pull agent and the push agent services are implemented as replicated services over a structured overlay network for high availability that can serve multiple requests to pump out data from multiple pull sources and push the data to multiple information sinks.

Claims

1. A system for pushing data to a plurality of data sinks, the system comprising:

a processor; and

a memory coupled to the processor, the memory storing computer-readable instructions that, upon execution by the processor, cause the system to;

publish a protocol specifying a data format supported by an assigned data sink of the plurality of data sinks, wherein the push service is tasked with pushing incoming data to the assigned data sink;

receive a message indicating incoming data is available for delivery to the assigned data sink;

schedule delivery of the incoming data from a push job queue to the assigned data sink; and

create a new runtime instance of the push service that transforms incoming data according to the protocol supported by the assigned data sink.

2. The system of claim 1, the memory storing additional computer-readable instructions that, upon execution by the processor, cause the system to:

accept an assignment of an additional assigned data sink to the push service, wherein the assignment of the additional data sink serves to additionally task the push service with pushing incoming data to the additional assigned data sink.

3. The system of claim 1, wherein the incoming data is retrieved from a data source and placed in an incoming data queue by a pull service component.

4. The system of claim 2, wherein the additional assignment of data sinks to the push service results from adjustments to a partitioning of the plurality of data sinks among a plurality of push service instances.

5. A method for pushing data to a plurality of data sinks, the method comprising:

providing a push service for distributing incoming data to the plurality of data sinks;

publishing a protocol specifying a data format supported by an assigned data sink of the plurality of data sinks, wherein the push service is tasked with pushing incoming data to the assigned data sink;

receiving a message indicating incoming data is available for delivery to the assigned data sink;

scheduling delivery of the incoming data from a push job queue to the assigned data sink; and

creating a new runtime instance of the push service that transforms incoming data according to the protocol supported by the assigned data sink.

6. The method of claim 5, further comprising:

accepting an assignment of an additional assigned data sink to the push service, wherein the assignment of the additional data sink serves to additionally task the push service with pushing incoming data to the additional assigned data sink.

7. The method of claim 5, wherein the incoming data is retrieved from a data source and placed in an incoming data queue by a pull service.

8. The method of claim 6, wherein the additional assignment of data sinks to the push service results from adjustments to a partitioning of the plurality of data sinks among a plurality of push service instances.

9. A computer-readable storage memory device comprising instructions for pushing data to a plurality of data sinks, wherein the instructions, when executed, cause a processor to:

provide a push service for distributing incoming data to the plurality of data sinks;

publish a protocol specifying a data format supported by an assigned data sink of the plurality of data sinks, wherein the push service is tasked with pushing incoming data to the assigned data sink;

receive a message indicating incoming data is available for delivery to the assigned data sink;

schedule delivery of the incoming data from a push job queue to the assigned data sink; and

create a new runtime instance of the push service that transforms incoming data according to the protocol supported by the assigned data sink.

10. The computer-readable storage memory device of claim 9, further comprising instructions that, when executed, cause a processor to:

accept an assignment of an additional assigned data sink to the push service, wherein the assignment of the additional data sink serves to additionally task the push service with pushing incoming data to the additional assigned data sink.

11. The computer-readable storage memory device of claim 9, wherein the incoming data is retrieved from a data source and placed in an incoming data queue by a pull service.

12. The computer-readable storage memory device of claim 10, wherein the additional assignment of data sinks to the push service results from adjustments to a partitioning of the plurality of data sinks among a plurality of push service instances.