Extended correlation methods in a content transformation engine

• US 8,521,740 B2
• Filed: 07/03/2007
• Issued: 08/27/2013
• Est. Priority Date: 04/04/2006
• Status: Expired due to Fees

• Current Assignee: Boomerang Technology Holdings, LLC
• Original Assignee: Boomerang Technology Holdings, LLC
• Inventors: Gibson, Anthony, Henderson, Bruce R.
• Primary Examiner(s): Badawi, Sherief
• Application Number:
  US11/773,292
• Publication Number:
  US 20080016502A1
• Time in Patent Office: 2,247 Days
• Field of Search: 709/226, 709/203, 709/217, 707/737, 707/736, 707/804, 707/754, 707/631, 707/756, 707/749
• US Class Current: 707/737
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24544 Join order optimisation
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

In one embodiment, a system for accumulating and correlating data. The system comprises a plurality of silos each having raw data and a plurality of feedlets that transform the raw data into a plurality of high value data feeds. The system also has at least one fuselet that receives the plurality of high value data feeds and generates a complex composite data feed by correlating information in at least one high value data feed.

Claims

1. A system for constructing a complex composite data feed in a syndication format from source data in native, non-syndication format stored in source data repositories external to the system, the system comprising:

at least one computer having a memory and tangible processor;

a plurality of content transformation agents, customized for the plurality of source data repositories, which, in accordance with the customization, dynamically select and transform the source data from non-syndication format within the plurality of source data repositories into a plurality of data feeds in the syndication format comprising payload data from data primitives stored originally in the source data repositories; and

at least one content fusion and aggregation agent that receives the plurality of transformed data feeds and generates a complex composite data feed, customized based on predetermined organizational criteria and specific user criteria, in the syndication format as a union of selected data primitives from the plurality of transformed data feeds,

wherein the complex composite data feed is generated by correlating data primitives within the plurality of transformed data feeds using detected relationships among the data primitives in the plurality of transformed data feeds and selecting the selected data primitives via customized criteria from the plurality of transformed data feeds for the complex composite data feed based on the correlated data primitives.

2. The system of claim 1,

wherein at least one data feed is in a web-accessible format.

3. The system of claim 1,

wherein the complex composite data feed is in a web-accessible format.

4. The system of claim 1,

wherein the at least one content fusion and aggregation agent further receives at least one other data feed available via the Internet and fuses the data feed available via the Internet with the plurality of data feeds to generate the complex composite data feed.

5. The system of claim 1,

wherein the at least one content fusion and aggregation agent generates the complex composite data feed by at least one of:

correlating across multiple data feeds, correlating based on semantic information in a data feed, correlating using temporal conditions, correlating based on logical operators, correlating based on a probability, correlating based on geo-spatial information, correlating according to metadata contained in a data feed, correlating according to an externally-defined correlation plan, and correlating according to a run-time-loaded correlation plan.

6. The system of claim 1,

wherein the content transformation agents comprise computer readable instructions stored on a recordable medium for:

configuring a temporary data structure according to a target data schema;

extracting key pieces of information from the source data;

correlating the key pieces of information; and

inserting the correlated pieces of information into the temporary data structure.

7. The system of claim 6,

wherein the instructions for correlating the key pieces of information further comprises instructions for:

correlating information from more than one source data repository, correlating based on semantic information, correlating using temporal conditions, correlating based on logical operators, correlating based on a probability, correlating based on geo-spatial information, correlating according to message metadata, correlating according to an externally-defined correlation plan, and correlating according to a run-time-loaded correlation plan.

8. The system of claim 1,

wherein the at least one content fusion and aggregation agent comprises computer readable instructions stored on a recordable medium for:

configuring a temporary data structure according to a target data schema;

extracting key pieces of information from a data feed;

correlating the key pieces of information; and

inserting the correlated pieces of information into the temporary data structure.

9. The system of claim 8,

wherein the instructions for correlating the key pieces of information further comprise instructions for:

correlating information from multiple high-value feeds, correlating based on semantic information, correlating using temporal conditions, correlating based on logical operators, correlating based on a probability, correlating based on geo-spatial information, correlating according to message metadata, correlating according to an externally-defined correlation plan, and correlating according to a run-time-loaded correlation plan.

10. A system for constructing a complex composite data feed in a syndication format from source data in native, non-syndication format stored in source data repositories external to the system, the system comprising:

at least one computer having a memory and tangible processor;

a content fusion and aggregation agent comprising;

a collector module operable to receive a message, including source data dynamically selected from a source data repository by a content transformation agent customized for the source data repository to select the source data, and transform the message into payload data in a neutral syndication format from data primitives stored originally in the source data repositories;

a correlator module operable to receive the message in the neutral syndication format and transform the message into a content element; and

an emitter module operable to receive the content element and output the content element in a target syndication format as an element of a complex composite data feed, customized based on predetermined organizational criteria and specific user criteria and generated by correlating information of a plurality of messages from detected relationships among the data primitives in the plurality of messages, and selecting data primitives via customized criteria from the plurality of messages for the complex composite data feed based on the correlated information.

11. The system of claim 10,

wherein the source data repository is an event source and the message is an event.

12. The system of claim 10,

wherein the collector module is an assembler module, and wherein the assembler module interacts with the source data repository to generate the message.

13. The system of claim 10,

wherein the correlator module is further operable to transform the message into a content element by correlating information extracted from the message.

14. The system of claim 13,

wherein the correlator module is further operable to transform the message into a content element by one of:

correlating across multiple messages, correlating based on semantic information, correlating using temporal conditions, correlating based on logical operators, correlating based on a probability, correlating based on geo-spatial information, correlating according to message metadata, correlating according to an externally-defined correlation plan, and correlating according to a run-time-loaded correlation plan.

15. The system of claim 10,

wherein the source data repository is a content transformation agent and the message is a data feed.

16. The system of claim 10,

wherein the collector module is a content transformation agent and the neutral format is a data feed.

17. A method for constructing a complex composite data feed in a syndication format from source data in native, non-syndication format stored in source data repositories external to a system in which the method is performed, the method comprising:

communicating, using a computer with memory and a tangible processor, with a source data repository having source data in non-syndication format;

dynamically selecting and transforming the source data from non-syndication format within the source data repository into a transformed data feed in the syndication format comprising payload data from data primitives stored originally in the source data repositories using a content transformation agent customized for the source data repository, the transformed data feed in the syndication format comprising selected source data selected and transformed into the payload data from the data primitives by the content transformation agent in accordance with the customization; and

fusing, by at least one content fusion and aggregation agent, the transformed data feed to generate a complex composite data feed, customized based on predetermined organizational criteria and specific user criteria, in the syndication format as a union of selected data primitives from the transformed data feed and selected data primitives in syndication format from other transformed data feeds from other source data repositories;

wherein the fusing comprises correlating data primitives within the transformed data feed and the other transformed data feeds using detected relationships among the data primitives in the transformed data feed and the other transformed data feeds, and selecting the selected data primitives via customized criteria from the transformed data feed and the other transformed data feeds for the complex composite data feed based on the correlated data primitives.

18. The method of claim 17,

wherein the correlating the selected data primitives comprises one of:

correlating across multiple data feeds, correlating based on semantic information, correlating using temporal conditions, correlating based on logical operators, correlating based on a probability, correlating based on geo-spatial information, correlating according to message metadata, correlating according to an externally-defined correlation plan, and correlating according to a run-time-loaded correlation plan.

19. The method of claim 17,

wherein the selected data primitives comprise an event.

20. The method of claim 17,

wherein the data feed is generated responsive to the receipt of an activation message.