Ecosystem method of aggregation and search and related techniques

• US 20060004691A1
• Filed: 06/20/2005
• Published: 01/05/2006
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Synacor, Inc.
• Original Assignee: Technorati, Inc. (Synacor, Inc.)
• Inventors: Sifry, David L.
• Granted Patent: US 7,596,571 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

Techniques are described for effecting data aggregation in a network. Event notifications are received from a plurality of content sites on the network. Each event notification indicates occurrence of an event associated with a corresponding one of the content sites. Each event corresponds to a change to content associated with the corresponding content site. In response to each event notification, event metadata are generated. The event metadata include a time stamp for the event and change data corresponding to the change to the content. The event metadata are indexed in a plurality of databases with reference to a plurality of index metrics including the time stamp. The indexing of the event metadata for each event occurs within days, hours, or even minutes of receipt of the corresponding event notification thereby enabling retrieving of the changes to the content from the plurality of databases using any of the index metrics in near real time.

Claims

1. A computer-implemented method of data aggregation in a network, comprising:

receiving event notifications from a plurality of content sites on the network, each event notification indicating occurrence of an event associated with a corresponding one of the content sites, each event corresponding to a change to content associated with the corresponding content site;

in response to each event notification, generating event metadata, the event metadata including a time stamp for the event and change data corresponding to the change to the content; and

indexing the event metadata in a plurality of databases with reference to a plurality of index metrics including the time stamp, the indexing of the event metadata for each event occurring within seven days of receipt of the corresponding event notification thereby enabling retrieving of the changes to the content from the plurality of databases using any of the index metrics.

2. The method of claim 1 wherein the event notifications are generated by event notification code associated with content publishing code.

3. The method of claim 2 wherein the content publishing and event notification codes both reside on selected ones of the content sites.

4. The method of claim 3 wherein the event notification code is integrated with the content publishing code on at least some of the selected content sites.

5. The method of claim 2 wherein at least a portion of the content publishing and event notification codes resides on a third party service provider network separate from and associated with selected ones of the content sites.

6. The method of claim 2 wherein the content publishing code comprises any of a web log publishing tool, a wiki web page editing tool, a social network profile editing tool, a content management system tool, and a personal publishing tool.

7. The method of claim 1 further comprising receiving and indexing aggregated change data from a third party service provider site, the aggregated change data corresponding additional changes to content corresponding to additional content sites associated with the third party service provider site over a period of time.

8. The method of claim 1 further comprising, in response to selected ones of the event notifications, retrieving at least a portion of the event metadata from the corresponding content sites.

9. The method of claim 8 further wherein retrieving at least a portion of the event metadata comprises any of spidering the corresponding site, inferring known metadata locations for feeds, and inferring known metadata locations for profile pages.

10. The method of claim 1 the event metadata for selected ones of the event notifications is determined at least in part with reference to the selected event notifications.

11. The method of claim 1 wherein the event metadata for each event further relates to any of a uniform resource locator (URL) associated with the content site corresponding to the event, a permalink associated with the change in content, a content creator identifier, an event type identifier, an event identifier, the content, the change to the content, at least one keyword relating to the content, inbound links to the content, outbound links from the content, and any structured data or metadata associated with the event.

12. The method of claim 1 wherein the change data for selected events include new content posted on the corresponding content sites.

13. The method of claim 1 wherein the plurality of databases comprises a global database in which the event metadata corresponding to all events are indexed.

14. The method of claim 1 wherein the plurality of databases comprises an ordered-by-time database in which the event metadata corresponding to all events are primarily indexed with reference to the time stamp.

15. The method of claim 1 wherein the event metadata for selected ones of the events include inbound link data relating to links pointing to the content associated with the selected events, and wherein the plurality of databases comprises an ordered-by-authority database in which the event metadata corresponding to the selected events are primarily indexed with reference to the inbound link data.

16. The method of claim 1 wherein the event metadata for selected ones of the events include keyword data relating to keywords identified in the content associated with the selected events, and wherein the plurality of databases comprises an ordered-by-keyword database in which the event metadata corresponding to the selected events are primarily indexed with reference to the keyword data.

17. The method of claim 1 wherein the event metadata for first selected ones of the events include inbound link data relating to links pointing to the content associated with the first selected events, and wherein the event metadata for second selected ones of the events include keyword data relating to keywords identified in the content associated with the second selected events, and wherein the plurality of databases comprises a global database in which the event metadata corresponding to all events are indexed, an ordered-by-time database in which the event metadata corresponding to all events are primarily indexed with reference to the time stamp, an ordered-by-authority database in which the event metadata corresponding to the first selected events are primarily indexed with reference to the inbound link data, and an ordered-by-keyword database in which the event metadata corresponding to the second selected events are primarily indexed with reference to the keyword data.

18. The method of claim 1 wherein each of the plurality of databases has an associated index, and wherein indexing the event metadata in each of the plurality of databases comprises:

indexing recently received event metadata into a first plurality of incremental indexes corresponding to a smaller one of a plurality of incremental index sizes;

combining the first plurality of the incremental indexes to create one of a second plurality of incremental indexes corresponding to a larger one of the plurality of incremental index sizes; and

repeating the indexing and combining for each of the plurality of incremental index sizes to thereby incrementally integrate all of the event metadata into the index.

19. The method of claim 1 further comprising retrieving portions of the event metadata from the plurality of databases in response to queries.

20. The method of claim 19 wherein each of the plurality of databases comprises a master copy of the database and a plurality of slave copies of the database, and wherein retrieving portions of the event metadata in response to the queries comprises retrieving the event metadata from the slave copies.

21. The method of claim 1 wherein the event metadata for selected ones of the events include inbound link data relating to links pointing to the content associated with the selected events, and wherein the plurality of databases comprises an ordered-by-authority database in which the event metadata corresponding to the selected events are primarily indexed with reference to the inbound link data, the method further comprising identifying authoritative individuals for each of a plurality of categories with reference to the ordered-by-authority database.

22. The method of claim 21 wherein identifying the authoritative individuals for each category comprises identifying specific content in the category generated by the authoritative individuals with reference to a number of inbound links associated with the specific content.

23. The method of claim 22 wherein identifying the authoritative individuals for each category further comprises weighting the number of inbound links with reference to an authority metric associated with related content from which the inbound links originated.

24. The method of claim 21 wherein identifying the authoritative individuals for each category comprises identifying first specific content in the category directed to which second specific content associated with the authoritative individuals includes outbound links.

25. The method of claim 24 wherein identifying the authoritative individuals is done with reference to the time stamps associated with the second specific content.

26. The method of claim 1 further comprising tracking responses of a plurality of individuals to a specific event with reference to the plurality of databases.

27. The method of claim 26 wherein tracking of the responses is done with reference to responsive content generated by the individuals and indexed in the plurality of databases.

28. The method of claim 26 further comprising generating representations of the responses which are operable to be presented as feeds in live media.

29. The method of claim 28 wherein the representations are operable to represent changes in the responses over time.

30. The method of claim 1 wherein the indexing of the event metadata for each event occurs within one of one day, 12 hours, six hours, three hours, two hours, one hour, 10 minutes, and 2 minutes.