System and method for aggregating web feeds relevant to a geographical locale from multiple sources

• US 8,578,274 B2
• Filed: 09/28/2009
• Issued: 11/05/2013
• Est. Priority Date: 09/26/2008
• Status: Active Grant

• Current Assignee: American Express Kabbage Incorporated (American Express Company)
• Original Assignee: Radius Intelligence Inc. (American Express Company)
• Inventors: Druzgalski, Adrian, Shirazi, Darian
• Primary Examiner(s): Vu, Kieu
• Assistant Examiner(s): Bycer, Eric J
• Application Number:
  US12/568,595
• Publication Number:
  US 20100083124A1
• Time in Patent Office: 1,499 Days
• Field of Search: 715234-242, 715/254, 715/738
• US Class Current: 715/738
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries

Abstract

Systems and methods for aggregating web feeds relevant to a geographical locale from multiple sources are disclosed. One embodiment includes, aggregating a web feed using identifier information included in source metadata, parsing the web feed according to mapping information in the source metadata to generate a normalized web feed from the web feed having a data structure specified by the mapping information, determining the geographical locale that is relevant to the web feed, performing filtering on the normalized web feed to determine whether the normalized web feed includes qualified content for publication, and/or publishing the qualified content to be accessible to the user on the user device through a user interface on which the qualified content is depicted as being relevant to the geographical locale.

Claims

1. A method of collecting feeds relevant to a geographical locale from multiple sources to be electronically published to a user through a user device, comprising:

receiving a web feed having source metadata that includes a geographic location identifier, and a plurality of tags including title data, summary data, and text data and in a source format;

generating mapping information for mapping the tags in the source format to a normalized data structure having a title field, a summary field, and a text field, the mapping information generated from (i) a probabilistic model using stored statistical information to assign the plurality of tags to the title field, the summary field and the text field of the normalized data structure, and (ii) heuristic data that assigns the plurality of tags to the title field, the summary field and the text field of the normalized data structure based on a size of each of the plurality of tags;

parsing the web feed according to the mapping information generate a normalized web feed from the web feed, the normalized web feed having the normalized data structure specified by the mapping information;

determining a relevant geographical locale for the normalized web feed by parsing text data in the text field of the normalized web feed to detect location keywords therein;

when the relevant geographic locale differs from the geographic location identifier in the source metadata, identifying a geographic location for the normalized web feed based exclusively on the location keywords;

performing filtering on the normalized web feed to determine whether the normalized web feed includes qualified content for publication; and

publishing the qualified content to be accessible to the user on the user device through a user interface on which the qualified content is depicted as being relevant to the geographical location.

2. The method of claim 1, further comprising,

identifying an image from the normalized web feed;

wherein, the image is published on the user interface as being associated with the qualified content.

3. The method of claim 2, wherein, the identifying the image comprises:

identifying an image link from the normalized web feed; and

applying an image filter on the image.

4. The method of claim 3, wherein, the applying the image filter comprises:

using dimensions of the images to determine presence of advertisements or icons in the images, and deleting the images from the normalized web feed when the presence of advertisements are detected in the images.

5. The method of claim 3, wherein, the applying the image filter comprises:

performing character recognition to detect text in the images to determine presence of advertisements in the images, and deleting the images from the normalized web feed when the presence of advertisements are detected in the images.

6. The method of claim 1, further comprising, data mining trending data of the qualified content and prioritizing placement of the qualified content in the user interface according to the trending data.

7. The method of claim 6, wherein, the trending data includes local trending data and global trending data.

8. The method of claim 7, wherein, the global trending data includes trends collected by third party sites or social networking sites.

9. The method of claim 6, wherein, the trending data includes user trends.

10. The method of claim 9, further comprising, logging user action to determine user trends.

11. The method of claim 1, wherein, the performing the filtering comprises:

identifying content in the normalized web feed with obscenities or profanities; and

disqualifying the content having obscenities or profanities from publication.

12. The method of claim 1, wherein, the performing the filtering comprises:

identifying an advertisement link in the normalized web feed; and

disqualifying the advertisement link from publication.

13. The method of claim 1, wherein, the performing the filtering comprises:

identifying content in the normalized web feed having corrupt HTML code or broken links; and

disqualifying the content having corrupt HTML code or broken links from publication.

14. The method of claim 1, wherein, the performing the filtering comprises:

extracting timing data from the normalized web feed; and

using the timing data to determine whether the web feed includes content that is temporally relevant.

15. The method of claim 1, wherein, the determining the relevant geographical locale further comprises:

converting the location keywords to a latitude and longitude dataset; and

using the latitude and longitude dataset to determine the relevant geographical locale.

16. The method of claim 1, further comprising, using the normalized web feed to ensure that the web feed is aggregated only once.

17. The method of claim 1, wherein, the web feeds include news feeds.

18. The method of claim 1, wherein, the web feeds are temporally relevant.

19. The method of claim 1, wherein, the web feeds include one or more of, RSS feeds, Atom, JSON, and raw XML.

20. A method performed by a computer in communication with a machine-readable storage medium having stored thereon a set of instructions which when executed, causes a processor of the computer to identify and publish web feeds relevant to a geographical locale collected from multiple sources, the method comprising:

receiving a web feed having source metadata that includes a geographic location identifier, and a plurality of tags including title data, summary data, and text data and in a source format;

generating mapping information for mapping the tags in the source format to a normalized data structure having a title field, a summary field, and a text field, the mapping information generated from (i) a probabilistic model using stored statistical information to assign the plurality of tags to the title field, the summary field and the text field of the normalized data structure, and (ii) heuristic data that assigns the plurality of tags to the title field, the summary field and the text field of the normalized data structure based on a size of each of the plurality of tags;

parsing the web feed according to the mapping information, the source metadata and the plurality of tags;

creating a normalized web feed from the web feed, the normalized web feed having the normalized data structure specified by the mapping information;

determining a relevant geographic locale corresponding to the normalized web feed by parsing text data in the text field of the normalized web feed to detect location keywords therein

when the relevant geographic locale differs from the geographic location identifier in the source metadata, identifying the geographic location for the normalized web feed based exclusively on the location keywords;

publishing valid content in the normalized web feed such that the valid content is accessible to the user through a user interface presented on a user computing device on which the qualified content is depicted as being associated with the geographical location;

determining trending data of the valid content; and

prioritizing placement of the valid content in the user interface first according to the geographic location and then according to the trending data.

21. The method of claim 20, wherein, the valid content is identified by performing filtering on the normalized web feed.

22. The method of claim 20, wherein, the web feed is aggregated from a third party host using identifier information included in the source metadata.

23. A system for publishing web feeds relevant to a geographical locale collected from multiple sources, the system, comprising:

one or more client devices connected via a network to a host server;

a listing repository to receive and store web feeds, each web feed having source metadata that includes a geographic location identifier, and a plurality of tags including title data, summary data, and text data and in a source format;

an aggregator engine, coupled to the listing repository, for aggregating the web feeds using the geographic location identifier in the source metadata;

a normalization module in the aggregator engine for generating mapping information for mapping the tags in the source format to a normalized data structure having a title field, a summary field, and a text field, the mapping information generated from (i) a probabilistic model using stored statistical information to assign the plurality of tags to the title field, the summary field and the text field of the normalized data structure, and (ii) heuristic data that assigns the plurality of tags to the title field, the summary field and the text field of the normalized data structure based on a size of each of the plurality of tags; and

the normalization module further for parsing the web feeds to generate normalized web feeds having the a normalized data structure;

a parser for determining a relevant geographic locale corresponding to the normalized web feed by parsing text data in the text field of the normalized web feeds to detect location keywords therein and when the relevant geographic locale differs from the geographic location identifier in the source metadata, identifying the geographic location for the normalized web feeds based exclusively on the location keywords;

a filter module in the aggregator engine that filters the normalized web feeds to determine whether the normalized web feeds include qualified stories for publication;

multiple content repositories each coupled to the aggregator engine for storing the qualified stories; and

a publish server coupled to the multiple content repositories for publishing the qualified stories accessible to a user through a user interface on which the qualified content is depicted as being relevant to the geographical location.

24. The system of claim 23, wherein, the qualified stores include news feeds that are published in real time.

25. The system of claim 23, further comprising,

a trending engine coupled to the publish server;

wherein, the trending engine determines trending data of the valid stories.

26. The system of claim 25, wherein, the trending engine data mines local trends and collects global trends from third party sources or social networking sites.

27. The system of claim 26, wherein, each of the multiple content repositories is coupled to the aggregator engine;

and

wherein, the aggregator engine stores the valid stories in redundancy on each of the multiple content repositories.

28. The system of claim 25, wherein, the trending engine collects user action data to determine user trends.

29. The system of claim 25, wherein, the publish server prioritizes placement of the valid content in the user interface according to the trending data.

30. The system of claim 23, wherein, each of the multiple content repositories stores the valid stories in redundancy.

31. A method for collecting feeds relevant to a geographical locale from multiple sources to be electronically published to a user through a user device, comprising:

receiving a web feed having source metadata that includes a geographic location identifier, and a plurality of tags including title data, summary data, and text data and in a source format;

generating mapping information for mapping the tags in the source format to a normalized data structure having a title field, a summary field, and a text field, the mapping information generated from (i) a probabilistic model using stored statistical information to assign the plurality of tags to the title field, the summary field and the text field of the normalized data structure, and (ii) heuristic data that assigns the plurality of tags to the title field, the summary field and the text field of the normalized data structure based on a size of each of the plurality of tags;

parsing the web feed according to the mapping information to generate a normalized web feed from the web feed, the normalized web feed having the normalized data structure specified by the mapping information;

determining a relevant geographical locale for the normalized web feed by parsing text data in the text field of the normalized web feed to detect location keywords therein; and

when the relevant geographical locale differs from the geographic location identifier in the source metadata, identifying the relevant geographical locale as a relevant geographic location for the normalized web feed based on the location keywords;

filtering the normalized web feed to determine whether the normalized web feed includes qualified content for publication based on the relevant geographic location; and

publishing the qualified content to the user through a user interface wherein the qualified content is depicted as being from the relevant geographic location.

32. A method of using a device to access feeds that are relevant to a geographical locale aggregated from multiple sources through a host site, comprising:

receiving a first set of web feeds and a second set of web feeds, each web feed of the first and second sets of web feeds having source metadata that includes a geographic location identifier, and a plurality of tags including title data, summary data, and text data and in a source format;

generating mapping information for mapping the tags in the source format to a normalized data structure having a title field, a summary field, and a text field, the mapping information generated from (i) a probabilistic model using stored statistical information to assign the plurality of tags to the title field, the summary field and the text field of the normalized data structure, and (ii) heuristic data that assigns the plurality of tags to the title field, the summary field and the text field of the normalized data structure based on a size of each of the plurality of tags;

determining relevant geographic locales corresponding to the first set of web feeds and the second set of web feeds by parsing text data in the text fields of the web feeds to detect location keywords therein; and

when the relevant geographical locales differ from the geographic location identifier in the source metadata identifying the geographic locations of the web feeds based exclusively on the location keywords;

retrieving first relevant web feeds from the first set and second set of web feeds relevant to a first geographical locale;

when the first geographic locale corresponds to a geographic locale of a user, displaying the first relevant web feeds on a user interface when the user visits the host site;

in response to the user selecting a particular feed in the first relevant feeds, opening a link to the particular feed in a second user interface such that the user views content of the particular feed on a provider site of the content;

wherein, the second user interface includes a widget having a sharing feature allowing the user to share the content with other users;

retrieving trended feeds related to a detected trend in response to the user selecting the detected trend and displaying the trended feeds in the user interface;

wherein, the trended feeds are also relevant to the first geographical locale based on the location keywords; and

in response to the user selecting a second geographical locale through the user interface, retrieving second web feeds relevant to the second geographical locale based on the location keywords and displaying the second relevant web feeds in the second user interface.

33. The method of claim 32, wherein, the sharing feature allows the user to share the content with other users on a third-party social networking site.

34. The method of claim 32, wherein, the user interface further includes a commenting feature allowing the user to share comments regarding the content with other users.

35. The method of claim 32, wherein, the first geographical locale is associated with the user by default based on profile data of the user.

36. The method of claim 32, wherein, the first and second set of web feeds are temporally relevant.

37. The method of claim 32, wherein, the trended feeds are temporally relevant.