Data gathering and distribution system

• US 20060026114A1
• Filed: 07/28/2004
• Published: 02/02/2006
• Est. Priority Date: 07/28/2004
• Status: Active Grant

• Current Assignee: Mybizintel Incorporated
• Original Assignee: Mybizintel Incorporated
• Inventors: Brawn, Barry, Gregoire, Ken
• Granted Patent: US 7,409,393 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A system and method for gathering and distributing data. The system is for extracting information from the world wide web and classifying the information in accordance with certain profiles. The information may include business intelligence, which may be categorized according to its relevance to predefined industry profiles or company profiles. The information may be further categorized according to its relevance to particular countries. If the information relates to a new company, the system builds a new company profile based upon the information. Users may create a user profile containing their information preferences, such as industry groups or particular countries or companies, and the system provides reports or alerts to the users referencing extracted information that is filtered by the user profile.

Claims

1. A system for gathering and classifying relevant data from the world wide web, the system comprising:

an extractor for crawling the world wide web and producing extracted information from at least one website;

an industry database containing a list of industry groups;

a company database containing profiles of companies;

an information database containing data records, each of said data records having an associated industry group selected from said list of industry groups;

a classifier for receiving said extracted information, said classified including a company comparison component for determining if said extracted information relates to a company profiled in said company database, and an industry component for determining if said extracted information relates to an industry listed in said list of industry groups, and a classification component responsive to said company comparison component and said industry comparison component for storing said extracted information in said information database as one of said data records.

2. The system claimed in claim 1, wherein said classifier rejects said extracted information if said industry component determines that said extracted information is unrelated to an industry listed in said list of industry groups.

3. The system claimed in claim 1, wherein each of said profiles of companies includes a company name, contact information, address information, line-of-business information, and website information.

4. The system claimed in claim 3, wherein said profiles of companies further includes at least one industry group selected from said list of industry groups, and wherein said at least one industry group is related to said line-of-business information.

5. The system claimed in claim 1, wherein said list of industry groups comprises industry categories and subcategories, and wherein each of said categories and subcategories includes associated keywords for associating a company with said category or subcategory.

6. The system claimed in claim 1, wherein each of said data records includes a data type indicator, a data source indicator, and an update date.

7. The system claimed in claim 1, wherein said industry component generates associated industry data, wherein said company component generates associated company data, and wherein said classification component stores said associated industry data and said associated company data with said extracted information.

8. The system claimed in claim 7, wherein said classifier further includes a country component for determining if said extracted information is associated with a country and generating associated country data, and wherein said classification component stores said associated country data with said extracted information.

9. The system claimed in claim 1, further including a report generator and a memory storing a user profile, said user profile specifying user selected characteristics, said report generator filtering said data records based upon said user selected characteristics to identify desired data records and outputting a report containing said desired data records.

10. The system claimed in claim 9, further comprising a user interface accessible to a remote computer over the world wide web, said user interface including a profile building component for receiving user input from said remote computer and for building said user profile based upon said user input.

11. The system claimed in claim 1, wherein said data records include keywords and wherein said industry component includes one or more filters for reducing said extracted information to a subset of information, and a keyword module for comparing said subset of extracted information with said keywords.

12. The system claimed in claim 11, wherein said industry component further includes a normalization component for receiving a match count from said keyword module and for normalizing said match count with respect to a number of words in said subset of information, and a threshold module for determining whether said extracted information is associated with an industry based upon said normalized match count.

13. A system for gathering and classifying relevant data from the world wide web, the system comprising:

an extractor for navigating the world wide web and extracting information from at least one website;

a first memory storing a categorization scheme, said categorization scheme defining a plurality of categories;

a second memory storing a plurality of entity profiles, each entity profile being associated with at least one of said categories;

a third memory storing information records, each of said information records having an associated category selected from said plurality of categories;

a classifier for receiving said extracted information, said classified including an entity comparison component for determining if said extracted information relates to one of said entity profiles, and a categorization component for determining if said extracted information relates to one of said categories, and a classification component responsive to said entity comparison component and said categorization component for storing said extracted information in said third memory as one of said information records.

14. The system claimed in claim 13, wherein said plurality of categories comprise industry groups and subgroups.

15. The system claimed in claim 14, wherein each of said industry groups and subgroups further includes associated keywords, and wherein said categorization component determines that said extracted information relates to said one of said categories based upon a keyword search locating one or more of said associated keywords for said one of said categories.

16. The system claimed in claim 13, wherein said classifier rejects said extracted information if said categorization component determines that said extracted information is unrelated to any of said plurality of categories.

17. The system claimed in claim 13, wherein each of said entity profiles includes a name, contact information, address information, product or service information, website information, and at least one category selected from said plurality of categories, and wherein said at least one category is related to said product or service information.

18. The system claimed in claim 13, wherein each of said information records includes a data type indicator, a data source indicator, and an update date.

19. The system claimed in claim 13, wherein said classifier further includes a country component for determining if said extracted information is associated with a country and generating associated country data, and wherein said classification component stores said associated country data with said extracted information.

20. The system claimed in claim 13, further including a report generator and a memory storing a user profile, said user profile specifying user selected characteristics, said report generator filtering said information records based upon said user selected characteristics to identify desired information records and outputting a report containing said desired information records.

21. The system claimed in claim 20, further comprising a user interface accessible to a remote computer over the world wide web, said user interface including a profile building component for receiving user input from said remote computer and for building said user profile based upon said user input.

22. The system claimed in claim 13, wherein said information records include keywords and wherein said categorization component includes one or more filters for reducing said extracted information to a subset of information, and a keyword module for comparing said subset of extracted information with said keywords.

23. The system claimed in claim 22, wherein said categorization component further includes a normalization component for receiving a match count from said keyword module and for normalizing said match count with respect to a number of words in said subset of information, and a threshold module for determining whether said extracted information is associated with a category based upon said normalized match count.

25. The method claimed in claim 20, further including steps of providing a plurality of company profiles, each of said company profiles having at least one associated industry group selected from said categorization scheme, and wherein said step of classifying further includes determining if said extracted information is related to one of said company profiles, and wherein said step of linking includes linking said extracted information with said one of said company profiles.

26. The method claimed in claim 21, wherein said step of classifying further includes determining if said extracted information relates to an unprofiled company, building a new company profiles based upon said extracted information, and adding said new company profile to said plurality of company profiles.

27. The method claimed in claim 22, wherein said step of building includes extracting corporate information regarding name, address, country, and line-of-business from said website and populating said new company profile with said corporate information.

28. The method claimed in claim 20, wherein said step of linking includes storing said extracted information in memory with an identifier, said identifier referencing said at least one of said industry groups.

29. The method claimed in claim 20, wherein said categorization scheme comprises industry groups and industry subgroups, each of said industry subgroups comprising a subcategory of one industry group.

30. The method claimed in claim 20, further including a step of determining whether additional information is available from said website and, if so, repeating said steps of extracting, classifying, and linking.

31. The method claimed in claim 20, further including a step of storing said extracted information in a data record, said data record including a data type, a data source, and a time and date.

32. The method claimed in claim 27, wherein said data type is selected from a type consisting of news, government, trade event, and stock exchange information.

33. The method claimed in claim 27, further including steps of receiving user selections, building a user profile based upon said user selections, and filtering said data records based upon said user profile to identify desired data records.

34. The method claimed in claim 29, further including a step of generating a report for said user referencing said desired data records.

35. The method claimed in claim 29, wherein said user selections include one or more of said industry groups, and wherein said step of filtering includes selecting said data records related to said one or more of said industry groups based upon links established by said step of linking.

24. A method for gathering and classifying information from the world wide web, the method comprising the steps of:

providing a categorization scheme of industry groups;

extracting information from a website;

classifying said extracted information by determining if said extracted information is relevant to at least one of said industry groups; and

linking said extracted information with said at least one of said industry groups.

36. The method claimed in claim 24, wherein said step of providing a categorization scheme includes associating keywords with each of said industry groups, and wherein said step of classifying includes filtering said extracted information to produce a subset of extracted information and comparing said subset of extracted information with said keywords.

37. The method claimed in claim 36, wherein said step of classifying further includes producing a match count from said step of comparing, normalizing said match count with respect to a number of words in said subset of extracted information, and determining if said extracted information is associated with an industry based upon said normalized match count.