Job categorization system and method

• US 20060212466A1
• Filed: 06/30/2005
• Published: 09/21/2006
• Est. Priority Date: 03/11/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Khanna, Sandeep, Ting, Joseph, Hyder, Adam
• Granted Patent: US 7,702,674 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A computer system and method for capture, managing and presenting data obtained from various often unrelated postings via the Internet for examination by a user. This system includes a scraping module having one or more scraping engines operable to scrape information data sets from listings on the corporate sites and web sites, direct feeds, and other sources, wherein the scraping module receives and stores the scraped listing information data sets in a database. The system also has a management platform coordinating all operation of and communication between the sources, system administrators and processing modules. The processing modules in the platform include scraping management module analyzing selected scraped data stored in the database, and a categorization module that examines and categorizes each data set stored in the database into one or more of a predetermined set of categories and returns categorized data sets to the database.

Claims

1. A computer system for managing capture and processing of job listing information data captured through a data network from a plurality of job related sources for compilation into a searchable data structure, the system comprising:

an administrative portal module providing system administration and operational control through a network interface; and

a scraping management module coordinating operation of and communication between one or more job scraping engines to obtain scraped job information data sets from corporate career sites and job boards identified by a site management module in the administrative portal module and store the scraped data sets in a database;

a job listing data categorization module operable to examine and categorize each job data set stored in the database into one or more of a predetermined set of job categories and return categorized job data sets to the database; and

a quality management module coupled to the scraping management module analyzing each scraped job data set stored in the database for conformance to predetermined quality rules.

2. The system according to claim 1 wherein the data network includes the Internet.

3. The system according to claim 1 further comprising:

a search bank synchronizer communicating with the database for compiling and transferring categorized job data sets from the database to a job search bank.

4. The system according to claim 3 wherein the categorization module comprises:

a job categorization database; and

a categorization module determining a confidence value in each predetermined job category for each scraped job listing information data set by comparing text of each scraped job data set with previously categorized job data text in the job categorization database.

5. The system according to claim 4 wherein the administration portal comprises a categorization review module permitting a reviewer to verify categorizations determined by the document categorization platform service in the categorization module.

6. The system according to claim 4 wherein each job data set returned to the database includes an assigned job category determined by the categorization module and an assigned confidence value for that category.

7. The system according to claim 6 wherein each data set returned to the database further includes a confidence value for each predetermined job category.

8. The system according to claim 4 wherein each job data set returned to the database includes a manual review flag set if the assigned confidence value is below a predetermined threshold value.

9. The system according to claim 8 wherein the administration portal comprises a categorization review module permitting a reviewer to verify categorizations determined by the categorization module

10. The system according to claim 1 wherein the quality management module sets a quality flag associated with each scraped job data set not meeting the predetermined criteria.

11. The system according to claim 10 wherein the administrative portal further comprises a quality review module communicating with the quality management module permitting a reviewer to manually examine job data sets for which the quality flag has been set.

12. A method of obtaining, handling and compiling job information data sets comprising:

scraping job information data sets from one or more job listings on one or more corporate career sites or job boards available through the Internet;

storing a job data set corresponding to each scraped job listing found in a database;

analyzing each scraped data set stored in the database for conformance to predetermined quality criteria; and

categorizing each data set stored in the database into one or more predetermined job categories and returning the categorized job information data set to the database.

13. The method according to claim 12 further comprising obtaining job information data sets from one or more of customer sites through an XML feed.

14. The method according to claim 12 wherein the categorizing operation further comprises:

assigning a confidence value for each job information data set for each of the predetermined job categories.

15. The method according to claim 12 wherein the categorizing operation comprises:

comparing text of each scraped job information data set with text of previously categorized job information data sets in a job categorization database; and

determining a confidence value in each predetermined category for each scraped data set.

16. The method according to claim 15 further comprising:

flagging each categorized scraped data set that has a confidence value below a predetermined value for manual review; and

providing a manual review module permitting a reviewer to verify any flagged categorizations through an administration portal.

17. The method according to claim 12 further comprising assigning a confidence value for the job category assigned to each data set returned to the database.

18. The method according to claim 17 further comprising flagging any data set returned to the database having an assigned confidence level below a predetermined threshold.

19. The method according to claim 12 further comprising:

transferring selected categorized job information data sets from the job search bank through a web client server cluster to a job seeker in response to a query by the job seeker.

20. The method according to claim 12 wherein the scraping operation further comprises:

accessing one of the job boards or corporate career sites through the Internet;

flagging any scraped job information data set not meeting the predetermined quality criteria;

permitting a manual review of flagged job information data sets returned to the database, and the categorizing operation further comprises;

comparing data in each scraped job information data set with previously categorized job data set data in a categorization database; and

determining a confidence value in each predetermined job category for each scraped job information data set.

21. The method according to claim 20 further comprising:

flagging each categorized scraped data set that has a confidence value below a predetermined value for manual review; and

providing a manual review module in an administration portal permitting a reviewer to verify any flagged categorizations.

22. The method according to claim 20 further comprising transferring selected categorized data sets from the search bank through a web server to a user in response to a query by the user.

23. A computer readable medium encoding a computer program of instructions for executing a computer process for scraping job description data from corporate career sites and job boards, the computer process comprising:

scraping listing information data from one or more listings on sites available through the Internet;

storing a scraped data set corresponding to each scraped listing information in a database;

analyzing each scraped data set stored in the database for conformance to predetermined quality criteria; and

categorizing each data set stored in the database into one or more predetermined categories and returning the categorized data set to the database.

24. The computer readable medium of claim 23 wherein the process further comprises:

flagging any scraped data set not meeting the predetermined quality criteria;

permitting a manual review of flagged data sets returned to the database, and wherein the categorizing operation further comprises;

comparing text in each scraped data set with previously categorized data set text in a categorization database; and

determining a confidence value in each predetermined category for each scraped data set.