Electronic employee selection systems and methods

• US 7,310,626 B2
• Filed: 01/24/2005
• Issued: 12/18/2007
• Est. Priority Date: 08/03/2000
• Status: Expired due to Term

• Current Assignee: Cadient LLC
• Original Assignee: Kronos Talent Management Incorporated (Seahawk Acquisition Corp.)
• Inventors: Scarborough, David J., Chambless, Bjorn, Becker, Richard W., Check, Thomas F., Clainos, Deme M., Eng, Maxwell W., Levy, Joel R., Mertz, Adam N., Paajanen, George E., Smith, David R., Smith, John R.
• Primary Examiner(s): Hirl, Joseph P
• Application Number:
  US11/042,544
• Publication Number:
  US 20050273350A1
• Time in Patent Office: 1,058 Days
• Field of Search: 706/60, 706/14, 706/12, 600/545
• US Class Current: 706/60
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/063118 Staff planning in a project environment
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/1053 Employment or hiring
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

An automated employee selection system can use a variety of techniques to provide information for assisting in selection of employees. For example, pre-hire and post-hire information can be collected electronically and used to build an artificial-intelligence based model. The model can then be used to predict a desired job performance criterion (e.g., tenure, number of accidents, sales level, or the like) for new applicants. A wide variety of features can be supported, such as electronic reporting. Pre-hire information identified as ineffective can be removed from a collected pre-hire information. For example, ineffective questions can be identified and removed from a job application. New items can be added and their effectiveness tested. As a result, a system can exhibit adaptive learning and maintain or increase effectiveness even under changing conditions.

Claims

1. An apparatus for assisting in determining the suitability of an individual for employment by an employer, the apparatus comprising:

an electronic data interrogator configured to present a first set of a plurality of questions to the individual;

an electronic answer capturer configured to electronically store the individual'"'"'s responses to at least a selected plurality of the first set of questions presented to the individual;

an electronic predictor responsive to the stored answers and configured to predict at least one post-hire outcome for at least one job performance criterion if the individual were to be employed by the employer, the predictor providing a prediction of the at least one post-hire outcome for the at least one job performance criterion based upon correlations of the stored answers with answers to sets of questions by other individuals for which post-hire information has been collected; and

an electronic results provider providing an output indicative of the prediction of the at least one post-hire outcome for the at least one job performance criterion to assist in determining the suitability of the individual for employment by the employer;

wherein the electronic results provider is configured to provide a hiring recommendation based at least on the at least one post-hire outcome for the at least one job performance criterion;

wherein the electronic results provider is further configured to determine a score for the individual, further configured to determine whether the score is over a threshold, and further configured to identify the individual as a favorable candidate if the score is over the threshold.

2. An apparatus according to claim 1 wherein the post-hire outcome indicates whether the individual is predicted to be eligible for re-hire after termination.

3. An apparatus according to claim 1 wherein the post-hire outcomes indicate whether the individual is predicted to be involuntarily terminated and whether the individual is predicted to be eligible for re-hire after termination.

4. An apparatus according to claim 1 wherein at least one of the predicted outcomes is a predicted value for a continuous variable.

5. An apparatus according to claim 1 wherein the predicted outcome indicates whether the individual will belong to a particular group.

6. An apparatus according to claim 1 wherein at least one of the predicted outcomes is a predicted ranking of the individual for the outcome.

7. An apparatus according to claim 1 wherein at least one of the predicted outcomes indicates a predicted employment tenure for the individual.

8. An apparatus according to claim 1 wherein at least one of the predicted outcomes indicates a predicted number of accidents for the individual.

9. An apparatus according to claim 1 wherein at least one of the predicted outcomes indicates a predicted sales level for the individual.

10. An apparatus according to claim 1 wherein the predictor comprises an artificial intelligence-based prediction system.

11. An apparatus according to claim 1 wherein the data interrogator is located at a first location and the predictor is located at a second location which is remote from the first location.

12. An apparatus according to claim 11 wherein the data interrogator and the predictor are selectively electronically interconnected through a network.

13. An apparatus according to claim 12 wherein the network is the worldwide web.

14. An apparatus according to claim 12 wherein the network is a telephone network.

15. An apparatus according to claim 1 wherein the first set of questions may be varied.

16. An apparatus according to claim 15 wherein the predictor is configured to determine and indicate a lack of a correlation between one or more questions of the first set of questions and at least one of the predicted outcomes, whereby questions which lack the correlation may be discarded or modified.

17. An apparatus according to claim 1 wherein at least one of the predicted outcomes is longevity with an employer and the answers to sets of questions by other individuals comprise answers by employees of the employer for whom longevity has been determined.

18. An apparatus according to claim 1 in which the predictor comprises at least one model which provides a predictor of the probability of the individual exhibiting at least one of the predicted outcomes, the model being based on correlations between the at least one of the predicted outcomes and the answers to questions by the other individuals, including answers by at least some employees of the employer, the model taking at least selected answers of the stored answers as inputs to the model, a probability of the individual exhibiting the at least one of the predicted outcomes being provided as an output of the model.

19. An apparatus according to claim 1 wherein the predictor is responsive to the stored answers and configured to predict plural outcomes if the individual were to be employed by the employer.

20. A method for assessing suitability of persons for employment based on information for hired employees, the method comprising:

collecting pre-hire data comprising pre-hire applicant information for employment applicants;

collecting post-hire data comprising post-hire measures of the job effectiveness of hired employees;

constructing an artificial intelligence model identifying associations of patterns within the pre-hire data associated with patterns of job effectiveness in the post-hire data;

collecting pre-hire information for a new applicant;

applying the artificial intelligence model to the pre-hire information for the new applicant to provide a prediction of the new applicant'"'"'s suitability for employment;

determining a score for the new applicant based at least on the prediction; and

determining whether the score based at least on the prediction is over a threshold.

21. The method of claim 20 further comprising:

collecting post-hire information for the new applicant; and

using at least the pre-hire and post-hire information for the new applicant to refine the artificial intelligence model.

22. The method of claim 20 further comprising:

constructing at least one other artificial intelligence model of a different type; and

assessing the relative effectiveness of the artificial intelligence models at predicting suitability of employees for employment based on actual employment effectiveness of employees hired based on the models.

23. An apparatus for assisting in determining the suitability of an individual for employment by an employer, the apparatus comprising:

means for electronically presenting a first set of a plurality of questions to the individual;

means for electronically storing the individual'"'"'s responses to at least a selected plurality of the first set of questions presented to the individual;

responsive to the stored answers, means for predicting at least one post-hire outcome for at least one job performance criterion if the individual were to be employed by the employer, the means for predicting providing a prediction of the at least one post-hire outcome for the at least one job performance criterion based upon correlations of the at least one characteristic with answers to sets of questions by other individuals and the closeness of the stored answers to such correlations;

means for determining a score for the individual based at least on the prediction of the at least one post-hire outcome for the at least one job performance criterion;

means for determining whether the score meets a threshold; and

means for providing an output indicative of the at least one post-hire outcome for the at least one job performance criterion and whether the score meets the threshold to assist in determining the suitability of the individual for employment by the employer.

24. An artificial intelligence-based system for predicting employee behaviors based on pre-hire information collected for the employee, the system comprising:

an electronic device for presenting an employment application comprising a set of questions to an employment candidate, wherein the electronic device is configured to transmit answers of the employment candidate to a central store of employee information, wherein the central store of employee information comprises information collected for a plurality of candidate employees and a plurality of hired employees;

an artificial intelligence-based model constructed from information collected from the hired employees based on answers provided by the hired employees and employment behaviors observed for the hired employees;

a software system for supplying the answers of the employment candidate to the artificial intelligence-based model to produce predicted employment behaviors for the employment candidate, generate a score for the employment candidate based at least on the predicted employment behaviors, and determine whether the score exceeds a threshold; and

a report generator to produce a hiring recommendation report for the employment candidate based on the predicted employment behaviors of the employment candidate and whether the score exceeds the threshold.

25. A computer-implemented method of predicting employment performance characteristics for a candidate employee based on pre-hire information collected for hired employees, the method comprising:

collecting data indicating pre-hire information for a plurality of the hired employees;

collecting data indicating post-hire outcomes for the hired employees;

constructing an artificial intelligence-based model from the pre-hire information and the post-hire outcomes for the employees;

from the candidate employee, electronically collecting data indicating pre-hire information of the candidate employee;

applying the model to the collected pre-hire information of the candidate employee to generate one or more predicted post-hire outcomes for the candidate employee;

based at least on the one or more predicted post-hire outcomes for the candidate employee, determining a score for the candidate employee; and

determining whether the score for the candidate employee exceeds a threshold.

26. The method of claim 25 wherein collecting data from the candidate employee comprises electronically presenting a set of questions at an electronic device and electronically collecting answers to the questions at the electronic device.

27. The method of claim 25 wherein the pre-hire information comprises one or more pre-hire characteristics and constructing the model comprises:

identifying one or more pre-hire characteristics as ineffective predictors; and

responsive to identifying the pre-hire characteristics as ineffective predictors, omitting the ineffective predictors from the model.

28. The method of claim 25 further comprising:

providing a report indicating applicant flow.

29. The method of claim 25 wherein constructing the model comprises:

constructing a plurality of proposed models, wherein at least two of the models are of different types; and

selecting a proposed model having a superior validity coefficient as the model to be used for the model.

30. The method of claim 25 wherein at least one of the predicted post-hire outcomes is denoted as a probability that a particular value range of a job effective measure will be observed for a candidate employee.

31. The method of claim 25 wherein at least one of the predicted post-hire outcomes is denoted as a value for a continuous variable.

32. The method of claim 25 wherein at least one of the predicted post-hire outcomes is denoted as a relative ranking for an outcome.

33. The method of claim 32 wherein the ranking is relative to other employment candidates.

34. The method of claim 32 wherein the ranking is relative to post-hire outcomes observed for the hired employees.

35. The method of claim 25 further comprising:

storing a relative importance of one or more particular post-hire outcomes; and

generating automated hiring recommendations based on the predicted post-hire outcomes for the candidate employees and the stored relative importance of the post-hire outcomes.

36. The method of claim 25 further comprising:

refining the model based on newly-observed post-hire outcomes.

37. The method of claim 25 wherein the pre-hire information comprises answers to questions on a job application, the method further comprising:

identifying one or more questions as ineffective predictors;

responsive to identifying the questions as ineffective predictors, modifying the job application by removing the questions;

collecting new pre-hire information for additional candidate employees based on the modified job application;

collecting new post-hire information for the additional candidate employees; and

constructing a refined artificial-intelligence model based on the additional pre-hire and post-hire information for the additional candidate employees.

38. The method of claim 37 further comprising:

responsive to determining pre-hire and post-hire information has been collected for a sufficient number of additional employees, providing an indication that a refined model can be constructed.

39. The method of claim 37 further comprising:

providing a report indicating the identified questions are ineffective predictors.

40. The method of claim 37 further comprising:

adding one or more new questions to the modified job application before collecting additional pre-hire information.

41. The method of claim 40 wherein the new questions are composed based on job skills appropriate for a particular job related to the job application.

42. The method of claim 40 further comprising:

evaluating the predictive accuracy of the new questions.

43. An artificial intelligence-based employee performance prediction system comprising:

a set of pre-hire characteristic identifiers;

a set of post-hire outcome identifiers;

a collection of data for employees, wherein the data includes values associated with the pre-hire identifiers and the post-hire identifiers;

an artificial intelligence-based model chosen from a set of candidate models, the artificial intelligence-based model exhibiting superior ability in comparison to other of the candidate models at predicting values associated with the post-hire outcome identifiers based on values associated with the pre-hire characteristic identifiers; and

a score generator configured to generate a score for a candidate based on values predicted by the artificial intelligence-based model processing answers to a series of questions by a candidate employee, wherein the score for the candidate employee is compared to a threshold to determine whether the score is over the threshold.

44. A computer-readable medium having a collection of employment-related data, the data comprising:

pre-hire information for a plurality of employees, wherein the pre-hire information comprises information electronically-collected from an applicant, wherein the information comprises a plurality of pre-hire characteristics;

post-hire information for at least some of the plurality of employees, wherein the information comprises a plurality of post-hire outcomes;

a data structure identifying which of the pre-hire characteristics are effective in predicting a set of one or more of the post-hire outcomes for at least one job performance criterion for a job applicant;

a set of scores for candidate employees, wherein the scores are generated via the set of one or more of the post-hire outcomes; and

an indication of which of the scores for the candidate employees are over a threshold.

45. A method for providing an automated hiring recommendation for a new potential employee, the method comprising:

collecting pre-hire information for potential employees;

storing the pre-hire information for the potential employees in a database;

after hiring a plurality of the potential employees, collecting employment performance information for at least some of the hired employees;

storing the employment performance information collected from the hired employees;

constructing an artificial intelligence-based model based on correlations between the pre-hire information and the employment performance information collected from one or more of the hired employees;

collecting pre-hire information for a new potential employee;

based on the artificial intelligence-based model, providing an automated hiring recommendation for the new potential employee, wherein the providing comprises generating a score for the new potential employee based at least on predictions generated from the artificial intelligence-based model and determining whether the score exceeds a threshold;

after hiring the new potential employee, collecting employment performance information for the new potential employee;

adding the employment performance information for the new potential employee to the database; and

modifying the artificial intelligence-based model based on the pre-hire and employment performance information for the new potential employee.

46. A method for providing an automated hiring recommendation service for an employer, the method comprising:

stationing a plurality of electronic devices at a plurality of employer sites, wherein the electronic devices are configured to accept directly from one or more job applicants answers to questions presented at the electronic devices;

sending the answers of at least one of the job applicants to a remote site for analysis;

applying an artificial intelligence-based predictive model to the answers of the least one of the job applicant to generate an automated hiring recommendation;

generating overall scores for the job applicants based at least on output of the artificial intelligence-based predictive model;

determining whether the overall scores exceed a threshold; and

automatically sending the hiring recommendation to the employer, wherein the hiring recommendation indicates whether the overall scores exceed a threshold.

47. An apparatus for assisting in determining suitability of an individual for employment by an employer, the apparatus comprising:

means for electronically presenting a first set of a plurality of means for questioning to the individual;

means for electronically storing answers of the individual to at least a selected plurality of the first set of means for questioning presented to the individual;

responsive to the stored answers, means for predicting at least one post-hire outcome for at least one job performance criterion if the individual were to be employed by the employer, the means for predicting providing a prediction of the at least one post-hire outcome for the at least one job performance criterion based upon correlations of the stored answers with answers to sets of questions by other individuals for which post-hire information has been collected; and

means for providing an output indicative of the at least one post-hire outcome for the at least one job performance criterion to assist in determining the suitability of the individual for employment by the employer;

wherein the means for providing output is configured to provide a hiring recommendation based at least on the at least one post-hire outcome for the at least one job performance criterion;

wherein the means for providing output is further configured to determine a score for the candidate and further configured to determine whether the score is over a threshold;

wherein the output indicative of the at least one post-hire outcome for the at least one job performance criterion indicates predicted absenteeism for the individual.