Electronic employee selection systems and methods

• US 20020046199A1
• Filed: 08/02/2001
• Published: 04/18/2002
• Est. Priority Date: 08/03/2000
• Status: Active Grant

• Current Assignee: Cadient LLC
• Original Assignee: Unicru, Inc. (Seahawk Acquisition Corp.)
• Inventors: Smith, David R., Smith, John R., Becker, Richard W., Clainos, Deme M., Scarborough, David J., Chambless, Bjorn, Check, Thomas F., Eng, Maxwell W., Mertz, Adam N., Paajanen, George E., Levy, Joel R.
• Granted Patent: US 7,080,057 B2
• Time in Patent Office: Days
• US Class Current: 706/21
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/063118 Staff planning in a project environment
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/1053 Employment or hiring
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

An automated employee selection system can use a variety of techniques to provide information for assisting in selection of employees. For example, pre-hire and post-hire information can be collected electronically and used to build an artificial-intelligence based model. The model can then be used to predict a desired job performance criterion (e.g., tenure, number of accidents, sales level, or the like) for new applicants. A wide variety of features can be supported, such as electronic reporting. Pre-hire information identified as ineffective can be removed from a collected pre-hire information. For example, ineffective questions can be identified and removed from a job application. New items can be added and their effectiveness tested. As a result, a system can exhibit adaptive learning and maintain or increase effectiveness even under changing conditions.

Claims

1. An apparatus for assisting in determining the suitability of an individual for employment by an employer, the apparatus comprising:

an electronic data interrogator operable to present a first set of a plurality of questions to the individual;

an electronic answer capturer operable to electronically store the individual'"'"'s responses to at least a selected plurality of the first set of questions presented to the individual;

an electronic predictor responsive to the stored answers and operable to predict at least one post-hire outcome if the individual were to be employed by the employer, the predictor providing a prediction of the outcome based upon correlations of the stored answers with answers to sets of questions by other individuals for which post-hire information has been collected; and

an electronic results provider providing an output indicative of the outcome to assist in determining the suitability of the individual for employment by the employer.

2. An apparatus according to claim 1 wherein the post-hire outcome indicates whether the individual is predicted to be eligible for re-hire after termination.

3. An apparatus according to claim 1 wherein the post-hire outcome indicates whether the individual is predicted to be involuntarily terminated.

4. An apparatus according to claim 1 wherein the post-hire outcomes indicate whether the individual is predicted to be involuntarily terminated and whether the individual is predicted to be eligible for re-hire after termination.

5. An apparatus according to claim 1 wherein at least one of the predicted outcomes is a predicted probability that a particular outcome value range will be observed.

6. An apparatus according to claim 1 wherein at least one of the predicted outcomes is a predicted value for a continuous variable.

7. An apparatus according to claim 1 wherein the predicted outcome is a predicted range of values for a continuous variable.

8. An apparatus according to claim 1 wherein the predicted outcome indicates whether the individual will belong to a particular group.

9. An apparatus according to claim 1 wherein at least one of the predicted outcomes is a predicted ranking of the individual for the outcome.

10. An apparatus according to claim 1 wherein at least one of the predicted outcomes indicates a predicted employment tenure for the individual.

11. An apparatus according to claim 1 wherein at least one of the predicted outcomes indicates a predicted number of accidents for the individual.

12. An apparatus according to claim 1 wherein at least one of the predicted outcomes indicates a predicted sales level for the individual.

13. An apparatus according to claim 1 wherein the predictor comprises an artificial intelligence-based prediction system.

14. An apparatus according to claim 1 wherein the data interrogator is located at a first location and the predictor is located at a second location which is remote from the first location.

15. An apparatus according to claim 14 wherein the data interrogator and the predictor are selectively electronically interconnected through a network.

16. An apparatus according to claim 15 wherein the network is the worldwide web.

17. An apparatus according to claim 15 wherein the network is a telephone network.

18. An apparatus according to claim 15 wherein the network is a satellite network.

19. An apparatus according to claim 1 wherein the first set of questions may be varied.

20. An apparatus according to claim 19 wherein the predictor is operable to determine and indicate a lack of a correlation between one or more questions of the first set of questions and at least one of the predicted outcomes, whereby questions which lack the correlation may be discarded or modified.

21. An apparatus according to claim 1 wherein at least one of the predicted outcomes is longevity with an employer and the answers to sets of questions by other individuals comprise answers by employees of the employer for whom longevity has been determined.

22. An apparatus according to claim 1 in which the predictor comprises at least one model which provides a predictor of the probability of the individual exhibiting at least one of the predicted outcomes, the model being based on correlations between the at least one of the predicted outcomes and the answers to questions by the other individuals, including answers by at least some employees of the employer, the model taking at least selected answers of the stored answers as inputs to the model, a probability of the individual exhibiting the at least one of the predicted outcomes being provided as an output of the model.

23. An apparatus according to claim 22 wherein the model comprises at least one neural network.

24. An apparatus according to claim 1 wherein the predictor is responsive to the stored answers and operable to predict plural outcomes if the individual were to be employed by the employer.

26. The method of claim 25 further comprising:

collecting post-hire information for the new applicant; and

using at least the pre-hire and post-hire information for the new applicant to refine the artificial intelligence model.

27. The method of claim 25 further comprising:

constructing at least one other artificial intelligence model of a different type; and

assessing the relative effectiveness of the artificial intelligence models at predicting suitability of employees for employment based on actual employment effectiveness of employees hired based on the models.

25. A method for assessing suitability of persons for employment based on information for hired employees, the method comprising:

collecting pre-hire applicant information for hired employees before they are hired;

collecting post-hire measures of the job effectiveness of hired employees;

constructing an artificial intelligence model identifying associations of patterns within the pre-hire data associated with patterns of job effectiveness in the post-hire data;

collecting pre-hire information for a new applicant; and

applying the artificial intelligence model to the pre-hire information for the new applicant to provide a prediction of the new applicant'"'"'s suitability for employment.

28. An apparatus for assisting in determining the suitability of an individual for employment by an employer, the apparatus comprising:

means for electronically presenting a first set of a plurality of questions to the individual;

means for electronically storing the individual'"'"'s responses to at least a selected plurality of the first set of questions presented to the individual;

responsive to the stored answers, means for predicting at least one post-hire outcome if the individual were to be employed by the employer, the means for predicting providing a prediction of the outcome based upon correlations of the at least one characteristic with answers to sets of questions by other individuals and the closeness of the stored answers to such correlations; and

means for providing an output indicative of the outcome to assist in determining the suitability of the individual for employment by the employer.

29. An artificial intelligence-based system for predicting employee behaviors based on pre-hire information collected for the employee, the system comprising:

an electronic device for presenting an employment application comprising a set of questions to an employment candidate, wherein the electronic device is operable to transmit answers of the employment candidate to a central store of employee information, wherein the central store of employee information comprises information collected for a plurality of candidate employees and a plurality of hired employees;

an artificial intelligence-based model constructed from information collected from the hired employees based on answers provided by the hired employees and employment behaviors observed for the hired employees;

a software system for supplying the answers of the employment candidate to the artificial intelligence-based model to produce predicted employment behaviors for the employment candidate; and

a report generator to produce a hiring recommendation report for the employment candidate based on the predicted employment behaviors of the employment candidate.

30. A computer-implemented method of predicting employment performance characteristics for a candidate employee based on pre-hire information collected for hired employees, the method comprising:

collecting data indicating pre-hire information for a plurality of the hired employees;

collecting data indicating post-hire outcomes for the hired employees;

constructing an artificial intelligence-based model from the pre-hire information and the post-hire outcomes for the employees;

from the candidate employee, electronically collecting data indicating pre-hire information of the candidate employee; and

applying the model to the collected pre-hire information of the candidate employee to generate one or more predicted post-hire outcomes for the candidate employee.

31. The method of claim 30 wherein collecting data from the candidate employee comprises electronically presenting a set of questions at an electronic device and electronically collecting answers to the questions at the electronic device.

32. The method of claim 30 wherein the pre-hire information comprises one or more pre-hire characteristics and constructing the model comprises:

identifying one or more pre-hire characteristics as ineffective predictors; and

responsive to identifying the pre-hire characteristics as ineffective predictors, omitting the ineffective predictors from the model.

33. The method of claim 30 further comprising:

providing a report indicating applicant flow.

34. The method of claim 30 wherein constructing the model comprises:

constructing a plurality of proposed models, wherein at least two of the models are of different types; and

selecting a superior proposed model as the model to be used.

35. The method of claim 34 wherein at least two of the proposed models are different neural network types.

36. The method of claim 35 wherein the two proposed models are both feed-forward neural networks.

37. The method of claim 35 wherein the two proposed models are chosen from the following:

back propagation, conjugate gradients, quasi-Newton, Levenberg-Marquardt, quick propagation, delta-bar-delta, linear, radial basis function, and generalized regression network.

38. The method of claim 30 wherein at least one of the predicted post-hire outcomes is denoted as a probability that a particular value range of a job effective measure will be observed for a candidate employee.

39. The method of claim 30 wherein at least one of the predicted post-hire outcomes is denoted as a value for a continuous variable.

40. The method of claim 30 wherein at least one of the predicted post-hire outcomes is denoted as a relative ranking for an outcome.

41. The method of claim 40 wherein the ranking is relative to other employment candidates.

42. The method of claim 40 wherein the ranking is relative to the hired employees.

43. The method of claim 30 further comprising:

storing a relative importance of one or more particular post-hire outcomes; and

generating automated hiring recommendations based on the predicted post-hire outcomes for the candidate employees and the importance of the post-hire outcomes.

44. The method of claim 30 further comprising:

refining the model based on newly-observed post-hire outcomes.

45. The method of claim 30 wherein the pre-hire information comprises answers to questions on a job application, the method further comprising:

identifying one or more questions as ineffective predictors;

responsive to identifying the questions as ineffective predictors, modifying the job application by removing the questions;

collecting new pre-hire information for additional candidate employees based on the modified job application;

collecting new post-hire information for the additional candidate employees; and

constructing a refined artificial-intelligence model based on the additional pre-hire and post-hire information for the additional candidate employees.

46. The method of claim 45 further comprising:

responsive to determining pre-hire and post-hire information has been collected for a sufficient number of additional employees, providing an indication that a refined model can be constructed.

47. The method of claim 45 further comprising:

providing a report indicating the identified questions are ineffective predictors.

48. The method of claim 45 further comprising:

adding one or more new questions to the modified job application before collecting additional pre-hire information.

49. The method of claim 48 wherein the new questions are composed based on job skills appropriate for a particular job related to the job application.

50. The method of claim 48 further comprising:

evaluating the effectiveness of the new questions.

51. An artificial intelligence-based employee performance prediction system comprising:

a set of pre-hire characteristic identifiers;

a set of post-hire outcome identifiers;

a collection of data for employees, wherein the data includes values associated with the pre-hire identifiers and the post-hire identifiers; and

an artificial intelligence-based model chosen from a set of candidate models, the artificial intelligence-based model exhibiting superior ability at predicting values associated with the post-hire outcome identifiers based on values associated with the pre-hire characteristic identifiers in comparison to the other candidate models.

52. A computer-readable medium having a collection of employment-related data, the data comprising:

pre-hire information for a plurality of employees, wherein the pre-hire information comprises information electronically-collected from an applicant, wherein the information comprises a plurality of pre-hire characteristics;

post-hire information for at least some of the plurality of employees, wherein the information comprises a plurality of post-hire outcomes; and

a data structure identifying which of the pre-hire characteristics are effective in predicting a set of one or more of the post-hire outcomes for a job applicant.

53. A method for providing an automated hiring recommendation for a new potential employee, the method comprising:

collecting pre-hire information for potential employees;

storing the pre-hire information for the potential employees in a database;

after hiring a plurality of the potential employees, collecting employment performance information for at least some of the hired employees;

storing the employment performance information collected from the hired employees;

constructing an artificial intelligence-based model based on correlations between the pre-hire information and the employment performance information collected from one or more of the hired employees;

collecting pre-hire information for a new potential employee;

based on the artificial intelligence-based model, providing an automated hiring recommendation for the new potential employee;

after hiring the new potential employee, collecting employment performance information for the new potential employee;

adding the employment performance information for the new potential employee to the database; and

modifying the artificial intelligence-based model based on the pre-hire and employment performance information for the new potential employee.

54. A method for providing an automated hiring recommendation service for an employer, the method comprising:

stationing a plurality of electronic devices at a plurality of employer sites, wherein the electronic devices are operable to accept directly from one or more job applicants answers to questions presented at the electronic devices;

sending the answers of at least one of the job applicants to a remote site for analysis;

applying an artificial intelligence-based predictive model to the answers of the least one of the job applicant to generate an automated hiring recommendation; and

automatically sending the hiring recommendation to the employer.