Electronic employee selection systems and methods

• US 8,046,251 B2
• Filed: 10/08/2004
• Issued: 10/25/2011
• Est. Priority Date: 08/03/2000
• Status: Expired due to Fees

• Current Assignee: Cadient LLC
• Original Assignee: Kronos Talent Management Incorporated (Seahawk Acquisition Corp.)
• Inventors: Smith, John R., Smith, David R., Becker, Richard W., Clainos, Deme M., Scarborough, David J., Chambless, Bjorn, Check, Thomas F., Eng, Maxwell W., Mertz, Adam N., Paajanen, George E., Levy, Joel R.
• Primary Examiner(s): LOFTIS, JOHNNA RONEE
• Application Number:
  US10/962,191
• Publication Number:
  US 20050246299A1
• Time in Patent Office: 2,573 Days
• Field of Search: 705/7.17, 705/321, 706/21
• US Class Current: 705/7.17
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/063118 Staff planning in a project environment
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/1053 Employment or hiring
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

An automated employee selection system can use a variety of techniques to provide information for assisting in selection of employees. For example, pre-hire and post-hire information can be collected electronically and used to build an artificial-intelligence based model. The model can then be used to predict a desired job performance criterion (e.g., tenure, number of accidents, sales level, or the like) for new applicants. A wide variety of features can be supported, such as electronic reporting. Pre-hire information identified as ineffective can be removed from a collected pre-hire information. For example, ineffective questions can be identified and removed from a job application. New items can be added and their effectiveness tested. As a result, a system can exhibit adaptive learning and maintain or increase effectiveness even under changing conditions.

Claims

1. An electronic assessment system for assessing an individual applicant for employment by an employer, the system comprising:

an electronic applicant terminal operable to present a plurality of questions to the applicant and to receive electronically the applicant'"'"'s responses to the questions;

an applicant screening computer configured to provide applicant results automatically in response to receiving the electronically received responses; and

an electronic report viewer operable to present to the employer a viewable report containing the applicant results,

wherein the computer is configured to generate one or more predictions for one or more behavioral criteria for the applicant based on correlations between the electronically received responses and previous responses given by hired individuals for which on-the-job observations of the one or more behavioral criteria have been collected after the hired individuals were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance; and

the computer is configured to predict the one or more behavioral criteria for the applicant via a model based on correlations between the electronically received responses and previous responses given by hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by hired individuals for which observations of the one or more behavioral criteria have been collected and the observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the electronically received responses from the applicant as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the applicant as output.

2. The system of claim 1, the previous responses characterized by having been collected from the individuals before the individuals were hired into the job for which the one or more behavioral criteria were collected.

3. The system of claim 1, the previous responses characterized by having been collected from the individuals in response to the plurality of questions.

4. An electronic prediction system for assessing an individual applicant for employment by an employer, the system comprising:

an electronic applicant terminal operable to present a plurality of questions to the applicant and to receive electronically responses by the applicant to the questions;

an applicant screening computer responsive to the electronically received responses and operable to predict expected performance for the applicant if the applicant were to be employed by the employer, the computer providing applicant results indicative of expected performance based upon correlations of the electronically received answers with answers to questions by other individuals for which job duty performance information has been collected; and

an electronic report viewer operable to present to the employer a viewable report containing the applicant results;

wherein the expected performance comprises one or more behavioral criteria selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance; and

the computer is configured to predict the expected performance for the applicant via a model based on correlations between the electronically received responses and answers to questions by the other individuals for which information indicating the one or more behavioral criteria has been collected;

wherein the model comprises a neural network trained with the answers to questions by the other individuals for which information indicating the one or more behavioral criteria has been collected and the collected information indicating sales volume;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the electronically received responses from the applicant as input; and

wherein the model, responsive to the inputs, generates a prediction of expected performance for the applicant as output.

5. The apparatus of claim 4 wherein the job duty performance information has been collected electronically.

6. The apparatus of claim 4 wherein the job duty performance information has been stored electronically.

7. The apparatus of claim 4 characterized in that the applicant terminal is configured to communicate with the applicant screening computer over the Internet.

8. An electronic assessment system for assessing an individual for a potential human resources action by an employer, the system comprising:

an electronic terminal operable to present a plurality of questions to the individual and to receive electronically the individual'"'"'s responses to the questions;

a computer configured to provide results automatically in response to receiving the electronically received responses; and

an electronic report viewer operable to present to the employer a viewable report containing the results,

wherein the computer is configured to generate one or more predictions for one or more behavioral criteria for the individual based on correlations between the electronically received responses and previous responses given by hired individuals for which on-the-job observations of the one or more behavioral criteria have been collected after the hired individuals were hired;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance;

and

the computer is configured to predict the one or more behavioral criteria for the individual via a model based on correlations between the electronically received responses and previous responses given by hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by hired individuals for which observations of the one or more behavioral criteria have been collected and the observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the electronically received responses from the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the individual as output.

9. The electronic assessment system of claim 8, characterized in that the potential human resources action is hiring the individual for a job.

10. The electronic assessment system of claim 8, characterized in that the potential human resources action is promoting the individual.

11. An electronic assessment system for assessing an individual for a potential human resources action by an employer comprising:

an electronic terminal operable to present an abbreviated set of questions to the individual and to receive electronically the individual'"'"'s responses to the questions;

a computer configured to provide results automatically in response to receiving the electronically received responses; and

an electronic report viewer operable to present to the employer a viewable report containing the results,

wherein the computer is configured to generate one or more predictions for one or more behavioral criteria for the individual based on correlations between the electronically received responses and previous responses given by hired individuals to a full set of questions, the abbreviated set of questions being selected from the full set of questions, wherein on-the-job observations of the one or more behavioral criteria have been collected for the hired individuals post-hire;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance; and

the computer is configured to predict the one or more behavioral criteria for the individual via a model based on correlations between the electronically received responses and previous responses given by the hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by hired individuals and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the electronically received responses from the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the individual as output.

12. The system of claim 11, further characterized in that the abbreviated set of questions are selected from the full set of questions based on validated correlations between the abbreviated set and the one or more behavioral criteria.

13. A system for assessing an individual applicant for employment by an employer, the system comprising:

an electronic applicant terminal logged on to a website and operable to present a plurality of questions to the applicant and to receive electronically the applicant'"'"'s responses to the questions;

a computer associated with the website and configured to provide applicant results automatically in response to receiving the electronically received responses;

an electronic network connecting the electronic applicant terminal to the computer in accordance with a uniform resource locator associated with website, the uniform resource locator having been entered by the applicant into the electronic applicant terminal; and

an electronic report viewer operable to present to the employer a viewable report containing the applicant results,

wherein the computer is configured to generate one or more predictions for one or more behavioral criteria for the applicant based on correlations between the electronically received responses and previous responses given by hired individuals for which on-the-job observations of the one or more behavioral criteria have been collected after the hired individuals were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance; and

the computer is configured to predict the one or more behavioral criteria for the applicant via a model based on correlations between the electronically received responses and previous responses given by hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by hired individuals and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the electronically received responses from the applicant as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the applicant as output.

14. A method for assessing an individual applicant for employment by an employer, the method comprising:

hosting an employer website identified by a uniform resource locator;

providing via the uniform resource locator access to the website by the applicant at an applicant terminal;

transmitting questions from a computer to the applicant terminal;

receiving at the computer responses from the applicant to the questions transmitted over the Internet;

generating one or more predictions for one or more behavioral criteria for the applicant based on correlations between the received responses and previous responses given by hired individuals for which on-the-job observations of the one or more behavioral criteria have been collected after the hired individuals were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance; and

the computer is configured to predict the one or more behavioral criteria for the applicant via a model based on correlations between the received responses and previous responses given by the hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by hired individuals and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the applicant as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the applicant as output.

15. A method for assessing an individual applicant for employment by an employer, the method comprising:

making a connection over a network between a job applicant telephone and a computer;

transmitting questions from the computer to the job applicant telephone;

receiving at the computer responses from the applicant to the questions transmitted, the responses having been transmitted over the network from the job applicant telephone;

generating one or more predictions for one or more behavioral criteria for the applicant based on correlations between the received responses and previous responses given by hired individuals for which on-the-job observations of the one or more behavioral criteria have been collected after the hired individuals were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance; and

the computer is configured to predict the one or more behavioral criteria for the applicant via a model based on correlations between the received responses and previous responses given by the hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by hired individuals and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the applicant as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the applicant as output.

16. A method of assessing the suitability of an individual for a job action, the method comprising:

making a connection between a computer and a terminal;

receiving at the computer responses entered at the terminal by the individual in response to questions;

scoring the received responses according to correlations between the responses and previous responses given by workers for which on-the-job observations of one or more behavioral criteria have been collected after the workers were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance of the individual; and

the computer is configured to predict, via a model, the one or more behavioral criteria of the individual based on correlations between the received responses and previous responses given by the workers for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by the workers and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the individual as output.

17. The method of claim 16, wherein making the connection comprises logging the individual on to a website.

18. The method of claim 16, wherein the scoring is performed automatically.

19. The method of claim 16, wherein the scoring is performed in real time.

20. The method of claim 16, further comprising displaying a rank order of the individual.

21. The method of claim 16, wherein the previous responses were given by the workers in response to said questions.

22. The method of claim 16, wherein the scoring predicts the turnover potential.

23. The method of claim 16, wherein the scoring provides information on a probability of not terminating early.

24. The method of claim 16, wherein the scoring is indicative of a probability of successful job duty performance.

25. The method of claim 16, wherein the individual is a job applicant and the job action is employment.

26. The method of claim 16, wherein a model for determining the correlations is made before the responses are received.

27. An apparatus for assessing the suitability of an individual for a job action, the apparatus comprising:

means for making a connection between a computer and a terminal;

means for receiving at the computer responses entered at the terminal by the individual in response to questions;

means for scoring the received responses according to correlations between the responses and previous responses given by workers for which on-the-job observations of one or more behavioral criteria have been collected after the hired workers were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance of the individual; and

means for scoring is configured to predict via a model the one or more behavioral criteria of the individual based on correlations between the received responses and previous responses given by workers for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by the workers and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the individual as output.

28. The apparatus of claim 27, wherein the means for making a connection comprises a website.

29. The apparatus of claim 27, wherein the previous responses were given by the workers in response to said questions.

30. The apparatus of claim 27, wherein the terminal comprises a telephone.

31. The apparatus of claim 27, wherein the means for making a connection comprises an Internet and the responses are entered at the terminal by pointing and clicking.

32. The apparatus of claim 27, wherein the computer comprises a testing program.

33. The apparatus of claim 27, further comprising a scoring database.

34. The apparatus of claim 27, wherein a model for determining the correlations is made before the responses are received.

35. A computer program embodied in one or more computer-readable storage media and causing a computer to perform the functions of:

making a connection between a terminal and a computer;

receiving at the computer responses entered at the terminal by an individual in response to questions;

scoring the received responses according to correlations between the responses and previous responses given by workers for which on-the-job observations of one or more behavioral criteria have been collected after the hired workers were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance of the individual; and

the computer program is configured to predict the one or more behavioral criteria of the individual via a model based on correlations between the received responses and previous responses given by workers for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by the workers and the on-the-job observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the individual as output.

36. An electronic prediction system for assessing the suitability of job applicants for an employer, the electronic prediction system comprising:

a plurality of applicant terminals connected to the Internet;

an applicant screening server connected through the Internet to the applicant terminals, the applicant screening server having a testing computer program and storing test meta-data;

an employer website configured to present questions to the applicants at the applicant terminals and to receive applicant responses entered at the applicant terminals in response to presentation of the questions, the questions having been validated by correlating job duty performance ratings of a plurality of hired workers with previous responses given by the workers to the questions;

a scoring system for automatically scoring the applicant responses, the scoring system being validated to predict one or more behavioral criteria selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance;

a scoring database connected to the applicant screening server;

an applicant input system located on the employer'"'"'s premises and configured electronically to receive input from an applicant at the employer'"'"'s premises after the applicant has come to the employer'"'"'s premises and logged on; and

a viewing system for permitting the employer to view applicant results from the electronic prediction system and the applicant'"'"'s rank order;

wherein the scoring system is configured to predict the one or more behavioral criteria for the applicant via a model based on correlations between the received input and the previous responses given by the workers for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by the workers and the observations of the one or more behavioral criteria for the workers;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the applicant as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the applicant as output.

37. An apparatus for assisting in determining suitability of an individual for employment by an employer, the apparatus comprising:

electronic data interrogator means for presenting a first set of a plurality of means for questioning to the individual;

electronic answer capturer means for electronically storing answers of the individual to at least a selected plurality of the first set of means for questioning presented to the individual;

electronic predictor means responsive to the stored answers and for predicting at least one post-hire outcome if the individual were to be employed by the employer, the predictor means providing a prediction of the outcome based upon correlations of the stored answers with answers to sets of means for questioning by other individuals for which post-hire information has been collected, wherein the post-hire information comprises observations of one or more post-hire outcomes selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance for the other individuals; and

electronic results provider means for providing an output indicative of the outcome to assist in determining the suitability of the individual for employment by the employer;

wherein the data interrogator means is located at a first location and the predictor means is located at a second location which is remote from the first location;

the data interrogator means and the predictor means are selectively electronically interconnected through a network; and

the network is an electronic network;

wherein the electronic predictor means is configured to predict the one or more post-hire outcomes for the individual via a model based on correlations between the stored answers and the answers given by the other individuals for which observations of the one or more post-hire outcomes have been collected;

wherein the model comprises a neural network trained with the answers by the other individuals and the observations of the one or more post-hire outcomes for the other individuals;

wherein the model is refined via plural validation cycles validating prediction of the one or more post-hire outcomes by the model;

wherein the model accepts a plurality of inputs comprising the electronically stored answers of the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more post-hire outcomes for the individual as output.

38. One or more computer-readable storage media having computer-executable instructions for performing a method for assessing an individual applicant for employment by an employer, the method comprising:

making a connection over a network between a job applicant telephone and a computer;

transmitting questions from the computer to the job applicant telephone;

receiving at the computer responses from the applicant to the questions transmitted, the responses having been transmitted over the network from the job applicant telephone;

generating one or more predictions for one or more behavioral criteria for the applicant based on correlations between the received responses and previous responses given by hired individuals for which on-the-job observations of the one or more behavioral criteria have been collected after the hired individuals were hired;

wherein the one or more behavioral criteria comprise one or more job performance criteria;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance and the on-the-job observations comprise observations of the one or more behavioral criteria for the hired individuals; and

the computer is configured to predict the one or more behavioral criteria for the applicant via a model based on correlations between the received responses and previous responses given by the hired individuals for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by the hired individuals and the observations of the one or more behavioral criteria for the hired individuals;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the received responses of the applicant as model input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the applicant as output.

39. One or more computer-readable storage media having computer-executable instructions for performing a method of assessing the suitability of an individual for a job action, the method comprising:

making a connection between a computer and a terminal;

receiving at the computer responses entered at the terminal by the individual in response to questions;

scoring the received responses according to correlations between the responses and previous responses given by workers for which on-the-job observations of one or more behavioral criteria have been collected after the workers were hired;

wherein the one or more behavioral criteria are selected from the group consisting of employee delinquency, productivity, tenure, turnover, job effectiveness, and job performance;

wherein the on-the-job observations comprise observations of the one or more behavioral criteria; and

the computer is configured to predict the one or more behavioral criteria of the individual via a model based on correlations between the received responses and previous responses given by the workers for which observations of the one or more behavioral criteria have been collected;

wherein the model comprises a neural network trained with the previous responses given by the workers and the observations of the one or more behavioral criteria;

wherein the model is refined via plural validation cycles validating prediction of the one or more behavioral criteria by the model;

wherein the model accepts a plurality of inputs comprising the responses received from the individual as input; and

wherein the model, responsive to the inputs, generates a prediction of the one or more behavioral criteria for the individual as output.