Intelligently interactive profiling system and method

• US 20090319346A1
• Filed: 03/19/2009
• Published: 12/24/2009
• Est. Priority Date: 09/13/2002
• Status: Active Grant

• Current Assignee: Natural Selection, Inc.
• Original Assignee: Natural Selection, Inc.
• Inventors: Fogel, David B., Fogel, Gary B.
• Granted Patent: US 7,958,079 B2
• Time in Patent Office: Days
• US Class Current: 705/11
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06Q 10/06398 Performance of employee with respect to a job function
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

One aspect of the invention is a method for identifying at least one property of data. An example of the method includes receiving data, and making assessments regarding the data. The method also includes applying at least one behavioral operator, and outputting results. The method further comprises receiving feedback concerning system performance. Additionally, the method includes adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

Claims

1. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for identifying at least one property of data, the method comprising the following operations:

receiving data;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

2. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

3. The computer readable storage medium of claim 2, wherein the operations further comprise repeating the receiving data, making assessments, applying, analyzing, outputting, receiving feedback, and adjusting operations.

4. The computer readable storage medium of claim 2, wherein the machine learning method is an evolutionary algorithm.

5. The computer readable storage medium of claim 2, wherein the machine learning method is an evolutionary clustering algorithm.

6. The computer readable storage medium of claim 2, wherein the machine learning method is reinforcement learning.

7. The computer readable storage medium of claim 2, wherein the machine learning method is hill-climbing.

8. The computer readable storage medium of claim 2, wherein the machine learning method is annealing.

9. The computer readable storage medium of claim 2, wherein the machine learning method is meta-heuristics.

10. The computer readable storage medium of claim 2, wherein the operations further comprise receiving user knowledge.

11. The computer readable storage medium of claim 2, wherein the operation of analyzing the data further comprises repeatedly analyzing the data.

12. The computer readable storage medium of claim 2, wherein the operations further comprise performing data integrity testing on a detected anomaly.

13. The computer readable storage medium of claim 2, wherein the operations further comprise generating an alert concerning a detected anomaly.

14. The computer readable storage medium of claim 2, wherein the operations further comprise altering at least one operational rule based on a detected anomaly.

15. The computer readable storage medium of claim 2, wherein the operations further comprise:

proactively generating at least one suggestion;

outputting the at least one generated suggestion; and

soliciting feedback concerning the at least one generated suggestion.

16. The computer readable storage medium of claim 15, wherein the operations further comprise:

receiving feedback concerning at least one of the at least one generated suggestions; and

interpreting the feedback received concerning at least one of the at least one generated suggestions.

17. The computer readable storage medium of claim 15:

wherein the operation of proactively generating at least one suggestion comprises repeatedly generating suggestions; and

wherein the operation of outputting the at least one suggestion comprises outputting each of the generated suggestions.

18. The computer readable storage medium of claim 2, wherein the operation of receiving data comprises repeatedly receiving data.

19. The computer readable storage medium of claim 2, wherein the data comprises commercial data.

20. The computer readable storage medium of claim 2, wherein the data comprises government data.

21. The computer readable storage medium of claim 2, wherein the operation of outputting results comprises:

outputting rules and results; and

outputting information configured to display the rules and results according to user preferences.

22. The computer readable storage medium of claim 2, wherein the operation of outputting results comprises outputting information configured to indicate membership in at least one membership function in a plurality of membership functions.

23. The computer readable storage medium of claim 2, wherein the operation of outputting results comprises outputting information configured to display a plurality of membership functions and an indicator showing a relationship between the results and the membership functions

24. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

wherein the machine learning method involves a neural network, and wherein the at least one parameter is a weight.

25. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

wherein the operation of analyzing the data further comprises developing at least one mathematical model to explain outcomes.

26. The computer readable storage medium of claim 25, wherein the operations further comprise using the at least one mathematical model to delete at least one behavioral operator.

27. The computer readable storage medium of claim 25, wherein the operations further comprise using the at least one mathematical model to modify at least one behavioral rule.

28. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

wherein the operation of analyzing the data further comprises developing at least one mathematical model to explain outcomes;

using the at least one mathematical model to generate at least one new rule; and

using the at least one new rule as one of the behavioral operators.

29. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

receiving feedback regarding the outputted results; and

adding at least one new operational rule based on the feedback regarding the outputted results.

30. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

receiving feedback regarding the outputted results; and

adjusting at least one operational operator based on the feedback received regarding the outputted results.

31. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

outputting results;

receiving feedback regarding the outputted results;

adjusting at least one behavioral operator based on the feedback received regarding the outputted results; and

analyzing the data, wherein the operation of analyzing the data comprises generating at least one machine generated mathematical model to explain outcomes, and comprises detecting if there are any anomalies in the data.

32. The computer readable storage medium of claim 31, wherein the operations further comprise:

proactively generating at least one suggestion;

outputting the at least one generated suggestion; and

soliciting feedback concerning the at least one generated suggestion.

33. The computer readable storage medium of claim 32, wherein the operations further comprise:

receiving feedback concerning at least one of the at least one generated suggestions; and

interpreting the feedback received concerning at least one of the at least one generated suggestions.

34. The computer readable storage medium of claim 32, wherein the operations further comprise:

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

35. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

checking integrity of the data;

applying at least one behavioral operator;

using machine learning to detect if there are any anomalies in the data;

outputting results;

proactively generating at least one suggestion;

outputting the at least one generated suggestion;

soliciting feedback concerning the at least one generated suggestion;

receiving feedback concerning at least one of the at least one generated suggestions; and

interpreting the feedback received concerning at least one of the at least one generated suggestions.

36. The computer readable storage medium of claim 35, wherein the operations further comprise repeating the receiving data, making assessments, applying, outputting, receiving feedback, and adjusting operations.

37. The computer readable storage medium of claim 35, wherein the operations further comprise:

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

38. The computer readable storage medium of claim 35, wherein the operation of using machine learning to detect if there are any anomalies in the data comprises using evolutionary learning.

39. The computer readable storage medium of claim 35, wherein the operations further comprise analyzing the data, and wherein the operation of analyzing the data comprises generating at least one machine generated mathematical model to explain outcomes.

40. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding features and the data;

receiving user knowledge;

applying at least one behavioral operator;

outputting results;

wherein the operation of outputting results comprises outputting information configured to display a plurality of membership functions and an indicator showing a relationship between the results and the membership functions;

receiving feedback regarding the outputted results;

adjusting at least one of the at least one behavioral operators based on the feedback received regarding the outputted results;

adding at least one new behavioral operator based on the feedback received regarding the outputted results;

analyzing the data;

wherein the operation of analyzing the data comprises utilizing a digital processing apparatus to develop at least one mathematical model to explain outcomes;

using the at least one mathematical model to generate at least one new behavioral operator;

including the at least one new behavioral operator in the behavioral operators;

using the at least one mathematical model to delete at least one behavioral operator;

using the at least one mathematical model to modify at least one behavioral operator;

wherein the operation of analyzing the data further comprises detecting if there are any anomalies in the data;

utilizing the digital processing apparatus to perform additional data integrity testing on a detected anomaly;

generating an alert concerning the detected anomaly;

altering at least one behavioral operator based on the detected anomaly;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

proactively generating at least one suggestion;

outputting the at least one generated suggestion;

soliciting feedback concerning the at least one generated suggestion;

receiving at the digital processing apparatus, feedback concerning at least one of the at least one generated suggestions; and

interpreting the feedback received concerning at least one of the at least one generated suggestions.

41. A computer readable storage medium tangibly embodying machine-readable code executable by a digital processing apparatus for evaluating employee performance, the code comprising:

a data integrity module configured to examine integrity of the data, wherein the data regards an employee;

a behavioral operator module configured to generate and evaluate behavioral operators;

an anomaly detection module configured to detect anomalies in the data;

a machine learning module configured to analyze the data; and

an interface/controller module coupled to the data integrity module, the behavioral operators module, the anomaly detection module, and the machine learning module;

wherein the interface/controller module is configured to receive the data.

42. The computer readable storage medium of claim 41, wherein the interface/controller module is further configured to:

proactively generate suggestions;

output the generated suggestions; and

solicit feedback concerning the generated suggestions.

43. The computer readable storage medium of claim 42, wherein the interface/controller module is further configured to interpret feedback concerning the generated suggestions.

44. The computer readable storage medium of claim 41, wherein the interface/controller module is further configured to:

receive feedback concerning system performance; and

adjust parameters based on the feedback received concerning system performance.

45. The computer readable storage medium of claim 41:

wherein the interface/controller module is further configured to output results and to receive feedback regarding the outputted results; and

wherein the behavioral operators module is further configured to adjust the behavioral operators based on the feedback received regarding the outputted results.

46. The computer readable storage medium of claim 41:

wherein the interface/controller module is further configured to output results and to receive feedback regarding outputted results; and

wherein the behavioral operators module is further configured to add new behavioral operators based on the feedback received regarding the outputted results.

47. A profiling system, comprising:

a storage; and

a processor coupled to the storage, wherein the processor is programmed to perform the following operations for evaluating the performance of an employee;

receiving data regarding the employee;

making assessments regarding the data;

applying at least one behavioral operator;

outputting results;

receiving feedback regarding the outputted results;

adjusting at least one behavioral operator based on the feedback received regarding the outputted results;

analyzing the data, wherein the operation of analyzing the data comprises generating at least one machine generated mathematical model to explain outcomes;

proactively generating at least one suggestion;

outputting the at least one generated suggestion; and

soliciting feedback concerning the at least one generated suggestion.

48. A method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

utilizing a digital processing apparatus to analyze the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving at the digital processing apparatus, feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

49. A method for evaluating employee performance, the method comprising the following operations:

receiving data regarding an employee;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving, at a digital processing apparatus, feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

wherein the operation of analyzing the data further comprises developing at least one mathematical model to explain outcomes;

using the at least one mathematical model to generate at least one new rule; and

using the at least one new rule as one of the behavioral operators.

50. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for identifying the risk presented by a shipment, the method comprising the following operations:

receiving data regarding the shipment;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

51. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for identifying the terrorism risk posed by an item, the method comprising the following operations:

receiving data regarding the item;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

52. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for identifying the risk an individual is a terrorist, the method comprising the following operations:

receiving data regarding the individual;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

53. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the risk that a disease is present, the method comprising the following operations:

receiving data regarding a patient;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

54. The computer readable storage medium of claim 53, wherein the disease is cancer.

55. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the risk that a disease is present, the method comprising the following operations:

receiving data regarding a patient;

making assessments regarding the data;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance;

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method;

wherein the operation of analyzing the data further comprises developing at least one mathematical model to explain outcomes;

using the at least one mathematical model to generate at least one new rule; and

using the at least one new rule as one of the behavioral operators.

56. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the chances a person will enjoy dating another person, the method comprising the following operations:

receiving data regarding the other person;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

57. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the risk of fraud presented by a credit transaction, the method comprising the following operations:

receiving data regarding the credit transaction;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

58. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the risk of fraud in a financial filing, the method comprising the following operations:

receiving data regarding the financial filing;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

59. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the chances a person will enjoy a particular movie, the method comprising the following operations:

receiving data regarding the movie;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

60. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the chances a person will want to eat at a particular restaurant, the method comprising the following operations:

receiving data regarding the restaurant;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

61. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for evaluating the chances a person will enjoy a particular vacation destination, the method comprising the following operations:

receiving data regarding the particular vacation destination;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.

62. A computer readable storage medium tangibly embodying a program of machine-readable instructions executable by a digital processing apparatus to perform a method for predicting the onset of a failure in equipment, the method comprising the following operations:

receiving data regarding the equipment;

making assessments regarding the data, wherein the operation of making assessments regarding the data further comprises making assessments regarding features;

applying at least one behavioral operator;

analyzing the data, wherein the operation of analyzing the data comprises detecting if there are any anomalies in the data;

outputting results;

receiving feedback concerning system performance; and

adjusting at least one parameter based on the feedback received concerning system performance, wherein the at least one parameter is a parameter of a machine learning method.