Operation and method for prediction and management of the validity of subject reported data

• US 9,129,215 B2
• Filed: 12/10/2010
• Issued: 09/08/2015
• Est. Priority Date: 04/02/2001
• Status: Expired due to Term

• Current Assignee: eResearchTechnology, Inc. (Explorer Holdings, Inc.)
• Original Assignee: eResearchTechnology, Inc. (Explorer Holdings, Inc.)
• Inventors: Shiffman, Saul, Engfer, Douglas R., Paty, Jean
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US12/965,719
• Publication Number:
  US 20110082827A1
• Time in Patent Office: 1,733 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

A system for developing and implementing empirically derived algorithms to generate decision rules to predict invalidity of subject reported data and fraud with research protocols in surveys allows for the identification of complex patterns of variables that detect or predict subject invalidity of subject reported data and fraud with the research protocol in the survey. The present invention may also be used to monitor invalidity of subject reported data within a research protocol to determine preferred actions to be performed. Optionally, the invention may provide a spectrum of invalidity, from minor invalidity needing only corrective feedback, to significant invalidity requiring subject removal from the survey. The algorithms and decision rules can also be domain-specific, such as detecting invalidity or fraud among subjects in a workplace satisfaction survey, or demographically specific, such as taking into account gender or age. The algorithms and decision rules may be optimized for the specific sample of subjects being studied.

Claims

1. A computer implemented method for predicting invalidity of subject reported data collected during a current clinical trial or current survey, the method comprising:

analyzing one or more historical validity markers associated with at least one earlier clinical trial or earlier survey, the one or more historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a predictive algorithm for predicting invalidity of subject reported data based at least on the analyzing of the one or more historical validity markers;

translating the predictive algorithm into a decision rule for use with the current clinical trial or current survey, the decision rule specifying an action;

applying the decision rule to one or more validity makers associated with the current clinical trial or current survey; and

determining that the action should be taken based at least on the application of the decision rule to the one or more validity markers associated with the current clinical trial or current survey.

2. The method of claim 1, wherein applying the decision rule comprises comparing the one or more historical validity markers to the decision rule to determine if the action should be taken.

3. The method of claim 1, wherein the decision rule specifies a threshold of invalidity.

4. The method of claim 1, wherein the application of the decision rule to the one or more validity markers is performed by a portable electronic device capable of displaying information and receiving and storing input from a user.

5. The method of claim 1, further comprising creating an evaluability database adapted to store data related to validity of subject reported data.

6. The method of claim 5, wherein the evaluability database is tailored to a condition or medication associated with the current clinical trial or the current survey.

7. The method of claim 1, wherein the analyzing comprises analyzing historical protocol data and wherein the generating is based on quantitative analysis of the one or more historical validity markers and the historical protocol data.

8. The method of claim 1, wherein the analyzing comprises analyzing at least one database containing at least one set of data from a group consisting of historical validity markers and historical protocol data.

9. The method of claim 1, wherein the one or more historical validity markers are from more than one event associated with an earlier clinical trial or earlier survey.

10. The method of claim 1, wherein the one or more historical validity markers are from two or more earlier clinical trials or earlier surveys.

81. The method of claim 1, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

82. The method of claim 1, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

11. A computer implemented method for determining validity of subject reported data collected during a current clinical trial or current survey and determining if action should be taken, the method comprising:

analyzing historical validity markers and historical protocol data, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating, based at least on the analyzing of the historical validity markers and the historical protocol data, an algorithm reflective of the historical validity markers and the historical protocol data;

translating the algorithm into a decision rule for analyzing the validity of subject reported data, the decision rule specifying an action;

obtaining validity markers for at least one event associated with the current clinical trial or the current survey;

applying the decision rule to the validity markers associated with the current clinical trial or current survey; and

determining that the action should be taken based at least on the application of the decision rule to the validity markers associated with the current clinical trial or current survey.

12. The method of claim 11, wherein the decision rule specifies a threshold of invalidity.

13. The method of claim 11, wherein the validity markers are obtained by a portable electronic device capable of displaying information and receiving and storing input from a user.

14. The method of claim 11, wherein the generating employs at least one of a group of multiple linear regression, discriminant function analysis, logistic regression, neural networks, classification trees and regression trees.

15. The method of claim 11, wherein the analyzing comprises analyzing at least one database containing the historical validity markers and the historical protocol data.

16. A computer implemented method for determining validity of subject reported data collected during a current clinical trial or current survey, the method comprising:

analyzing historical validity markers and historical protocol data, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating, based at least on the analyzing of the historical validity markers and the historical protocol data, a spectrum of invalidity representative of the historical validity markers not compliant with the historical protocol data, the spectrum of invalidity specifying a plurality of actions;

obtaining one or more validity markers from at least one event associated with the current clinical trial or the current survey; and

comparing the spectrum of invalidity to the validity markers; and

determining that one of the plurality of actions should be taken based at least on the comparison of the spectrum of invalidity to the validity markers.

17. The method of claim 16, wherein the one of the plurality of actions is a corrective action.

18. The method of claim 17, wherein the one or more validity markers are obtained by a portable electronic device capable of displaying information and receiving and storing input from a user.

28. The medium of claim 17, wherein applying the decision rule comprises comparing the one or more validity markers to the decision rule to determine if the action should be taken.

83. The method of claim 16, wherein one of the plurality of actions comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

19. A computer implemented method for detecting subject fraud, the method comprising:

analyzing historical validity markers and historical protocol data associated with at least one earlier clinical trial or earlier survey, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating, based at least on the analyzing of the historical validity markers and the historical protocol data, a fraud detection algorithm for detecting subject fraud;

translating the fraud detection algorithm into a fraud detection rule for use with a current clinical trial or a current survey, the fraud detection rule specifying an action;

applying the fraud detection rule to one or more validity markers associated with the current clinical trial or the current survey; and

determining that the action should be taken based at least on the application of the fraud detection rule to the one or more validity markers associated with the current clinical trial or the current survey.

84. The method of claim 19 or 20, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

20. A computer implemented method of detecting subject fraud, the method comprising:

analyzing information indicating validity of subject reported data associated with at least one earlier clinical trial or earlier survey, the information comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating, based at least on the analyzing of the information, a fraud detection algorithm for detecting subject fraud during a current clinical trial or current survey;

translating the fraud detection algorithm into a fraud detection rule for use with the current clinical trial or the current survey, the fraud detection rule specifying an action;

applying the fraud detection rule to one or more validity markers associated with subject reported data from the current clinical trial or current survey; and

determining that the action should be taken based at least on the application of the fraud detection rule to the one or more validity markers.

21. The method of claim 20, wherein applying the fraud detection rule comprises comparing the one or more validity markers to the fraud detection rule to determine if the action should be taken.

22. The method of claim 20, wherein the fraud detection rule specifies a threshold of invalidity.

23. The method of claim 21, wherein the analyzing is performed by a portable electronic device capable of displaying information and receiving and storing input from a user.

24. The method of claim 21, further comprising creating an evaluability database adapted to store data related to subject fraud.

25. The method of claim 24, wherein the evaluability database is tailored to a condition or medication associated with the current clinical trial or current survey.

26. The method of claim 21, wherein the analyzing employs at least one database containing at least one data set selected from a group consisting of historical validity markers and historical protocol data.

27. A non-transitory computer-readable medium suitable for use in an electronic device, the medium comprising instructions that, when executed by the electronic device, cause the electronic device to perform:

analyzing one or more historical validity markers associated with at least one earlier clinical trial or earlier survey, the one or more historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating, based at least on the analyzing of the one or more historical validity markers, a predictive algorithm for predicting invalid subject reported data;

translating the predictive algorithm into a decision rule for use with a current clinical trial or current survey, the decision rule specifying an action;

applying the decision rule to one or more validity markers associated with the current clinical trial or the current survey; and

determining that the action should be taken based at least on the application of the decision rule to the one or more validity markers associated with the current clinical trial or the current survey.

29. The medium of claim 27, wherein the instructions, when executed by the electronic device, further cause the electronic device to receive the one or more validity markers from a portable electronic device capable of displaying information and receiving and storing input from a user.

30. The medium of claim 27, wherein the instructions, when executed by the electronic device, further cause the electronic device to create an evaluability database adapted to store data related to information on validity of subject reported data.

31. The medium of claim 27, wherein the one or more historical validity markers are from more than one event.

32. The medium of claim 27, wherein the one or more historical validity markers are from more than one earlier clinical trials or earlier surveys.

33. The medium of claim 27, wherein generating the predictive algorithm is based further on analysis of historical protocol data from more than one earlier clinical trial or earlier survey.

34. The medium of claim 27, wherein the one or more historical validity markers comprise two or more historical validity markers.

85. The medium of claim 27, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

35. A non-transitory computer-readable medium suitable for use in an electronic device, the medium comprising instructions that, when executed by the electronic device, cause the electronic device to perform:

analyzing one or more historical validity markers and historical protocol data associated with one or more earlier clinical trials or earlier surveys, the one or more historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating an algorithm for predicting validity of subject reported data based at least on the analyzing of the one or more historical validity markers and the historical protocol data;

translating the algorithm into a decision rule for analyzing information on validity of subject response data, the decision rule specifying an action;

obtaining validity markers for at least one event associated with a current clinical trial or current survey;

applying the decision rule to the validity markers associated with the current clinical trial or the current survey; and

determining that the action should be taken based at least on the application of the decision rule to the validity markers associated with the current clinical trial or the current survey.

36. The medium of claim 35, wherein the action is a corrective action.

37. The medium of claim 35, wherein the validity markers are obtained by a portable electronic device capable of displaying information and receiving and storing input from a user.

38. The medium of claim 35, wherein the generating employs at least one of a group of multiple linear regression, discriminant function analysis, logistic regression, neural networks, classification trees and regression trees.

86. The medium of claim 35, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

39. A non-transitory computer-readable medium suitable for use in an electronic device, the medium comprising instructions that, when executed by the electronic device, cause the electronic device to perform:

analyzing historical validity markers and historical protocol data associated with one or more earlier clinical trials or earlier surveys, the one or more historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a spectrum of invalidity representative of the historical validity markers not compliant with the historical protocol data based at least on the analyzing of the historical validity markers and the historical protocol data, the spectrum of invalidity specifying a plurality of actions;

obtaining validity markers for at least one event associated with a current clinical trial or current survey;

comparing the spectrum of invalidity to the validity markers; and

determining that one of the plurality of actions should be taken based at least on the comparison of the spectrum of invalidity to the validity markers.

40. The medium of claim 39, wherein at least one of the plurality of actions is a corrective action.

41. The medium of claim 40, wherein the validity markers are obtained by a portable electronic device capable of displaying information and receiving and storing input from a user.

87. The medium of claim 39, wherein one of the plurality of actions comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

42. A non-transitory computer-readable medium suitable for use in an electronic device, the medium comprising instructions that, when executed by the electronic device, cause the electronic device to perform:

analyzing historical validity markers and historical protocol data associated with one or more earlier clinical trials or earlier surveys, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a fraud detection algorithm for detecting subject fraud based at least on the analyzing of the historical validity markers and the historical protocol data;

translating the fraud detection algorithm into a fraud detection rule, the fraud detection rule specifying an action; and

determining that the action should be taken based at least on an application of the fraud detection rule to validity markers associated with a current clinical trial.

43. The medium of claim 42, wherein the action is a corrective action.

44. The medium of claim 42, wherein the validity markers are obtained from a portable electronic device capable of displaying information and receiving and storing input from a user.

45. The medium of claim 42, wherein the generating employs at least one of a group of multiple linear regression, discriminant function analysis, logistic regression, neural networks, classification trees and regression trees.

46. The medium of claim 42, wherein the historical validity markers are from more than one event.

47. The medium of claim 42, wherein the historical validity markers are from more than one earlier clinical trial or earlier survey.

48. The medium of claim 42, wherein the historical protocol data is from more than one earlier clinical trial or earlier survey.

49. The medium of claim 42, wherein the historical protocol data is from at least one clinical trial.

88. The medium of claim 42 or 50, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

50. A non-transitory computer-readable medium suitable for use in an electronic device, the medium comprising instructions that, when executed by the electronic device, cause the electronic device to perform:

analyzing information on the validity of subject reported data associated with one or more earlier clinical trials or earlier surveys, the information comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a fraud detection algorithm for detecting subject fraud based at least on the analyzing of the information; and

translating the fraud detection algorithm into a fraud detection rule for use during a current clinical trial, the fraud detection rule specifying an action; and

determining that the action should be taken based at least on an application of the fraud detection rule to validity markers associated with a current clinical trial or current survey.

51. The medium of claim 50, wherein the information is historical information on validity of subject reported data, from more than one event associated with the one or more earlier clinical trials or earlier surveys.

52. The medium of claim 50, wherein the validity markers associated with the current clinical trial or the current survey comprise:

timeliness of a subject response during the current clinical trial or the current survey;

proximity of data collected during the current clinical trial or the current survey to an expected data range;

a number of missed data entry occasions during the current clinical trial or the current survey;

or

a number of times an entry was changed during the current clinical trial or the current survey.

53. The medium of claim 50, wherein the electronic device is a portable electronic device.

54. A computer system comprising an electronic device, wherein the electronic device comprises a non-transitory computer-readable medium comprising instructions that, when executed by the computer system, cause the computer system to perform:

analyzing historical validity markers and historical protocol data associated with at least one earlier clinical trial or earlier survey, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a predictive algorithm for predicting invalid subject reported data based at least on the analyzing of the historical validity markers and the historical protocol data;

translating the predictive algorithm into a decision rule for use with a current clinical trial or current survey, the decision rule specifying an action; and

determining that the action should be taken based at least on an application of the decision rule to validity markers associated with the current clinical trial or current survey.

55. The computer system of claim 54, wherein the application of the decision rule comprises comparing the decision rule to the validity markers.

56. The computer system of claim 54, wherein the validity markers associated with the current clinical trial or the current survey are obtained via a portable electronic device capable of displaying information and receiving and storing input from a user.

57. The computer system of claim 54, wherein the instructions, when executed by the electronic device, further cause the computer system to create an evaluability database adapted to store data related to information on validity of subject reported data.

58. The computer system of claim 54, wherein the historical validity markers are historical information on the validity of subject reported data from more than one event during the one or more earlier clinical trials or earlier surveys.

59. The computer system of claim 54, wherein the historical validity markers are from more than one earlier clinical trial or earlier survey.

60. The computer system of claim 54, wherein the historical protocol data is from more than one earlier clinical trial or earlier survey.

61. The computer system of claim 54, wherein the analyzing of the historical validity markers and the historical protocol data employs one or more of multiple linear regression, discriminant function analysis, logistic regression, neural networks, classification trees, and regression trees.

89. The system of claim 54, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

62. A computer system comprising an electronic device, wherein the electronic device comprises a non-transitory computer-readable medium comprising instructions that, when executed by the computer system, cause the computer system to perform:

analyzing historical validity markers and historical protocol data associated with one or more earlier clinical trials or earlier surveys, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating an algorithm based at least on the analyzing of the historical validity markers and the historical protocol data;

translating the algorithm into a decision rule for analyzing information on validity of subject response data, the decision rule specifying an action;

obtaining validity markers for at least one event associated with a current clinical trial or current survey;

applying the decision rule to the validity markers; and

determining that the action should be taken based at least on the application of the decision rule to the validity markers for at least one event associated with the current clinical trial or the current survey.

63. The computer system of claim 62, wherein the action is a corrective action.

64. The computer system of claim 62, wherein the validity markers are obtained from a portable electronic device capable of displaying information and receiving and storing input from a user.

65. The computer system of claim 62, wherein the generating employs at least one of a group of multiple linear regression, discriminant function analysis, logistic regression, neural networks, classification trees and regression trees.

90. The system of claim 62, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

66. A computer system comprising an electronic device, wherein the electronic device comprises a non-transitory computer-readable medium comprising instructions that, when executed by the computer system, cause the computer system to perform:

analyzing historical validity markers and historical protocol data associated with one or more earlier clinical trials or earlier surveys, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a spectrum of invalidity representative of the historical validity markers not compliant with the historical protocol data based at least on the analyzing of the historical validity markers and the historical protocol data, the spectrum of invalidity specifying a plurality of actions;

obtaining validity markers for at least one event associated with a current clinical trial or current survey; and

determining that one of the plurality of actions should be taken based at least on a comparison of the spectrum of invalidity to the validity markers.

67. The computer system of claim 66, wherein the one of the plurality of actions is a corrective action.

68. The computer system of claim 67, wherein the validity markers are obtained from a portable electronic device capable of displaying information and receiving and storing input from a user.

91. The system of claim 66, wherein one of the plurality of actions comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

69. A computer system comprising an electronic device, wherein the electronic device comprises a non-transitory computer-readable medium comprising instructions that, when executed by the computer system, cause the computer system to perform:

analyzing historical validity markers and historical protocol data associated with one or more earlier clinical trials or earlier surveys, the historical validity markers comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a fraud detection algorithm for detecting subject fraud based at least on the analyzing of the historical validity markers and the historical protocol data;

translating the fraud detection algorithm into a fraud detection rule, the fraud detection rule specifying an action; and

determining that the action should be taken based at least on an application of the decision rule to validity markers associated with a current clinical trial or a current survey.

70. The computer system of claim 69, wherein the instructions, when executed by the electronic device, further cause the computer system to apply the decision rule to the validity markers.

71. The computer system of claim 69, wherein the instructions, when executed by the electronic device, further cause the computer system to obtain the validity markers from a portable electronic device capable of displaying information and receiving and storing input from a user.

72. The computer system of claim 69, wherein the generating employs at least one of a group of multiple linear regression, discriminant function analysis, logistic regression, neural networks, classification trees and regression trees.

73. The computer system of claim 69, wherein the validity markers associated with the current clinical trial or the current survey comprise:

timeliness of a subject response during the current clinical trial or the current survey;

proximity of data collected during the current clinical trial or the current survey to an expected data range;

a number of missed data entry occasions during the current clinical trial or the current survey;

or

a number of times an entry was changed during the current clinical trial or the current survey.

74. The computer system of claim 69, wherein the validity markers are associated with at least one event from the current clinical trial or the current survey.

75. The computer system of claim 69, wherein the historical protocol data is from more than one earlier clinical trial or earlier survey.

76. The computer system of claim 69, wherein the historical protocol data is from at least one earlier clinical trial.

92. The system of claim 69, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.

77. A computer system comprising an electronic device, wherein the electronic device comprises a non-transitory computer-readable medium comprising instructions that, when executed by the computer system, cause the computer system to perform:

analyzing information on a validity of subject reported data from one or more earlier clinical trials or earlier surveys, the information comprising;

timeliness of a subject response;

proximity of data to an expected data range;

a number of missed data entry occasions;

or

a number of times an entry was changed;

generating a fraud detection algorithm for detecting subject fraud based at least on the analyzing of the information;

translating the fraud detection algorithm into a fraud detection rule, the fraud detection rule specifying an action; and

determining that the action should be taken based at least on an application of the fraud detection rule to validity markers associated with a current clinical trial or a current survey.

78. The computer system of claim 77, wherein the information is historical information on the validity of subject reported data, from more than one event.

79. The computer system of claim 77, wherein the validity markers associated with the current clinical trial or the current survey are from at least one event.

80. The computer system of claim 77, wherein the electronic device is a portable electronic device.

93. The system of claim 77, wherein the action comprises:

removing a subject from the current clinical trial or current survey;

or

sending a message to a staff member associated with the current clinical trial or current survey.