Method for assessing the accuracy of test results

• US 20070179398A1
• Filed: 08/24/2005
• Published: 08/02/2007
• Est. Priority Date: 08/24/2005
• Status: Active Grant

• Current Assignee: Audiology Incorporated
• Original Assignee: Audiology Incorporated
• Inventors: Margolis, Robert
• Granted Patent: US 7,704,216 B2
• Time in Patent Office: Days
• US Class Current: 600/559
• CPC Class Codes: A61B 5/12 Audiometering
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04R 25/70 Adaptation of deaf aid to hearing loss, e.g. initial electronic fitting

Abstract

A method for developing and validating a quality assessment for a test is provided. A test method is also provided, which assesses the accuracy of a test. The methods include performing a behavioral test on a subject to measure a psychometric quantity, measuring one or more quality indicators, the quality indicators being any variables that are correlated with test accuracy, obtaining an independent measure of test accuracy, and deriving a predictive formula for estimating test accuracy from the quality indicators.

Claims

1. A method for developing a quality assessment for a test, comprising:

performing a behavioral test on a subject to measure a psychometric quantity;

measuring the psychometric quantity to provide a test measurement of the psychometric quantity;

measuring one or more quality indicators to provide one or more quality indicator measurements, the quality indicator being any variable that gives an indication of test accuracy;

obtaining an independent measurement of the psychometric quantity for the subject;

calculating a difference between the test measurement and the independent measurement;

determining a relationship between the quality indicator measurements and the difference between the test measurement and the independent measurement; and

using the relationship to predict a difference between the test result and the independent measure of the psychometric quantity, the difference being a quantitative measure of test accuracy.

2. The method of claim 1 wherein the performing a behavioral test comprises:

providing stimuli to the subject; and

receiving, from the subject, responses to the stimuli.

3. The method of claim 1 wherein the measuring one or more quality indicators comprises measuring quality indicators based upon variables seen when the stimuli is provided or when the responses are received.

4. The method of claim 1 wherein the obtaining an independent measurement comprises performing a method for measuring the psychometric quantity that is regarded as an accurate method for measuring that psychometric quantity.

5. The method of claim 4 wherein the obtaining an independent measurement comprises performing a standardized test method for measuring the psychometric quantity.

6. The method of claim 1 wherein the determining a relationship between the quality indicator measurements and the difference between the test measurement and the independent measurement comprises performing a multiple regression for the quality indicator measurements and the difference between the test measurement and the independent measurement.

7. The method of claim 1 wherein the using the relationship to predict a difference between the test result and the independent measure of the psychometric quantity comprises deriving a predictive formula based on the relationship, the predictive formula being used to calculate an estimated test accuracy from the quality indicators.

8. The method of claim 7 wherein the predictive formula is derived from a multiple regression performed for the quality indicator measurements and the difference between the test measurement and the independent measurement.

9. The method of claim 8 wherein the predictive formula is QA=Q_i−

Q_m=f(QI_n).

10. The method of claim 1 wherein at least one of the quality indicators is a behavior.

11. The method of claim 10 wherein at least one of the quality indicators is a subject behavior.

12. A test method, comprising:

performing a behavioral test on a subject to measure a psychometric quantity;

measuring the psychometric quantity to provide a test measurement of the psychometric quantity;

measuring one or more quality indicators to provide one or more quality indicator measurements, the quality indicator being any variable that gives an indication of test accuracy; and

calculating an estimated test accuracy using the one or more quality indicator measurements.

13. The method of claim 12 wherein the performing a behavioral test comprises:

providing stimuli to the subject; and

receiving, from the subject, responses to the stimuli.

14. The method of claim 12 wherein the measuring one or more quality indicators comprises measuring quality indicators based upon variables seen when the stimuli is provided or when the responses are received.

15. The method of claim 12 wherein the calculating an estimated test accuracy comprises inputting the one or more quality indicator measurements into a predictive formula, the predictive formula being derived from a multiple regression performed for quality indicator measurements and a difference between a test measurement and an independent measurement obtained for a previously tested subject.

16. The method of claim 12 wherein the calculating an estimated test accuracy comprises inputting the one or more quality indicator measurements into a predictive formula, the predictive formula being derived by a method comprising:

performing a behavioral test on a subject to measure a psychometric quantity;

measuring the psychometric quantity to provide a test measurement of the psychometric quantity;

measuring one or more quality indicators to provide one or more quality indicator measurements, the quality indicator being any variable that gives an indication of test accuracy;

obtaining an independent measurement of the psychometric quantity for the subject;

calculating a difference between the test measurement and the independent measurement;

determining a relationship between the quality indicator measurements and the difference between the test measurement and the independent measurement; and

deriving a predictive formula based on the relationship, the predictive formula being used to calculate an estimated test accuracy from the quality indicator measurements.

17. The method of claim 16 wherein the determining a relationship between the quality indicator measurements and the difference between the test measurement and the independent measurement comprises performing a multiple regression for the quality indicator measurements and the difference between the test measurement and the independent measurement.

18. The method of claim 17 wherein the predictive formula is derived from the multiple regression.

19. The method of claim 12 further comprising comparing the estimated test accuracy to other estimated test accuracy measurements obtained for a population group.

20. The method of claim 19 wherein the comparing the estimated test accuracy comprises presenting the estimated test accuracy as a percentile rank that compares with the other estimated test accuracy measurements.

21. The method of claim 19 wherein the population group comprises one or more subjects having one or more characteristics in common.

22. The method of claim 12 further comprising comparing each quality indicator measurement to corresponding quality indicator measurements obtained for a population group.

23. The method of claim 22 wherein the comparing each quality indicator measurement comprises presenting each quality indicator measurement as a percentile rank that compares with the corresponding quality indicator measurements.

24. The method of claim 22 wherein the population group comprises one or more subjects having one or more characteristics in common.

25. A hearing test method, comprising:

performing a hearing test on a subject to measure the subject'"'"'s hearing;

measuring at least one quality indicator to provide a quality indicator measurement, the quality indicator being any variable that gives an indication of reliability of the hearing test; and

calculating an estimated test accuracy using the one or more quality indicator measurements.

26. The method of claim 25 wherein the quality indicators are selected from the group consisting of masker alert rate, time per trial, average number of trials for a threshold, total elapsed time, false alarm rate, average test-retest difference, quality check fail rate, air-bone gap >

50 dB and air-bone gap <

−

10 dB.

27. The method of claim 25 wherein the performing a hearing test to measure a subject'"'"'s hearing comprises:

providing, to the subject, a sequence of acoustic stimuli;

receiving, from the subject, responses from the sequence of acoustic stimuli; and

identifying hearing threshold levels based on the subject'"'"'s responses.

28. The method of claim 25 further comprising adaptively selecting the acoustic stimuli of the sequence based upon the subject'"'"'s responses.

29. The method of claim 25 wherein the measuring at least one quality indicator comprises measuring at least one quality indicator based upon a variable seen when the acoustic stimuli provided and the responses are received.

30. The method of claim 25 wherein the performing a hearing test to measure a subject'"'"'s hearing comprises:

presenting air-conducted acoustic stimuli to the subject;

collecting responses from the subject, the responses being based on the air-conducted acoustic stimuli;

identifying threshold levels based on the subject'"'"'s responses to the air-conducted acoustic stimuli;

presenting bone-conducted acoustic stimuli to the subject;

collecting responses from the subject, the responses being based on the bone-conducted acoustic stimuli; and

identifying threshold levels based on the subject'"'"'s responses to the bone-conducted acoustic stimuli.

31. The method of claim 30 wherein the measuring at least one quality indicator comprises measuring at least one quality indicator based upon a variable seen during presentation of and responses to the air-conducted and bone-conducted acoustic stimuli.

32. The method of claim 25 wherein the step of performing a hearing test to measure a subject'"'"'s hearing comprises:

presenting a plurality of tone frequencies to the subject;

identifying a first ear and a second ear of the subject based on a threshold level of a tone frequency, the first ear having a lower threshold at the first tone frequency, and the second ear having a higher threshold at the first tone frequency;

performing air-conduction tests on the subject'"'"'s first ear and then second ear; and

performing bone-conduction tests on each ear.

33. The method of claim 25 wherein the calculating an estimated test accuracy comprises inputting the one or more quality indicator measurements into a predictive formula, the predictive formula being derived from a multiple regression performed for quality indicator measurements and a difference between a test measurement and an independent measurement obtained for a previously tested subject.

34. The method of claim 25 wherein the calculating an estimated test accuracy comprises inputting the one or more quality indicator measurements into a predictive formula, the predictive formula being derived by a method comprising:

performing a hearing test on a subject to measure hearing threshold levels;

measuring the threshold levels to provide a test measurement;

measuring one or more quality indicators to provide one or more quality indicator measurements, the quality indicator being any variable that gives an indication of test accuracy;

obtaining an independent measurement of the hearing threshold levels for the subject;

calculating a difference between the test measurement and the independent measurement;

determining a relationship between the quality indicator measurements and the difference between the test measurement and the independent measurement; and

deriving a predictive formula based on the relationship, the predictive formula being used to calculate an estimated test accuracy from the quality indicator measurements.

35. The method of claim 34 wherein the determining a relationship between the quality indicator measurements and the difference between the test measurement and the independent measurement comprises performing a multiple regression for the quality indicator measurements and the difference between the test measurement and the independent measurement.

36. The method of claim 25 further comprising comparing the estimated test accuracy to other estimated test accuracy measurements obtained for a population group.

37. The method of claim 36 wherein the comparing the estimated test accuracy comprises presenting the estimated test accuracy as a percentile rank that compares with the other estimated test accuracy measurements.

38. The method of claim 37 wherein the population group comprises one or more subjects having one or more characteristics in common.

39. The method of claim 38 wherein the characteristic is a hearing characteristic.

40. The method of claim 25 further comprising comparing each quality indicator measurement to corresponding quality indicator measurements obtained for a population group.

41. The method of claim 40 wherein the comparing each quality indicator measurement comprises presenting each quality indicator measurement as a percentile rank that compares with the corresponding quality indicator measurements.

42. The method of claim 41 wherein the population group comprises one or more subjects having one or more characteristics in common.

43. The method of claim 42 wherein the characteristic is a hearing characteristic.

44. The method of claim 25 wherein the hearing test is an automated test carried out by a computer program.

45. A method for testing hearing of a subject, the method comprising:

performing an automated hearing test; and

tracking quality indicators during the hearing test, wherein the quality indicators include a masker alert rate, time per trial, average number of trials for a threshold, total elapsed time, false alarm rate, average test-retest difference, quality check fail rate, air-bone gap >

50 dB and air-bone gap <

−

10 dB.