Expert system for assessing accuracy of models of physical phenomena and for selecting alternate models in the presence of noise

• US 5,373,456 A
• Filed: 11/02/1992
• Issued: 12/13/1994
• Est. Priority Date: 11/02/1992
• Status: Expired due to Fees

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Ferkinhoff, David J., Keay, Kathleen D., Gong, Kai F., Nardone, Steven C.
• Primary Examiner(s): Ramirez, Ellis B.
• Assistant Examiner(s): Shah, Kamini S.
• Application Number:
  US07/976,349
• Time in Patent Office: 771 Days
• Field of Search: 364/516, 364/517, 364/552, 364/554, 364/578, 382/39, 382/36, 382/37, 382/38, 382/25, 382/16, 382/10, 356/16, 356/17, 356/3, 235/400, 235/401, 235/411, 395/50
• US Class Current: 703/2
• CPC Class Codes: G01S 3/8006 Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals

Abstract

A system for providing an iterative method of assessing accuracy of selec models of physical phenomena and for determining selection of alternate models in response to a data sequence in the presence of noise. Initially, a residual sequence is generated reflecting difference values between in response to said data sequence and an expected data sequence as would be represented by a selected model. Feature estimate values of a plurality of predetermined data features in the residual sequence are then generated. In response to the feature estimate values, a threshold value is generated for each feature at an estimated ratio of data to noise. Probability values are generated in response to the threshold value, representing the likelihood that the feature exists in the data sequence, does not exist in the data sequence, and that the existence or non-existence in the data sequence is not determinable. Finally, a model is selected in response to the probability values for use during a subsequent iteration.

Claims

1. A system for assessing accuracy of selected models of physical phenomena and for determining selection of alternate models in response to a data sequence representing a sequence of values of a signal in the presence of noise comprising:

a residual value generator for generating residual data values reflecting difference values in response to the data sequence and an expected data sequence that is generated in response to a selected model;

a feature estimate value generator for generating feature estimate values of a plurality of predetermined data features in the residual sequence generated by the residual value generator;

a threshold value determination element for generating, in response to the feature estimate values generated by the feature estimate value generator, a threshold value for each feature at an estimated ratio of data to noise;

a feature probability value generator for generating, in response to the threshold value, probability values representing the likelihood that the feature exists in the data sequence, does not exist in the data sequence, and that the existence or non-existence in the data sequence is not determinable;

a model selector for selecting a model in response to the probability values generated by the feature probability value generator; and

a controller for controlling the operations of residual value generator, the feature estimate value generator, the threshold value determination element, the feature probability value generator and the model selector in a plurality of iterations, during each iteration the residual value generator using the model selected by the model selection module during the previous iteration.

2. A system as defined in claim 1 in which said threshold value determination element includes:

a test statistic value generator for generating, for each feature, a test statistic value in response to a difference between the residual sequence and the residual sequence without regard to the feature, and in further response to a predetermined noise value; and

a threshold value generator for generating said threshold value as a function of said test statistic.

3. A system as defined in claim 2 in which said function is selected so that a statistical distribution of the residual values with regard to the feature and the corresponding statistical distribution of the residual values without regard to the feature are the same.

4. A system as defined in claim 1 in which said feature probability value generator includes:

a feature present probability value generator for generating, in response to a statistical distribution function representing a statistical distribution of the residual values with regard to the feature and the corresponding statistical distribution of the residual values without regard to the feature, and the threshold value, a feature present probability value representing a likelihood that the feature is present in the data sequence;

a feature absence probability value generator for generating, in response to the statistical distribution function and the threshold value, a feature absent probability value representing a likelihood that the feature is absent from the data sequence; and

a feature indeterminable probability value generator for generating, in response to the feature present probability value and the feature absent probability value, a feature indeterminable probability value representing a likelihood that the presence or absence of the feature in the data sequence is indeterminable.

5. A system as defined in claim 4 in which the feature probability value generator further includes:

a feature detection probability generator for generating a feature detect probability value in response to the statistical distribution function and the threshold value; and

a feature detection failure probability generator for generating a feature detection failure probability value in response to the statistical distribution function and the threshold value.

6. A system as defined in claim 5 in which the feature present probability value generator generates said feature present probability value as a difference between the feature detect probability value and the feature detection failure probability value.

7. A system as defined in claim 5 in which the feature detection probability generator and said feature detection failure probability generator further generate a reference feature detect reference probability value and a reference feature detection failure probability value in response to a reference statistical distribution function and a threshold value, the feature absence probability value generator generating the feature absence probability value in response to the reference feature detect reference probability value and a reference feature detection failure probability value and the feature detection failure probability value.

8. A method of assessing accuracy of selected models of physical phenomena and for determining selection of alternate models in response to a data sequence representing value of a signal in the presence of noise, the method comprising the steps of iteratively:

generating a residual sequence reflecting difference values in response to the data sequence and an expected data sequence as would be represented by a selected model;

generating feature estimate values of a plurality of predetermined data features in the residual sequence;

generating, in response to the feature estimate values, a threshold value for each feature at an estimated ratio of data to noise;

generating, in response to the threshold value, probability values representing the likelihood that the feature exists in the data sequence, does not exist in the data sequence, and that the existence or non-existence in the data sequence is not determinable;

selecting a model in response to the probability values for use during a subsequent iteration.

9. A method as defined in claim 8 in which said threshold value generating step includes the steps of:

generating, for each feature, a test statistic value in response to a difference between the residual sequence and the residual sequence without regard to the feature, and in further response to a predetermined noise value; and

generating said threshold value as a function of said test statistic.

10. A method as defined in claim 9 in which said function is selected so that a statistical distribution of the residual values with regard to the feature and the corresponding statistical distribution of the residual values without regard to the feature have a selected relationship.

11. A method as defined in claim 8 in which said feature probability value generating step includes the steps of:

for generating, in response to a statistical distribution function representing a statistical distribution of the residual values with regard to the feature and the corresponding statistical distribution of the residual values without regard to the feature, and the threshold value, a feature present probability value representing a likelihood that the feature is present in the data sequence;

generating, in response to the statistical distribution function and the threshold value, a feature absent probability value representing a likelihood that the feature is absent from the data sequence; and

generating, in response to the feature present probability value and the feature absent probability value, a feature indeterminable probability value representing a likelihood that the presence or absence of the feature in the data sequence is indeterminable.

12. A method as defined in claim 11 in which the feature probability value generating step further includes the steps of:

generating a feature detect probability value in response to the statistical distribution function and the threshold value; and

generating a feature detection failure probability value in response to the statistical distribution function and the threshold value.

13. A method as defined in claim 12 in which the feature present probability value generating step includes the step of generating said feature present probability value as a difference between the feature detect probability value and the feature detection failure probability value.

14. A method as defined in claim 12 in which the feature detection probability generating step and said feature detection failure probability generating step further include the steps of generating a reference feature detect reference probability value and a reference feature detection failure probability value in response to a reference statistical distribution function and a threshold value, the feature absence probability value being generated in response to the reference feature detect reference probability value and a reference feature detection failure probability value and the feature detection failure probability value.

15. A system for use in connection with a computer to assess accuracy of selected models of physical phenomena and for determining selection of alternate models in response to a data sequence representing the value of a signal in the presence of noise comprising:

a residual value generator element for controlling the computer to generate a residual sequence reflecting difference values in response to the data sequence and an expected data sequence as would be represented by a selected model;

a feature estimate value generator element for controlling the computer to generate feature estimate values of a plurality of predetermined data features in the residual sequence;

a threshold value determination element for controlling the computer to generate, in response to the feature estimate values, a threshold value for each feature at an estimated ratio of data to noise;

a feature probability value generator element for controlling the computer to generate, in response to the threshold value, probability values representing the likelihood that the feature exists in the data sequence, does not exist in the data sequence, and that the existence or non-existence in the data sequence is not determinable;

a model selector element for controlling the computer to select a model in response to the probability values; and

a controller for controlling the operations of the computer in response to the residual value generator element, the feature estimate value generator element, the threshold value determination element, the feature probability value generator element and the model selector element in a plurality of iterations, during each iteration the computer in response to the residual value generator element using the model selected by the model selector element during the previous iteration.

16. A system as defined in claim 15 in which said threshold value determination element includes:

a test statistic value generator element for controlling the computer to generate, for each feature, a test statistic value in response to a difference between the residual sequence and the residual sequence without regard to the feature, and in further response to a predetermined noise value; and

a threshold value generator element for controlling the computer to generate said threshold value as a function of said test statistic.

17. A system as defined in claim 16 in which said function is selected so that a statistical distribution of the residual values with regard to the feature and the corresponding statistical distribution of the residual values without regard to the feature are the same.

18. A system as defined in claim 15 in which said feature probability value generator element includes:

a feature present probability value generator element for controlling the computer to generate, in response to a statistical distribution function representing a statistical distribution of the residual values with regard to the feature and the corresponding statistical distribution of the residual values without regard to the feature, and the threshold value, a feature present probability value representing a likelihood that the feature is present in the data sequence;

a feature absence probability value generator element for controlling the computer to generate, in response to the statistical distribution function and the threshold value, a feature absent probability value representing a likelihood that the feature is absent from the data sequence; and

a feature indeterminable probability value generator element for controlling the computer to generate, in response to the feature present probability value and the feature absent probability value, a feature indeterminable probability value representing a likelihood that the presence or absence of the feature in the data sequence is indeterminable.

19. A system as defined in claim 18 in which the feature probability value generator element further includes:

a feature detection probability generator element for controlling the computer to generate a feature detect probability value in response to the statistical distribution function and the threshold value; and

a feature detection failure probability generator element for controlling the computer to generate a feature detection failure probability value in response to the statistical distribution function and the threshold value.

20. A system as defined in claim 19 in which the feature present probability value generator element controls the computer to generate said feature present probability value as a difference between the feature detect probability value and the feature detection failure probability value.

21. A system as defined in claim 19 in which the feature detection probability generator element and said feature detection failure probability generator element further control the computer to generate a reference feature detect reference probability value and a reference feature detection failure probability value in response to a reference statistical distribution function and a threshold value, the feature absence probability value generator element controlling the computer to generate the feature absence probability value in response to the reference feature detect reference probability value and a reference feature detection failure probability value and the feature detection failure probability value.