Robust pattern recognition system and method using socratic agents

• US 8,331,657 B2
• Filed: 04/13/2012
• Issued: 12/11/2012
• Est. Priority Date: 09/13/2006
• Status: Active Grant

• Current Assignee: Aurilab LLC
• Original Assignee: Aurilab LLC
• Inventors: Baker, James K.
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/446,955
• Publication Number:
  US 20120203716A1
• Time in Patent Office: 242 Days
• Field of Search: 382155-161, 700/47, 700/48, 704231-257, 706 1- 62
• US Class Current: 382/159
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques

Abstract

A computer-implemented pattern recognition method, system and program product, the method comprising in one embodiment: creating electronically a linkage between a plurality of models within a classifier module within a pattern recognition system such that any one of said plurality of models may be selected as an active model in a recognition process; creating electronically a null hypothesis between at least one model of said plurality of linked models and at least a second model among said plurality of linked models; accumulating electronically evidence to accept or reject said null hypothesis until sufficient evidence is accumulated to reject said null hypothesis in favor of one of said plurality of linked models or until a stopping criterion is met; and transmitting at least a portion of the electronically accumulated evidence or a summary thereof to accept or reject said null hypothesis to a pattern classifier module.

Claims

1. A pattern recognition method, comprising:

creating, using a first one or more computers, a communication linkage among a plurality of pattern recognition modules;

selecting, using the first one or more of the computers, as one or more source pattern recognition modules one or more of the pattern recognition modules;

obtaining or having obtained, using the first one or more computers, in said one or more source pattern recognition modules one or more pieces of communicable knowledge data;

selecting or having selected or receiving a selection, using the first one or more computers, of one or more of the pattern recognition modules as one or more recipient pattern recognition modules to receive said one or more pieces of communicable knowledge data;

sending or having sent, using the first one or more computers and the communication linkage, said one or more pieces of communicable knowledge data from said one or more source pattern recognition modules to said one or more recipient pattern recognition modules;

obtaining or having obtained or receiving or having received, using the first one or more computers, performance measurements comparing for each of one or more of the recipient pattern recognition modules respectively, its performance with the addition of the one or more pieces of communicable knowledge data with its performance without the one or more pieces of communicable knowledge data; and

communicating or having communicated, using the first one or more computers and the communication linkage said performance measurements from said one or more recipient pattern recognition modules back to said one or more source pattern recognition modules.

2. The pattern recognition method as defined in claim 1,

wherein said one or more pattern recognition modules are distributed across a plurality of computers that may or may not include one or more of the computers in the first one or more computers, and

wherein said communication linkage comprises electronic communications using one or more electronic communications networks.

3. The pattern recognition method as defined in claim 1,

wherein the plurality of pattern recognition modules are implemented on a single computer and said communication linkage is implemented using software implemented on that single computer.

4. The pattern recognition method as in claim 3, wherein the first one or more computers consists of the single computer.

5. The pattern recognition method as defined in claim 1,

wherein said one or more pieces of communicable knowledge data comprises at least one selected from the group of the following forms of knowledge representation:

an additional pattern model to be added to a set comprising a plurality of pattern models;

an addition or deletion to a pattern structure, where the pattern structure before the addition or deletion is present in both at least one of the source pattern recognition modules and at least one of the recipient pattern recognition modules;

a change of one or more labels in a set of labeled data;

a module comprising one or more pieces of represented knowledge originating in at least one of said one or more source recognition modules and associated software allowing said one of more pieces of represented knowledge to be utilized in at least one of said one or more recipient pattern recognition modules; and

a decision choice at a system decision point where the decision point is present in both at least one source pattern recognition module and at least one recipient pattern recognition module.

6. The pattern recognition method as defined in claim 1,

wherein said one or more pieces of communicable knowledge data comprises an additional pattern model to be added to a set comprising a plurality of pattern models.

7. The pattern recognition method as defined in claim 1,

wherein said one or more pieces of communicable knowledge data comprises a change in one or more labels in a set of labeled data.

8. The pattern recognition method as defined in claim 1, further comprising:

choosing a specific criterion, for said selecting or having selected or receiving a selection of one or more recipient pattern recognition modules step, based at least in part on whether a particular recipient pattern recognition module is capable of being a recipient of the one or more pieces of communicable knowledge data from the one or more source pattern recognition modules,

wherein said selecting or having selected or receiving a selection of the one or more recipient pattern recognition modules step comprises selecting or having selected or receiving a selection of a small subset of the recipient pattern recognition modules from a larger plurality of the pattern recognition modules.

9. The pattern recognition method as defined in claim 1, further comprising:

creating or having created at least one additional pattern recognition module by creating or having created a plurality of variants of at least one of the source pattern recognition modules; and

adding or having added this at least one additional pattern recognition module to the plurality of the pattern recognition modules.

10. The pattern recognition method as defined in claim 9, further comprising:

obtaining a plurality of streams of data using the first one or more computers;

creating, or having created using the first one or more computers, said plurality of variants at least in part by training at least one variant on a first stream of data from the plurality of streams of data and

training or having trained at least one other variant on a second stream of data from the plurality of streams of data that differs from the first stream of data.

11. The pattern recognition method as defined in claim 1, further comprising:

increasing or having increased the use in one or more of the source pattern recognition modules of the communicable knowledge data for which performance measurements in one or more of the recipient pattern recognition modules indicates improved performance from use of said communicable knowledge data compared to performance without said communicable knowledge data.

12. The pattern recognition method as defined in claim 1, further comprising:

decreasing or having decreased the use in one or more of the source pattern recognition modules of the communicable knowledge data for which performance measurements in one or more of the recipient pattern recognition modules indicates degraded performance from using said communicable knowledge data compared to performance without said communicable knowledge data.

13. The pattern recognition method as defined in claim 1, further comprising:

selecting, using the first one or more computers, one or more of the pieces of the communicable knowledge data for which performance measurements one or more of the recipient pattern recognition modules indicates improved performance using said selected communicable knowledge data compared to performance without said selected communicable knowledge data;

selecting or having selected or receiving a selection, using the first one or more computers, of one or more additional recipient pattern recognition modules, to receive said selected one or more pieces of communicable knowledge data for which the performance measurement on the recipient pattern recognition modules previously selected indicates improved performance using said selected communicable knowledge data compared to performance without said selected communicable knowledge data; and

sending or having sent, using the first one or more computers and the communication linkage, said selected one or more pieces of communicable knowledge data from said one or more source pattern recognition modules to said one or more recipient pattern recognition modules including the one or more additional recipient pattern recognition modules.

14. A pattern recognition system, comprising:

a first one or more computers configured to perform the following steps;

creating, using the first one or more computers, a communication linkage among a plurality of pattern recognition modules;

selecting or having selected or receiving a selection, using the first one or more of the computers, one or more of the pattern recognition modules as one or more source pattern recognition modules;

obtaining or having obtained, using the first one or more computers, in said one or more source pattern recognition modules one or more pieces of communicable knowledge data;

selecting, using the first one or more computers, as one or more recipient pattern recognition modules one or more of the pattern recognition modules to receive said one or more pieces of communicable knowledge data;

sending or having sent, using the first one or more computers and the communication linkage, said one or more pieces of communicable knowledge data from said one or more source pattern recognition modules to said one or more recipient pattern recognition modules;

obtaining or having obtained or receiving or having received, using the first one or more computers, performance measurements comparing for each of one or more of the recipient pattern recognition modules respectively, its performance with the addition of the one or more pieces of communicable knowledge data with its performance without the one or more pieces of communicable knowledge data; and

communicating or having communicated, using the first one or more computers and the communication linkage to said performance measurements from said one or more recipient pattern recognition modules back to said one or more source pattern recognition modules.

15. The pattern recognition system as defined in claim 14,

wherein said one or more pattern recognition modules are distributed across a plurality of computers that may or may not include one or more of the computers in the first one or more computers, and

wherein said communication linkage comprises electronic communications using one or more electronic communications networks.

16. The pattern recognition system as defined in claim 14,

wherein the plurality of pattern recognition modules are implemented on a single computer and said communication linkage is implemented using software implemented on that single computer.

17. The pattern recognition system as in claim 16, wherein the first one or more computers consists of the single computer.

18. The pattern recognition system as defined in claim 14,

wherein said one or more pieces of communicable knowledge data comprises at least one selected from the group of the following forms of knowledge representation:

an additional pattern model to be added to a set comprising a plurality of pattern models;

an addition or deletion to a pattern structure, where the pattern structure before the addition or deletion is present in both at least one of the source pattern recognition modules and at least one of the recipient pattern recognition modules;

a change of one or more labels in a set of labeled data;

a module comprising one or more pieces of represented knowledge originating in at least one of said one or more source recognition modules and associated software allowing said one of more pieces of represented knowledge to be utilized in at least one of said one or more recipient pattern recognition modules; and

a decision choice at a system decision point where the decision point is present in both at least one source pattern recognition module and at least one recipient pattern recognition module.

19. The pattern recognition system as defined in claim 14,

wherein said one or more pieces of communicable knowledge data comprises an additional pattern model to be added to a set comprising a plurality of pattern models.

20. The pattern recognition system as defined in claim 14,

wherein said one or more pieces of communicable knowledge data comprises a change in one or more labels in a set of labeled data.

21. The pattern recognition system as defined in claim 14, wherein the first one or more computers are further configured to perform the step:

choosing a specific criterion, for said selecting or having selected or receiving a selection of one or more recipient pattern recognition modules operation, based at least in part on whether a particular recipient pattern recognition module is capable of being a recipient of the one or more pieces of communicable knowledge data from the one or more source pattern recognition modules,

wherein said selecting or having selected or receiving a selection of the one or more recipient pattern recognition modules operation comprises selecting or having selected or receiving a selection of a small subset of the recipient pattern recognition modules from a larger plurality of the pattern recognition modules.

22. The pattern recognition system as defined in claim 14, wherein the first one or more computers are further configured to perform the steps:

creating or having created at least one additional pattern recognition module by creating or having created a plurality of variants of at least one of the source pattern recognition modules; and

adding or having added this at least one additional pattern recognition module to the plurality of the pattern recognition modules.

23. The pattern recognition system as defined in claim 22, wherein the first one or more computers are further configured to perform the steps:

obtaining a plurality of streams of data using the first one or more computers;

creating, or having created using the first one or more computers, said plurality of variants at least in part by training at least one variant on a first stream of data from the plurality of streams of data and

training or having trained at least one other variant on a second stream of data from the plurality of streams of data that differs from the first stream of data.

24. The pattern recognition system as defined in claim 14, wherein the first one or more computers are further configured to perform the step:

increasing or having increased the use in one or more of the source pattern recognition modules of the communicable knowledge data for which performance measurements in one or more of the recipient pattern recognition modules indicates improved performance from use of said communicable knowledge data compared to performance without said communicable knowledge data.

25. The pattern recognition system as defined in claim 14, wherein the first one or more computers are further configured to perform the step:

decreasing or having decreased the use in one or more of the source pattern recognition modules of the communicable knowledge data for which performance measurements in one or more of the recipient pattern recognition modules indicates degraded performance from using said communicable knowledge data compared to performance without said communicable knowledge data.

26. The pattern recognition system as defined in claim 14, wherein the first one or more computers are further configured to perform the steps:

selecting, using the first one or more computers, one or more of the pieces of the communicable knowledge data for which performance measurements one or more of the recipient pattern recognition modules indicates improved performance using said selected communicable knowledge data compared to performance without said selected communicable knowledge data;

selecting or having selected or receiving a selection, using the first one or more computers, of one or more additional recipient pattern recognition modules, to receive said selected one or more pieces of communicable knowledge data for which the performance measurement on the recipient pattern recognition modules previously selected indicates improved performance using said selected communicable knowledge data compared to performance without said selected communicable knowledge data; and

sending or having sent, using the first one or more computers and the communication linkage, said selected one or more pieces of communicable knowledge data from said one or more source pattern recognition modules to said one or more recipient pattern recognition modules including the one or more additional recipient pattern recognition modules.