Distributed pattern recognition training method and system

• US 7,562,015 B2
• Filed: 07/14/2005
• Issued: 07/14/2009
• Est. Priority Date: 07/15/2004
• Status: Active Grant

• Current Assignee: Aurilab LLC
• Original Assignee: Aurilab LLC
• Inventors: Baker, James K.
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US11/180,600
• Publication Number:
  US 20060015341A1
• Time in Patent Office: 1,461 Days
• Field of Search: 704/243, 704/244, 704/255, 709/218, 709/219
• US Class Current: 704/243
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A distributed pattern recognition training method includes providing data communication between at least one central pattern analysis node and a plurality of peripheral data analysis sites. The method also includes communicating from the at least one central pattern analysis node to the plurality of peripheral data analysis a plurality of kernel-based pattern elements. The method further includes performing a plurality of iterations of pattern template training at each of the plurality of peripheral data analysis sites.

Claims

1. A distributed pattern recognition training method, comprising:

providing data communication between at least one central pattern analysis node and a plurality of peripheral data analysis sites;

communicating, from said at least one central pattern analysis node to said plurality of peripheral data analysis sites, a plurality of kernel-based pattern elements; and

performing a plurality of iterations of pattern template training at each of said plurality of peripheral data analysis sites,

wherein each iteration of pattern template training comprises;

communicating, from said at least one central pattern analysis node, a candidate solution to a pattern discrimination problem computable from said plurality of kernel-based pattern elements;

at each of said plurality of peripheral data analysis sites, obtaining a plurality of data items; and

at each of said plurality of peripheral data analysis sites, computing statistics derived from said candidate solution, said plurality of kernel-based pattern elements and said plurality of data items.

2. The distributed pattern recognition training method according to claim 1, wherein each iteration of pattern template training further comprises:

communicating said statistics from said plurality of peripheral data analysis sites to said central pattern analysis node; and

performing a pattern template analysis at said central pattern analysis node to update said candidate solution based at least in part on said statistics communicated from said peripheral data analysis sites.

3. The distributed pattern recognition training method according to claim 2, further comprising:

for at least one of said iterations of pattern template training, communicating at least one candidate kernel-based pattern element from at least one of said peripheral data analysis sites to said central pattern analysis node; and

for at least one of said iterations of pattern template training, performing a selection operation among said plurality of kernel-based pattern elements and said at least one candidate kernel-based pattern element to change the number of said kernel-based pattern elements used in said candidate solution to be communicated to said plurality of peripheral data analysis sites in the next iteration of pattern template training.

4. The distributed pattern recognition training method according to claim 2, wherein said statistics communicated from a particular one of said plurality of peripheral data analysis sites include a vector proportional to the direction in which to change one of said kernel-based pattern elements to get the greatest rate of improvement of a specified objective function.

5. The distributed pattern recognition training method according to claim 2, wherein said performing a pattern template analysis at said central analysis node further comprises solving a constrained optimization problem for a selected subset of said kernel-based pattern elements.

6. The distributed pattern recognition training method according to claim 1, wherein said pattern recognition is speech recognition.

7. The distributed pattern recognition training method according to claim 1, wherein said pattern recognition is classification of the output of a speech recognition system.

8. The distributed pattern recognition training method according to claim 1, further comprising:

obtaining a set of data observations at least one of said plurality of peripheral data analysis sites, wherein said set of data observations include acoustic feature observations.

9. The distributed pattern recognition training method according to claim 1, further comprising:

obtaining a set of data observations at least one of said plurality of peripheral data analysis sites, wherein said set of data observations include sequences of words,

wherein each iteration of pattern template training further comprises;

communicating said statistics from said plurality of peripheral data analysis sites to said central pattern analysis node; and

performing a pattern template analysis at said central pattern analysis node to update said candidate solution based at least in part on said statistics communicated from said peripheral data analysis sites.

10. The distributed pattern recognition training method according to claim 1, wherein said plurality of kernel-based pattern elements communicated from said central pattern analysis node to said plurality of peripheral data analysis sites includes a plurality of data items.

11. The distributed pattern recognition training method according to claim 10, further comprising creating a plurality of functionals based on said plurality of data items and at least one reproducing kernel function.

12. The distributed pattern recognition training method according to claim 11, wherein at least one of said plurality of reproducing kernel functions is a function of the covariance matrix of a plurality of data vectors.

13. The distributed pattern recognition training method according to claim 11, wherein said statistics communicated from a particular one of said plurality of peripheral data analysis sites include a vector proportional to the direction in which to change at least one of said reproducing kernel functions to get the greatest rate of improvement of the degree of fit between said plurality of peripheral data items obtained at said particular one of said data analysis sites and a family of functionals based on said at least one reproducing kernel function.

14. The distributed pattern recognition training method according to claim 1, wherein said plurality of kernel-based pattern elements communicated from said central pattern analysis node to said plurality of peripheral data analysis sites includes a plurality of functionals.

15. The distributed pattern recognition training method according to claim 1, further comprising communicating a plurality of reproducing kernel functions from said central pattern analysis node to said plurality of peripheral data analysis sites.

16. The distributed pattern recognition training method according to claim 1,

wherein said pattern template analysis performed at said central pattern analysis node includes solving a constrained optimization problem,

wherein each iteration of pattern template training further comprises:

communicating said statistics from said plurality of peripheral data analysis sites to said central pattern analysis node; and

performing a pattern template analysis at said central pattern analysis node to update said candidate solution based at least in part on said statistics communicated from said peripheral data analysis sites.

17. The distributed pattern recognition training method according to claim 16, wherein said constrained optimization problem is a convex programming problem.

18. The distributed pattern recognition training method according to claim 16, wherein said constrained optimization problem is a linear programming problem.

19. The distributed pattern recognition training method according to claim 16, wherein said constrained optimization problem is a quadratic programming problem.

20. The distributed pattern recognition training method according to claim 1, wherein said plurality of kernel-based pattern elements contains at least one data item,

the method further comprising:

matching said plurality of kernel-based pattern elements to a plurality of peripheral data items.

21. The distributed pattern recognition training method according to claim 20, wherein the matching step comprises computing the distance between said at least one data item from said plurality of kernel-based pattern elements and each of a plurality of peripheral data items based on some distance metric.

22. The distributed pattern recognition training method according to claim 21, wherein said distance metric is the Euclidean distance.

23. The distributed pattern recognition training method according to claim 21, wherein said distance metric is a reproducing kernel in a Hilbert space.

24. The distributed pattern recognition training method according to claim 1, wherein at least one of said plurality of kernel-based pattern elements is a data item which is equal to a selected example data item in a set of training data.

25. The distributed pattern recognition training method according to claim 1, wherein at least one of said plurality of kernel-based pattern elements is the mean vector of a plurality of data items.

26. The distributed pattern recognition training method according to claim 1, further comprising computing constraint satisfaction statistics for kernel-based pattern elements obtained at least one of said peripheral data analysis sites.

27. The distributed pattern recognition training method according to claim 1, further comprising providing said data communication between said central pattern analysis node and said plurality of peripheral data analysis sites at least in part by utilizing a computer network.

28. The distributed pattern recognition training method according to claim 1, further comprising providing said data communication between said central pattern analysis node and said plurality of peripheral data analysis sites at least in part by physically transferring data storage media between said central pattern analysis node and said plurality of peripheral data analysis sites.

29. A computer readable storage device storing machine-readable program code for performing distributed pattern recognition training, the program code, when executed, causing a machine to perform the following steps:

providing data communication between at least one central pattern analysis node and a plurality of peripheral data analysis sites;

communicating, from said at least one central pattern analysis node to said plurality of peripheral data analysis sites, a plurality of kernel-based pattern elements; and

performing a plurality of iterations of pattern template training at each of said plurality of peripheral data analysis site,

wherein each iteration of pattern template training comprises;

communicating, from said at least one central pattern analysis node, a candidate solution to a pattern discrimination problem computable from said plurality of kernel-based pattern elements;

at each of said plurality of peripheral data analysis sites, obtaining a plurality of data items; and

at each of said plurality of peripheral data analysis sites, computing statistics derived from said candidate solution, said plurality of kernel-based pattern elements and said plurality of data items.

30. The computer readable storage device according to claim 29, wherein each iteration of pattern template training further comprises:

communicating said statistics from said plurality of peripheral data analysis sites to said central pattern analysis node; and

performing a pattern template analysis at said central pattern analysis node to update said candidate solution based at least in part on said statistics communicated from said peripheral data analysis sites.

31. The computer readable storage device according to claim 29, further comprising:

for at least one of said iterations of pattern template training, communicating at least one candidate kernel-based pattern element from at least one of said peripheral data analysis sites to said central pattern analysis node; and

for at least one of said iterations of pattern template training, performing a selection operation among said plurality of kernel-based pattern elements and said at least one candidate kernel-based pattern element to change the number of said kernel-based pattern elements used in said candidate solution to be communicated to said plurality of peripheral data analysis sites in the next iteration of pattern template training.

32. The computer readable storage device according to claim 29, wherein said pattern recognition is speech recognition.

33. The computer readable storage device according to claim 29, wherein said pattern recognition is classification of the output of a speech recognition system.

34. The computer readable storage device according to claim 29, further comprising:

obtaining a set of data observations at least one of said plurality of peripheral data analysis sites, wherein said set of data observations include acoustic feature observations.