Language model adaptation via network of similar users

• US 6,484,136 B1
• Filed: 10/21/1999
• Issued: 11/19/2002
• Est. Priority Date: 10/21/1999
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Zadrozny, Wlodek W., Kanevsky, Dimitri, Wolf, Catherine G.
• Primary Examiner(s): EDOUARD, PATRICK NESTOR
• Application Number:
  US09/422,383
• Time in Patent Office: 1,125 Days
• Field of Search: 704/1, 704/9, 704/10, 704/255, 704/257, 704/275, 704/277
• US Class Current: 704/9
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

A language recognition system, method and program product for recognizing language based input from computer users on a network of connected computers. Each computer includes at least one user based language model trained for a corresponding user for automatic speech recognition, handwriting recognition, machine translation, gesture recognition or other similar actions that require interpretation of user activities. Network computer users are clustered into classes of similar users according to user similarities such as, nationality, profession, sex, age, etc. User characteristics are collected by sensors and from databases and, then, distributed over the network during user activities. Language models with similarities among similar users on the network are identified. The language models include a language model domain, with similar language models being clustered according to their domains. Language models identified as similar are modified in response to user production activities. After modification of one language model, other identified similar language models are compared and adapted. Also, user data, including information about user activities and language model data, is transmitted over the network to other similar users. Language models are adapted only in response to similar user activities, when these activities are recorded and transmitted over the network. Language models are given a global context based on similar users that are connected together over the network.

Claims

1. A language recognition system for recognizing language based input from computers connected together over a network, each computer including at least one user based language model trained for a particular user, said system comprising:

means for clusterng language models located on ones of a network of computers into classes according to similar users;

means for identifying similar language models among said clustered language models;

means for modifying said language models responsive to user production activities;

means for comparing identified similar language models and adapting all of said identified similar language models responsive to modification of one or more of said similar language models; and

means for transmitting user data over said network, said user data including information about said user production activities and user language model data.

2. A language recognition system as in claim 1, further comprising:

means for identifying at least one language model domain, similar language models being clustered according to identified language model domains.

3. A language recognition system as in claim 2, wherein at least one of said plurality of connected computers includes language models from two or more different domains.

4. A language recognition system as in claim 2, wherein said language model domain identification means comprises:

means for identifying a medical language domain, a radiological language domain, a legal phrase domain, a scientific language domain, a colloquial language domain, and a child-like language domain.

5. A language recognition system as in claim 2, further comprising:

input means for receiving input from one or more connected said computer.

6. A language recognition system as in claim 5, wherein at least one type of said similar language models is located on a first computer and said input is from a second computer, said at least one type of similar language models being responsive to said input on said second computer.

7. A language recognition system as in claim 5, wherein at least one type of said similar language models is a probability set for a plurality of n-tuples of words and said means for modifying said at least one type of similar language model includes means for calculating a new probability set based on said input.

8. A language recognition system as in claim 5, wherein at least one type of said similar language model is a plurality of probability sets, each of said probability sets including probabilities for a plurality of n-tuples of words and, said means for modifying said at least one type of similar language model includes means for calculating a new probability set based on said input.

9. A language recognition system as in claim 5, wherein at least one type of said similar language model is a set of scores corresponding to individual ones of a plurality of phrases and said means for modifying said at least one type of similar language model includes means for calculating a new score set based on said input.

10. A language recognition system as in claim 5, wherein at least one type of said language model is a set of scores corresponding to interpretations of action sequences and said means for modifying said at least one type includes means for calculating a new score set based on said input.

11. A language recognition system as in claim 5, wherein said input means comprises means for manually inputting language.

12. A language recognition system as in claim 11, wherein said manual input means includes a keyboard, a mouse, a touch screen display and a handwriting pad.

13. A language recognition system as in claim 5, wherein said input means comprises means for verbally inputting language.

14. A language recognition system as in claim 1, wherein said means for clustering language models comprises means for identifying common characteristics of users.

15. A language recognition system as in claim 14, wherein identified said common characteristics are selected from the group including each of said computer users'"'"' profession, age, sex, biometrics, activities, social connections, knowledge, family status, family membership, group membership and genotype.

16. A language recognition system as in claim 15, wherein said means for identifying common characteristics comprises means for asking each new user a set of characteristic related questions.

17. A language recognition system as in claim 16, wherein language models are clustered by said means for clustering language models such that language models for users having at least one common characteristic are clustered into a class.

18. A language recognition system as in claim 1, further comprising;

means for receiving an adapted language model transmitted from another computer on said network.

19. A language recognition system as in claim 1, wherein the network of computers comprises:

at least one server;

at least one personal computer;

at least one hand held computer; and

at least one embedded computer.

20. A language recognition method for recognizing language from each of a plurality of computer users, said computer users using computers coupled together over a network, said method comprising the steps of:

clustering users into classes of similar users;

identifying similar language models among language models of similar users;

modifying a language model of one clustered user responsive to user production activities;

comparing identified similar language models and adapting all said identified similar language models responsive to modification of said language model; and

transmitting user data over said network, said user data including information about user activities and user language model data.

21. A language recognition method as in claim 20, wherein the step of comparing similar language models comprises the steps of:

comparing active user vocabularies;

comparing frequencies of use of words from vocabularies;

comparing frequencies of use of n-tuple of words;

comparing scores of similar sequences of user actions; and

comparing results of mathematical transformations.

22. A language recognition method as in claim 21, wherein the mathematical transformations are applied to recognition scores.

23. A language recognition method as in claim 22, wherein the recognition scores comprise word probabilities, word sequence probabilities and action sequence probabilities.

24. A language recognition method as in claim 20, wherein the step of modifying similar language models comprises the steps of:

adding new words to language model vocabularies;

removing unused words from language model vocabularies;

changing word frequencies for words responsive to use by at least one user;

changing n-tuples word frequencies responsive to use by at least one user;

changing action sequence scores responsive to use by at least one user; and

transforming probabilities and scores for word and action sequences responsive to use by at least one user.

25. A language recognition method as in claim 24, wherein the step of transforming probabilities comprises:

adapting language model scores responsive to user activities;

linearly combining language model score sets with a set of language model scores generated responsive to user activities;

linearly combining language model probability sets with a set of language model probabilities generated responsive to user activities; and

interpolating language model statistical data responsive to user activities.

26. A language recognition method as in claim 22, wherein the step of identifying similar language models comprises:

identifying a language model domain, similar language models being clustered according to said identified domain.

27. A language recognition method as in claim 22, wherein the step of clustering similar users comprises:

identifying users having a plurality of common characteristics.

28. A language recognition method as in claim 22, wherein during the step of transmitting data, data is transmitted to other users within the same cluster.

29. A computer program product for recognizing language from each of a plurality of computer users, said computer users using computers coupled together over a network, said computer program product comprising a computer usable medium having computer readable program code thereon, said computer readable program code comprising:

computer readable program code means for clustering users into classes of similar users;

computer readable program code means for identifying similar language models among language models of similar users;

computer readable program code means for modifying a language model of one clustered user responsive to user production activities;

computer readable program code means for comparing identified similar language models and adapting all said identified similar language models responsive to modification of said language model; and

computer readable program code means for transmitting user data over said network, said user data including information about user activities and user language model data.

30. A computer program product as in claim 29, wherein the computer readable program code means for comparing similar language models comprises:

computer readable program code means for comparing active user vocabularies;

computer readable program code means for comparing frequencies of use of words from vocabularies;

computer readable program code means for comparing frequencies of use of n-tuple of words;

computer readable program code means for comparing scores of similar sequences of user actions; and

computer readable program code means for comparing results mathematical transformations.

31. A computer program product as in claim 29, wherein the computer readable program code means for modifying similar language models comprises:

computer readable program code means for adding new words to language model vocabularies;

computer readable program code means for removing unused words from language model vocabularies;

computer readable program code means for changing word frequencies for words responsive to use by at least one user;

computer readable program code means for changing n-tuple word frequencies responsive to use by at least one user;

computer readable program code means for changing action sequence scores responsive to use by at least one user; and

computer readable program code means for transforming probabilities and scores for word and action sequences responsive to use by at least one user.

32. A computer program product as in claim 31, wherein computer readable program code means for transforming probabilities comprises:

computer readable program code means for adapting language model scores responsive to user activities;

computer readable program code means for linearly combining language model score sets with a set of language model scores generated responsive to user activities;

computer readable program code means for linearly combining language model probability sets with a set of language model probabilities generated responsive to user activities; and

computer readable program code means for interpolating language model statistical data responsive to user activities.

33. A computer program product as in claim 29, wherein the computer readable program code means for identifying similar language models comprises:

computer readable program code means for identifying a language model domain, similar language models being clustered according to said identified domain.

34. A computer program product as in claim 29, wherein the computer readable program code means for clustering similar users comprises:

computer readable program code means for identifying users having a plurality of common characteristics.

35. A computer program product as in claim 29, wherein the computer readable program code means for transmitting data, transmits data to other users within the same cluster.