Method and system for non-intrusive speaker verification using behavior models

• US 20030046072A1
• Filed: 09/12/2002
• Published: 03/06/2003
• Est. Priority Date: 03/01/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ramaswamy, Ganesh N., Chaudhari, Upendra V.
• Granted Patent: US 7,689,418 B2
• Time in Patent Office: Days
• US Class Current: 704/240
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces

Abstract

A system and method for verifying user identity, in accordance with the present invention, includes a conversational system for receiving inputs from a user and transforming the inputs into formal commands. A behavior verifier is coupled to the conversational system for extracting features from the inputs. The features include behavior patterns of the user. The behavior verifier is adapted to compare the input behavior to a behavior model to determine if the user is authorized to interact with the system.

Claims

1. A system for verifying user identity comprising:

a conversational system for receiving inputs from a user and transforming the inputs into formal commands; and

a behavior verifier coupled to the conversational system for extracting features from the inputs, the features including behavior patterns of the user, the behavior verifier being adapted to compare the input behavior to a behavior model to determine if the user is authorized to interact with the system.

2. The system as recited in claim 1, wherein the conversational system includes a natural language understanding unit for interpreting speech received as the inputs.

3. The system as recited in claim 1, wherein the inputs include at least one of speech, handwriting, text, and gestures.

4. The system as recited in claim 1, wherein the behavior verifier includes a feature extractor for extracting feature vectors from the inputs.

5. The system as recited in claim 4, wherein the feature vectors include at least one of language model scores, acoustic model scores, natural language and understanding scores.

6. The system as recited in claim 4, wherein the feature vectors include at least one of command predication scores and pronunciation scores.

7. The system as recited in claim 4, wherein the feature vectors include information about system responses to the user.

8. The system as recited in claim 4, wherein the feature vectors include at least one of duration between user commands and a dialog state between the user and the system.

9. The system as recited in claim 4, wherein the feature vectors include types of input modalities employed by the user.

10. The system as recited in claim 1, wherein the behavior model includes a plurality of models.

11. The system as recited in claim 1, wherein the behavior verifier includes a probability calculator, the probability calculator being adapted to calculate a first probability that the user is authorized to interact with the system based on the user'"'"'s behavior.

12. The system as recited in claim 11, wherein the behavior verifier includes a model constructor for constructing the behavior model for a user, the behavior model being employed by the probability calculator to compare behavior to current behavior of the user.

13. The system as recited in claim 11, further comprising:

an acoustic and biometric verifier for determining acoustic and biometric information from the user and determining a second probability that the user is authorized to interact with the system based on the user'"'"'s acoustic or biometric information; and

the behavior verifier including a probability mixer adapted for integrating the first probability with the second probability to verify that the user is authorized to interact with the system.

14. The system as recited in claim 11, wherein the first probability is compared to a threshold probability to determine if the user is authorized to use the system.

15. A method for verifying a user based on behavior comprising the steps of:

providing inputs to a conversational system for receiving the inputs from a user and transforming the inputs into formal commands;

extracting features from the inputs, the features including behavior patterns of the user; and

comparing the input behavior to a behavior model to determine if the user is authorized to interact with the system.

16. The method as recited in claim 15, wherein the conversational system includes a natural language understanding unit, the method further comprising the step of interpreting speech received as the inputs.

17. The method as recited in claim 15, wherein the inputs include at least one of speech, handwriting, text, and gestures.

18. The method as recited in claim 15, wherein the behavior verifier includes a feature extractor, and the method further comprising the step of extracting feature vectors from the inputs.

19. The method as recited in claim 18, wherein the feature vectors include at least one of language model scores, acoustic model scores and natural language understanding scores.

20. The method as recited in claim 18, wherein the feature vectors include at least one of command predication scores and pronunciation scores.

21. The method as recited in claim 18, wherein the feature vectors include information about system responses to the user.

22. The method as recited in claim 18, wherein the feature vectors include at least one of duration between user commands and a dialog state between the user and the system.

23. The method as recited in claim 18, wherein the feature vectors include types of input modalities employed by the user.

24. The method as recited in claim 15, wherein the behavior verifier includes a probability calculator, the method further comprising the step of calculating a first probability on the probability calculator, the first probability for indicating whether the user is authorized to interact with the system based on the user'"'"'s behavior.

25. The method as recited in claim 24, wherein the behavior verifier includes a model constructor, the method further comprising the step of constructing the behavior model for a user, the behavior model being employed by the probability calculator to compare behavior to current behavior of the user.

26. The method as recited in claim 24, further comprising:

an acoustic and biometric verifier for determining acoustic and biometric information from the user and the method further comprising the steps of;

determining a second probability for indicating whether the user is authorized to interact with the system based on the user'"'"'s acoustic or biometric information; and

integrating the first probability with the second probability, by employing a probability mixer, to verify that the user is authorized to interact with the system.

27. The method as recited in claim 24, wherein the first probability is compared to a threshold probability to determine if the user is authorized to use the system.

28. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for verifying a user based on behavior, the method steps comprising:

providing inputs to a conversational system for receiving the inputs from a user and transforming the inputs into formal commands;

extracting features from the inputs, the features including behavior patterns of the user; and

comparing the input behavior to a behavior model to determine if the user is authorized to interact with the system.

29. The program storage device as recited in claim 28, wherein the conversational system includes a natural language understanding unit, the method further comprising the step of interpreting speech received as the inputs.

30. The program storage device as recited in claim 28, wherein the inputs include at least one of speech, handwriting, text, and gestures.

31. The program storage device as recited in claim 28, wherein the behavior verifier includes a feature extractor, and the method further comprising the step of extracting feature vectors from the inputs.

32. The program storage device as recited in claim 31, wherein the feature vectors include at least one of language model scores, acoustic model scores and natural language understanding scores.

33. The program storage device as recited in claim 31, wherein the feature vectors include at least one of command predication scores and pronunciation scores.

34. The program storage device as recited in claim 31, wherein the feature vectors include information about system responses to the user.

35. The program storage device as recited in claim 31, wherein the feature vectors include at least one of duration between user commands and a dialog state between the user and the system.

36. The program storage device as recited in claim 31, wherein the feature vectors include types of input modalities employed by the user.

37. The program storage device as recited in claim 28, wherein the behavior verifier includes a probability calculator, the method further comprising the step of calculating a first probability on the probability calculator, the first probability for indicating whether the user is authorized to interact with the system based on the user'"'"'s behavior.

38. The program storage device as recited in claim 37, wherein the behavior verifier includes a model constructor, the method further comprising the step of constructing the behavior model for a user, the behavior model being employed by the probability calculator to compare behavior to current behavior of the user.

39. The program storage device as recited in claim 37, further comprising:

an acoustic and biometric verifier for determining acoustic and biometric information from the user and the method further comprising the steps of;

determining a second probability for indicating whether the user is authorized to interact with the system based on the user'"'"'s acoustic or biometric information; and

integrating the first probability with the second probability, by employing a probability mixer, to verify that the user is authorized to interact with the system.

40. The program storage device as recited in claim 37, wherein the first probability is compared to a threshold probability to determine if the user is authorized to use the system.