Method and system for non-intrusive speaker verification using behavior models

• US 7,689,418 B2
• Filed: 09/12/2002
• Issued: 03/30/2010
• Est. Priority Date: 03/01/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ramaswamy, Ganesh N., Chaudhari, Upendra V.
• Primary Examiner(s): Lerner; Martin
• Application Number:
  US10/242,540
• Publication Number:
  US 20030046072A1
• Time in Patent Office: 2,756 Days
• Field of Search: 704/250, 704/246, 704/275, 704/273, 381/116, 726/22, 726/23, 726/24, 726/25, 726/26, 382/116
• US Class Current: 704/250
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces

Abstract

A system and method for verifying user identity, in accordance with the present invention, includes a conversational system for receiving inputs from a user and transforming the inputs into formal commands. A behavior verifier is coupled to the conversational system for extracting features from the inputs. The features include behavior patterns of the user. The behavior verifier is adapted to compare the input behavior to a behavior model to determine if the user is authorized to interact with the system.

Claims

1. A system for verifying user identity comprising:

a conversational system that receives multi-modal inputs from a user interacting with the conversational system during a user session and transforms the received multi-modal inputs into formal commands executable by a program of instructions executable by a processor; and

a behavior verifier coupled to the conversational system to extract features from the multi-modal inputs and formal commands, wherein the extracted features include a combination of input modalities representative of the user'"'"'s current interaction behavior for performing a task during the user session, and wherein the behavior verifier compares the combination of input modalities representative of the user'"'"'s current interaction behavior for performing the task to a behavior model representative of the user'"'"'s past interaction behavior comprising a known combination of input modalities for performing the task used by the user during one or more previous user sessions to determine the identity of the user.

2. The system as recited in claim 1, wherein the conversational system includes a natural language understanding unit to interpret speech input.

3. The system as recited in claim 1, wherein the multi-modal inputs include speech, handwriting, text, or gestures or a combination thereof.

4. The system as recited in claim 1, wherein the behavior verifier includes a feature extractor to extract a set of n features from the multi-modal inputs and formal commands and construct n-dimensional feature vectors.

5. The system as recited in claim 1, wherein the extracted features include features representative of language modal scores, acoustic model scores, or natural language understanding scores, or a combination thereof.

6. The system as recited in claim 1, wherein the extracted features include features representative of command prediction scores.

7. The system as recited in claim 1, wherein the extracted features include features representative of a duration between executed user commands.

8. The system as recited in claim 1, wherein the extracted features include features representative of the types of input modalities employed by the user.

9. The system as recited in claim 1, wherein the behavior model includes a plurality of models representative of different interaction behaviors comprising combinations of input modalities of the user.

10. The system as recited in claim 1, wherein the behavior verifier includes a probability calculator to calculate a first probability representing a likelihood that the user is authorized to interact with the system based on the user'"'"'s current interaction behavior comprising the combination of input modalities.

11. The system as recited in claim 10, wherein the behavior verifier includes a model constructor to construct and periodically update a behavior model for one or more authorized users.

12. The system as recited in claim 10, wherein the first probability is compared to a threshold probability to determine if the user is authorized to use the system.

13. The system as recited in claim 1, wherein the extracted features include features representative of a dialog state between the user and the system.

14. A method for verifying user identity, comprising:

receiving multi-modal inputs from a user interacting with a conversational system during a user session and transforming the received multi-modal inputs into formal commands executable by a program of instructions executable by a processor;

extracting features from the multi-modal inputs and formal commands, wherein the extracted features include a combination of input modalities representative of the user'"'"'s current interaction behavior for performing a task during the user session; and

comparing the combination of input modalities representative of the user'"'"'s current interaction behavior for performing the task to a behavior model representative of the user'"'"'s past interaction behavior comprising a known combination of input modalities for performing the task used by the user during one or more previous user sessions to determine the identity of the user.

15. The method as recited in claim 14, further comprising interpreting received speech input using natural language understanding.

16. The method as recited in claim 14, wherein the multi-modal inputs include speech, handwriting, text, or gestures or a combination thereof.

17. The method as recited in claim 14, wherein extracting features comprises extracting a set of n features from the multi-modal inputs and formal commands and constructing n-dimensional feature vectors.

18. The method as recited in claim 14, wherein the extracted features include features representative of language model scores, acoustic model scores or natural language understanding scores, or a combination thereof.

19. The method as recited in claim 14, wherein the extracted features include features representative of command predication scores.

20. The method as recited in claim 14, wherein the extracted features include features representative of a duration between executed user commands.

21. The method as recited in claim 14, wherein the extracted features include features representative of the types of input modalities employed by the user.

22. The method as recited in claim 14, wherein processing the extracted features using a behavior modal includes calculating a first probability representing a likelihood that the user is authorized to interact with the system based on the user'"'"'s behavior.

23. The method as recited in claim 22, wherein the first probability is compared to a threshold probability to determine if the user is authorized to use the system.

24. The method as recited in claim 14, further comprising constructing and periodically updating a behavior model for one or more authorized users.

25. The system as recited in claim 14, wherein the extracted features include features representative of a dialog state between the user and the system.

26. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for verifying a user identity, the method steps comprising:

receiving multi-modal inputs from a user interacting with a conversational system during a user session and transforming the received multi-modal inputs into formal commands executable by the machine;

extracting features from the multi-modal inputs and formal commands, wherein the extracted features include a combination of input modalities representative of the user'"'"'s current interaction behavior for performing a task during the user session; and

comparing the combination of input modalities representative of the user'"'"'s current interaction behavior for performing the task to a behavior model representative of the user'"'"'s past interaction behavior comprising a known combination of input modalities for performing the task used by the user during one or more previous user sessions to determine the identity of the user.

27. The program storage device as recited in claim 26, further comprising instructions for interpreting speech input using natural language understanding.

28. The program storage device as recited in claim 26, wherein the multi-modal inputs include speech, handwriting, text, or gestures or a combination thereof.

29. The program storage device as recited in claim 26, wherein the instructions for extracting features comprise instructions for extracting a set of n features from the multi-modal inputs and formal commands and constructing n-dimensional feature vectors.

30. The program storage device as recited in claim 26, wherein the extracted features include features representative of language model scores, acoustic model scores or natural language understanding scores, or a combination thereof.

31. The program storage device as recited in claim 26, wherein the extracted features include features representative of command predication scores or pronunciation scores or both command predication scores and pronunciation scores.

32. The program storage device as recited in claim 26, wherein the extracted features include features representative of a duration between executed user commands.

33. The program storage device as recited in claim 26, wherein the extracted features include features representative of types of input modalities employed by the user.

34. The program storage device as recited in claim 26, wherein the instructions for processing the extracted features using a behavior model include instructions for calculating a first probability representing a likelihood that the user is authorized to interact with the system based on the user'"'"'s current interaction behavior.

35. The program storage device as recited in claim 34, wherein the first probability is compared to a threshold probability to determine if the user is authorized to use the system.

36. The program storage device as recited in claim 26, further comprising instructions for constructing and periodically updating a behavior model for one or more authorized users.

37. The program storage device as recited in claim 26, wherein the extracted features include features representative of a dialog state between the user and the system.