Detection of characteristics of human-machine interactions for dialog customization and analysis

• US 7,263,489 B2
• Filed: 01/11/2002
• Issued: 08/28/2007
• Est. Priority Date: 12/01/1998
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Riseman, James M., Heck, Larry P., Cohen, Michael H., Balogh, Jennifer E., Mirghafori, Naghmeh N.
• Primary Examiner(s): Opsasnick; Michael N
• Application Number:
  US10/046,026
• Publication Number:
  US 20020095295A1
• Time in Patent Office: 2,055 Days
• Field of Search: 704/270, 704/270.1, 37926501-26525
• US Class Current: 704/270
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4931 Directory assistance systems
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 7/12 for working between exchanges having different types of switching equipment, e.g. power-driven and step by step or decimal and non-decimal

Abstract

A system which uses automatic speech recognition to provide dialogs with human speakers automatically detects one or more characteristics, which may be characteristics of a speaker, his speech, his environment, or the speech channel used to communicate with the speaker. The characteristic may be detected either during the dialog or at a later time based on stored data representing the dialog. If the characteristic is detected during the dialog, the dialog can be customized for the speaker at an application level, based on the detected characteristic. The customization may include customization of operations and features such as call routing, error recovery, call flow, content selection, system prompts, or system persona. Data indicative of detected characteristics can be stored and accumulated for many speakers and/or dialogs and analyzed offline to generate a demographic or other type of analysis of the speakers or dialogs with respect to one or more detected characteristics.

Claims

1. A method comprising:

establishing a speech-based dialog between a person and a machine during a call, wherein the person uses a communication device to speak to the machine via a communication channel during the call;

automatically detecting a characteristic during the dialog in real time, wherein the characteristic is a characteristic of the person, the communication device, the communication channel, or an environment in which the person is located during the dialog, but the characteristic does not uniquely identify the person, the communication device, or any user account; and

selecting a destination to which the call should be routed, based on the detected characteristic, and not based on the meaning of any speech or the failure to recognize any speech during the dialog.

2. A method as recited in claim 1, wherein the characteristic is an approximate age of the person.

3. A method as recited in claim 1, wherein the characteristic is the gender of the person.

4. A method as recited in claim 1, wherein the characteristic is a type of speech being spoken by the person.

5. A method as recited in claim 1, wherein the characteristic is an emotional state of the person.

6. A method as recited in claim 1, wherein the characteristic is indicative of the truthfulness of speech of the person.

7. A method as recited in claim 1, wherein the characteristic is an acoustic characteristic.

8. A method as recited in claim 1, wherein the characteristic is indicative of a speech level of the dialog.

9. A method as recited in claim 1, wherein the characteristic is descriptive of a reason the person is experiencing an error.

10. A method as recited in claim 1, wherein the characteristic is a type of communication device the person is using to communicate with the machine.

11. A system as recited in claim 1, wherein the characteristic is a type of communication device the person is using to communication with the machine.

12. A method as recited in claim 1, wherein the characteristic is indicative of a noise level.

13. A method as recited in claim 12, wherein the characteristic is indicative of an acoustic noise level of the dialog.

14. A method as recited in claim 12, wherein the characteristic is indicative of a signal noise level of the dialog.

15. A method as recited in claim 1, wherein the characteristic is a noise level of an acoustic environment in which the person is located.

16. A method as recited in claim 15, wherein the characteristic is a noise type of an acoustic environment in which the person is located.

17. A method as recited in claim 15, wherein the characteristic is the level of reverberance of an acoustic environment in which the person is located.

18. A system comprising:

a front end to generate a set of features in response to speech from a person during a dialog with the person, wherein the person uses a communication device during a call communicate with the system via a communication channel;

a set of models;

a speech recognition engine to recognize the speech from the person based on the features and the models;

a characteristic detector to detect a characteristic of the person, the communication device, the communication channel, or an environment in which the person is located during the dialog, wherein the characteristic does not uniquely identify the person, the communication device, or any user account; and

a call routing unit to select a destination to which the call from the person should be routed, based on the detected characteristic, and not based on the meaning of any speech or the failure to recognize any speech during the dialog.

19. A method as recited in claim 18, wherein the characteristic is an approximate age of the person.

20. A system as recited in claim 18, wherein the characteristic is the gender of the person.

21. A system as recited in claim 18, wherein the characteristic is a type of speech being spoken by the person.

22. A system as recited in claim 18, wherein the characteristic is an emotional state of the person.

23. A system as recited in claim 18, wherein the characteristic is indicative of the truthfulness of speech of the person.

24. A system as recited in claim 18, wherein the characteristic is an acoustic characteristic.

25. A system as recited in claim 18, wherein the characteristic is indicative of a speech level of the dialog.

26. A system as recited in claim 18, wherein the characteristic is descriptive of a reason the person is experiencing an error.

27. A system as recited in claim 18, wherein the characteristic is indicative of a noise level.

28. A system as recited in claim 27, wherein the characteristic is indicative of an accoustic noise level of the dialog.

29. A system as recited in claim 27, wherein the characteristic is indicative of a signal noise level of the dialog.

30. A system as recited in claim 18, wherein the characteristic is a noise level of an accoustic environment in which the person is located.

31. A system as recited in claim 30, wherein the characteristic is a noise type of an acoustic environment in which the person is located.

32. A system as recited in claim 30, wherein the characteristic is the level of reverberance of an acoustic environment in which the person is located.

33. A method comprising:

establishing a speech-based dialog between a person and a machine, wherein the person uses a communication device to communicate with the machine via a communication channel during the dialog;

automatically detecting a characteristic during the dialog in real time, wherein the characteristic is a characteristic of the person, the communication device, the communication channel, or an environment in which the peson is located during the dialog, and wherein the characteristic does not uniquely identify the person, the communication device, or any user account; and

dynamically customizing a call flow of the dialog for the person during the dialog, based on the detected characteristic, and not based on the meaning of any speech or the failure to recognize any speech during the dialog.

34. A method as recited in claim 33, wherein the characteristic is a characteristic of the person'"'"'s speech during the dialog.

35. A method as recited in claim 33, wherein the characteristic is the person'"'"'s gender or approximate age.

36. A method as recited in claim 33, wherein the characteristic is the person'"'"'s emotional state.

37. A method as recited in claim 33, wherein the characteristic is an acoustic characteristic.

38. A method as recited in claim 33, wherein the characteristic is a type of communication device the person is using to communicate with the machine.