Establishing a multimodal personality for a multimodal application in dependence upon attributes of user interaction

• US 8,374,874 B2
• Filed: 09/11/2006
• Issued: 02/12/2013
• Est. Priority Date: 09/11/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Cross, Charles W. Jr., Pike, Hilary A.
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US11/530,628
• Publication Number:
  US 20080065387A1
• Time in Patent Office: 2,346 Days
• Field of Search: 704/270, 704/275
• US Class Current: 704/270
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

Establishing a multimodal personality for a multimodal application, including evaluating, by the multimodal application, attributes of a user'"'"'s interaction with the multimodal application; selecting, by the multimodal application, a vocal demeanor in dependence upon the values of the attributes of the user'"'"'s interaction with the multimodal application; and incorporating, by the multimodal application, the vocal demeanor into the multimodal application.

Claims

1. A method of establishing a vocal demeanor for an application having a speech interface that presents speech content according to the vocal demeanor, the method comprising:

determining, from input of a user, at least one first attribute of the user'"'"'s interaction with the application;

specifying, based on an evaluation of the at least one first attribute of the user'"'"'s interaction with the application, a particular voice family to be used by the application in presenting speech to the user via a speech interface, wherein the particular voice family identifies a particular person'"'"'s voice by describing an overall nature and timbre of the particular person'"'"'s voice in specific terms;

selecting the vocal demeanor for the application using the specified particular voice family that identifies the particular person'"'"'s voice as a criterion for the selecting, wherein the vocal demeanor that is selected is configured to present speech using the particular voice family that identifies the particular person'"'"'s voice that was specified based on the evaluation of the at least one first attribute of the user'"'"'s interaction; and

incorporating, by the application, the vocal demeanor into the application by presenting at least some speech content via the application in the voice family.

2. The method of claim 1, wherein the at least one first attribute of the user'"'"'s interaction comprises a manner in which the user interacts with the application, wherein the user may interact with the application using a speech interface and/or at least one non-speech interface.

3. The method of claim 1, wherein the at least one first attribute of the user'"'"'s interaction comprises at least one attribute of user speech.

4. The method of claim 3, wherein the at least one first attribute of user speech comprises speech rate, voice family, pitch, pitch range, and richness.

5. The method of claim 1, wherein incorporating the vocal demeanor into the application further comprises linking at least one markup element of a markup document of the application to a style of a cascading style sheet.

6. The method of claim 1, further comprising:

specifying, based on the evaluation of the at least one first attribute of the user'"'"'s interaction with the application, a gender to be used by the application in presenting speech to the user via the speech interface,

wherein the selecting comprises selecting the vocal demeanor using the specified gender as a second criterion for the selecting.

7. The method of claim 1, further comprising:

specifying, based on the evaluation of the at least one first attribute of the user'"'"'s interaction with the application, a reproduction direction to be used by the application in presenting speech to the user via the speech interface,

wherein the selecting comprises selecting the vocal demeanor using the specified reproduction direction as a second criterion for the selecting.

8. The method of claim 1, wherein selecting the vocal demeanor comprises selecting a predefined vocal demeanor having predefined speech attributes similar to the at least one first attribute of the user'"'"'s interaction with the application, the predefined speech attributes comprising the specified particular voice family.

9. An apparatus for establishing a vocal demeanor for an application having a speech interface that presents speech content according to the vocal demeanor, the apparatus comprising:

a computer processor; and

a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions to cause the computer processor to carry out acts comprising;

determining, from input of a user, at least one first attribute of the user'"'"'s interaction with the application;

specifying, based on an evaluation of the at least one first attribute of the user'"'"'s interaction with the application, a particular voice family to be used by the application in presenting speech to the user via a speech interface, wherein the particular voice family identifies a particular person'"'"'s voice by describing an overall nature and timbre of the particular person'"'"'s voice in specific terms;

selecting the vocal demeanor for the application using the specified particular voice family as a criterion for the selecting, wherein the vocal demeanor that is selected is configured to present speech using the particular voice family that was specified based on the evaluation of the at least one first attribute of the user'"'"'s interaction; and

incorporating the vocal demeanor into the application by presenting at least some speech content via the application in the voice family.

10. The apparatus of claim 9, wherein the at least one first attribute of the user'"'"'s interaction comprises a manner in which the user interacts with the application, wherein the user may interact with the application using a speech interface and/or at least one non-speech interface.

11. The apparatus of claim 9, wherein the at least one first attribute of the user'"'"'s interaction comprises at least one attribute of user speech.

12. The apparatus of claim 11, wherein the at least one first attribute of user speech further comprises speech rate, voice family, pitch, pitch range, and richness.

13. The apparatus of claim 9, wherein incorporating the vocal demeanor into the application further comprises linking at least one markup element of a markup document of the application to a style of a cascading style sheet.

14. The apparatus of claim 9, wherein the computer program instructions cause the computer processor to select the vocal demeanor by, at least in part, selecting a predefined vocal demeanor having predefined speech attributes similar to the at least one first attribute of the user'"'"'s interaction with the application, the predefined speech attributes comprising specified particular voice family.

15. A computer program product for establishing a vocal demeanor for an application having a speech interface that presents speech content according to the vocal demeanor, the computer program product being disposed upon a recordable storage medium, the computer program product comprising computer program instructions that, when executed, perform a method comprising:

determining, from input of a user, at least one first attribute of the user'"'"'s interaction with the application;

specifying, based on an evaluation of the at least one first attribute of the user'"'"'s interaction with the application, a particular voice family to be used by the application in presenting speech to the user via a speech interface, wherein the particular voice family identifies a particular person'"'"'s voice by describing an overall nature and timbre of the particular person'"'"'s voice in specific terms;

selecting the vocal demeanor for the application using the specified particular voice family as a criterion for the selecting, wherein the vocal demeanor that is selected is configured to present speech using the particular voice family that was specified based on the evaluation of the at least one first attribute of the user'"'"'s interaction; and

incorporating the vocal demeanor into the application by presenting at least some speech content via the application in the particular voice family.

16. The computer program product of claim 15, wherein the at least one first attribute of the user'"'"'s interaction comprises a manner in which the user interacts with the application, wherein the user may interact with the application using a speech interface and/or at least one non-speech interface.

17. The computer program product of claim 15, wherein the at least one first attribute of the user'"'"'s interaction comprises at least one attribute of user speech.

18. The computer program product of claim 17, wherein the at least one first attribute of user speech comprises speech rate, voice family, pitch, pitch range, and richness.

19. The computer program product of claim 15, wherein incorporating the vocal demeanor into the application further comprises linking at least one markup element of a markup document of the application to a style of a cascading style sheet.