DIGITAL PERSONAL ASSISTANT INTERACTION WITH IMPERSONATIONS AND RICH MULTIMEDIA IN RESPONSES

• US 20150199967A1
• Filed: 01/15/2014
• Published: 07/16/2015
• Est. Priority Date: 01/15/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Malekzadeh, Sogol, Reddy, Mouni, Howard, Robert J. III, Harrison, Deborah B.
• Granted Patent: US 9,514,748 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

Systems, methods, apparatuses, and computer program products are described for implementing a digital personal assistant. The digital personal assistant is capable of determining that a user has asked a question or made a statement that is intended to engage with a persona of the digital personal assistant. In response to determining that the user has asked such a question or made such a statement, the digital personal assistant provides a response thereto by displaying or playing back a multimedia object associated with a popular culture reference within or by a user interface of the digital personal assistant. Additionally or alternatively, in response to determining that the user has asked such a question or made such a statement, the digital personal assistant provides the response thereto by generating or playing back speech that comprises an impersonation of a voice of a person associated with the popular culture reference.

Claims

1. A method for implementing a digital personal assistant, comprising:

receiving a digital representation of an utterance of a user of the digital personal assistant;

based on at least an analysis of the digital representation of the utterance, determining that the utterance comprises an utterance intended to engage with a persona of the digital personal assistant; and

in response to at least the determining that the utterance comprises an utterance intended to engage with the persona of the digital personal assistant, causing a response to the utterance to be generated by the digital personal assistant that includes at least one of;

a multimedia object that is displayed within or played back by a user interface of the digital personal assistant, the multimedia object being associated with a popular culture reference; and

speech that is generated or played back by the digital personal assistant, the speech comprising an impersonation of a voice of a person associated with the popular culture reference.

2. The method of claim 1, wherein the response to the utterance further includes text that is displayed within the user interface of the digital personal assistant, the text comprising a quotation associated with the popular culture reference.

3. The method of claim 1, wherein the speech comprises a quotation associated with the popular culture reference.

4. The method of claim 1, wherein the response to the utterance further includes a visual representation of the digital personal assistant that is displayed within the user interface thereof and that evokes the popular culture reference.

5. The method of claim 4, wherein the visual representation of the digital personal assistant comprises an animation of the digital personal assistant.

6. The method of claim 1, wherein the multimedia object comprises:

image or video content that is displayed within the user interface of the digital personal assistant or audio content that is played back by the user interface of the digital personal assistant.

7. The method of claim 1, wherein the response to the utterance further includes:

a link that is displayed within the user interface of the digital personal assistant and that can be activated by the user to access content associated with the popular culture reference.

8. The method of claim 1, wherein causing the response to the utterance to be generated comprises:

identifying a plurality of eligible responses to the utterance; and

selecting the response to the utterance from among the plurality of eligible responses to the utterance.

9. The method of claim 1, wherein causing the response to the utterance to be generated comprises:

matching the utterance to a particular utterance type within a hierarchical tree of utterance types, each utterance type in the hierarchical tree of utterance types having one or more responses associated therewith; and

selecting the response to the utterance from among the response(s) associated with the particular utterance type.

10. The method of claim 1, wherein causing the response to the utterance to be generated comprises:

determining that the utterance is associated with a trending topic; and

in response to determining that the utterance is associated with the trending topic, selecting the response to the utterance from among one or more responses associated with the trending topic.

11. The method of claim 1, wherein causing the response to the utterance to be generated comprises:

determining that the utterance is an utterance for which there are one or more predefined responses intended to convey the persona of the digital personal assistant; and

in response to determining that the utterance is an utterance for which there are one or more predefined responses intended to convey the persona of the digital personal assistant, selecting the response to the utterance from among the one or more predefined responses.

12. The method of claim 1, wherein causing the response to the utterance to be generated comprises:

sending an audio file that includes the speech or information that identifies the audio file to a computing device executing the digital personal assistant.

13. The method of claim 1, wherein causing the response to the utterance to be generated comprises:

providing text to a computing device executing the digital personal assistant, the text to be processed by a text-to-speech component of the digital personal assistant to generate the speech.

14. A computing device, comprising:

at least one processor; and

a memory that stores computer program logic for execution by the at least one processor, the computer program logic including one or more components configured to perform operations when executed by the at least one processor, the one or more components including;

a digital personal assistant configured to capture audio that represents an utterance of a user and to transmit the audio to a digital personal assistant backend, the utterance comprising an utterance intended to engage with a persona of the digital personal assistant;

the digital personal assistant being further configured to provide a response to the utterance based at least on information received from the digital personal assistant backend, providing the response including generating or playing back speech that comprises an impersonation of a voice of a person associated with a popular culture reference.

15. The computing device of claim 14, wherein the digital personal assistant is configured to play back an audio file that includes the speech.

16. The computing device of claim 14, wherein the digital personal assistant is configured to apply text-to-speech conversion to text to generate the speech.

17. The computing device of claim 14, wherein providing the response further includes at least one of:

displaying or playing back a multimedia object by a user interface of the digital personal assistant, the multimedia object being associated with the popular culture reference;

displaying text within the user interface of the digital personal assistant, the text comprising a quotation associated with the popular culture reference;

displaying a visual representation of the digital personal assistant that evokes the popular culture reference within the user interface of the digital personal assistant;

and

displaying a link within the user interface of the digital personal assistant that can be activated by the user to access content associated with the popular culture reference.

18. A computing device, comprising:

at least one processor; and

a memory that stores computer program logic for execution by the at least one processor, the computer program logic including one or more components configured to perform operations when executed by the at least one processor, the one or more components including;

a digital personal assistant configured to capture audio that represents an utterance of a user and to transmit the audio to a digital personal assistant backend, the utterance comprising an utterance intended to engage with a persona of the digital personal assistant;

the digital personal assistant being further configured to provide a response to the utterance based at least on information received from the digital personal assistant backend, providing the response including displaying or playing back a multimedia object associated with a popular culture reference by a user interface of the digital personal assistant.

19. The computing device of claim 18, wherein the multimedia object comprises an image, video content, or audio content.

20. The computing device of claim 18, wherein providing the response further includes at least one of:

generating or playing back speech that comprises an impersonation of a voice of a person associated with the popular culture reference;

displaying text within the user interface of the digital personal assistant, the text comprising a quotation associated with the popular culture reference;

displaying a visual representation of the digital personal assistant that evokes the popular culture reference within the user interface of the digital personal assistant; and

displaying a link within the user interface of the digital personal assistant that can be activated by the user to access content associated with the popular culture reference.