Systems and Methods for Enhancing Responsiveness to Utterances Having Detectable Emotion

• US 20190325866A1
• Filed: 04/20/2018
• Published: 10/24/2019
• Est. Priority Date: 04/20/2018
• Status: Abandoned Application

• Current Assignee: Spotify AB (Spotify Technology SA)
• Original Assignee: Spotify AB (Spotify Technology SA)
• Inventors: BROMAND, Daniel, GUSTAFSSON, David, MITIC, Richard, MENNICKEN, Sarah
• Application Number:
  US15/958,453
• Publication Number:
  US 20190325866A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/0335 Pitch control
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

Methods, systems, and related products that provide emotion-sensitive responses to user'"'"'s commands and other utterances received at an utterance-based user interface. Acknowledgements of user'"'"'s utterances are adapted to the user and/or the user device, and emotions detected in the user'"'"'s utterance that have been mapped from one or more emotion features extracted from the utterance. In some examples, extraction of a user'"'"'s changing emotion during a sequence of interactions is used to generate a response to a user'"'"'s uttered command. In some examples, emotion processing and command processing of natural utterances are performed asynchronously.

Claims

1. A method comprising:

receiving a natural utterance via a user device, the natural utterance including a command and at least one emotion feature;

providing a digital representation of the natural utterance to each of an emotion processor and a command processor;

processing the digital representation with the emotion processor by extracting the at least one emotion feature from the digital representation; and

processing the digital representation with the command processor by extracting the command from the digital representation,

wherein the processing the digital representation with the emotion processor and the processing the digital representation with the command processor are performed in parallel.

2. The method of claim 1, wherein the processing the digital representation with the emotion processor includes mapping the at least one emotion feature to an emotion.

3. The method of claim 2, wherein the mapping is partially based on an identity of a user.

4. The method of claim 2, wherein the mapping is partially based on a type of the user device.

5. The method of claim 1, wherein the processing the digital representation with the command processer includes mapping the extracted command to an action associated with the extracted command.

6. The method of claim 2, further comprising providing a verbalized acknowledgement adapted to the emotion.

7. The method of claim 1, wherein the at least one emotion feature is one or more of a cadence, a volume, a pitch, a word, a string of words, a pace, and a tone of the natural utterance.

8. A system comprising:

a command processor;

the command processor being configured to receive a digital representation of a natural utterance and process the digital representation of the natural utterance by extracting a command from the digital representation; and

an emotion processor, the emotion processor being configured to process the digital representation of the natural utterance by extracting an emotion feature from the digital representation,

wherein the system is configured such that the processing by the command processor and the processing by the emotion processor are performed in parallel.

9. The system of claim 8, wherein the emotion processor is configured to map the at least one emotion feature to an emotion.

10. The system of claim 9, wherein the at least one emotion feature is mapped partially based on an identity of a user uttering the natural utterance.

11. The system of claim 9, wherein the at least one emotion feature is mapped partially based on a type of user device that receives the natural utterance.

12. The system of claim 8, wherein the system is configured to perform an action associated with the extracted command.

13. The system of claim 9, wherein the system comprises a response processor configured to provide a verbalized acknowledgement adapted to the emotion.

14. The system of claim 8, wherein the at least one emotion feature is one or more of a cadence, a volume, a pitch, a word, a string of words, a pace, and a tone of the natural utterance.

15. A non-transitory computer readable medium comprising:

a command processor having one or more first sequences of instructions which, when executed by the command processor, cause the command processor to process a digital representation of a natural utterance by extracting a command from the digital representation;

an emotion processor having one or more second sequences of instructions which, when executed by the emotion processor, cause the emotion processor to process the digital representation of the natural utterance by extracting an emotion feature from the digital representation,

wherein the one or more first sequences of instructions and the one or more second sequences of instructions are executed in parallel.