Systems and methods for determining user frustration when using voice control

• US 10,381,005 B2
• Filed: 11/28/2017
• Issued: 08/13/2019
• Est. Priority Date: 11/28/2017
• Status: Expired due to Fees

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Friedman, Scott A., Remegio, Prince R., Falkenmayer, Tim Uwe, Kyle, Roger Akira, Kakimi, Ryoma, Heide, Luke D., Puranik, Nishikant Narayan
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US15/824,627
• Publication Number:
  US 20190164544A1
• Time in Patent Office: 623 Days
• Field of Search: 704275
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04M 2203/2038 Call context notifications
• CPC Class Codes: H04M 3/487 Arrangements for providing information services, e.g. recorded voice services or time announcements
• CPC Class Codes: H04M 3/5166 in combination with interactive voice response systems or voice portals, e.g. as front-ends

Abstract

Systems, methods, and vehicle components for determining user frustration are disclosed. A method includes receiving, by a microphone communicatively coupled to a processing device, a voice input from a user, the voice input corresponding to an interaction between the user and a voice recognition system and including indicators of the user frustration. The method further includes determining, by the processing device, that the user is frustrated from the indicators, connecting, by the processing device, the user to a call center operator, transmitting, by the processing device, data to a call center computing device associated with the call center operator, the data corresponding to a user input and/or a vehicle response that resulted in the user frustration, receiving, by the processing device, commands from the call center computing device, and executing, by the processing device, processes that correspond to the commands.

Claims

1. A method for determining user frustration, the method comprising:

receiving, by a microphone communicatively coupled to a processing device, a voice input from a user, wherein;

the voice input corresponds to an interaction between the user and a voice recognition system, and

the voice input comprises one or more indicators of the user frustration;

determining, by the processing device, that the user is frustrated from the one or more indicators;

connecting, by the processing device, the user to a call center operator;

transmitting, by the processing device, data to a call center computing device associated with the call center operator, the data corresponding to one or more user inputs, one or more vehicle responses that resulted in the user frustration, and one or more vehicle system settings;

receiving, by the processing device, one or more commands from the call center computing device; and

executing, by the processing device, one or more processes that correspond to the one or more commands.

2. The method of claim 1, further comprising receiving, by the processing device, supplemental information corresponding to the one or more indicators of the user frustration.

3. The method of claim 2, wherein receiving the supplemental information comprises one or more of:

receiving one or more inputs supplied via input hardware; and

receiving biometric information regarding the user.

4. The method of claim 3, wherein receiving the one or more inputs comprise receiving one or more of the following:

a repeated activation of the same command, a cycling of menu items without selection of a function, and an increased grip on a component relative to a baseline grip.

5. The method of claim 1, wherein the one or more indicators comprise one or more of the following:

a voice tone, a keyword indicative of frustration, or a volume of the user'"'"'s voice.

6. The method of claim 1, further comprising, prior to receiving the voice input:

receiving, by the processing device, a first user input corresponding to a first user command;

determining, by the processing device, an action that corresponds to the first user command; and

completing, by the processing device, the action,

wherein the action does not correspond to an intended action by the user.

7. The method of claim 1, wherein receiving the voice input comprises monitoring one or more voice interactions between the user and the voice recognition system.

8. The method of claim 1, further comprising, prior to connecting the user to the call center operator:

prompting, by the processing device, the user to contact the call center operator; and

receiving, by the processing device, confirmation from the user in response to the prompting.

9. A system for determining user frustration, the system comprising:

a microphone;

a processing device communicatively coupled to the microphone; and

a non-transitory, processor-readable storage medium, the non-transitory, processor-readable storage medium comprising one or more programming instructions thereon that, when executed, cause the processing device to;

receive, from the microphone, data pertaining to a voice input from a user, wherein;

the voice input corresponds to an interaction between the user and a voice recognition system, and

the voice input comprises one or more indicators of the user frustration;

determine that the user is frustrated from the one or more indicators,

connect the user to a call center operator,

transmit data to a call center computing device associated with the call center operator, the data corresponding to one or more user inputs one or more vehicle responses that resulted in the user frustration, and one or more vehicle system settings,

receive one or more commands from the call center computing device, and

execute one or more processes that correspond to the one or more commands.

10. The system of claim 9, wherein the system is located in a vehicle.

11. The system of claim 9, wherein the non-transitory, processor-readable storage medium further comprises one or more programming instructions that, when executed, cause the processing device to receive supplemental information corresponding the one or more indicators of the user frustration.

12. The system of claim 11, wherein the one or more programming instructions that, when executed, cause the processing device to receive the supplemental information further cause the processing device to receive one or more of:

an input supplied via input hardware; and

biometric information regarding the user.

13. The system of claim 12, wherein the one or more inputs comprise one or more of the following:

a repeated activation of the same command, a cycling of menu items without selection of a function, and an increased grip on a component relative to a baseline grip.

14. The system of claim 9, wherein the one or more indicators comprise one or more of the following:

a voice tone, a keyword indicative of frustration, or a volume of the user'"'"'s voice.

15. The system of claim 9, wherein the non-transitory, processor-readable storage medium further comprises one or more programming instructions that, when executed, cause the processing device to:

receive a first user input corresponding to a first user command,

determine an action that corresponds to the first user command, and

complete the action,

wherein the action does not correspond to an intended action by the user.

16. The system of claim 9, wherein the one or more programming instructions that, when executed, cause the processing device to receive the voice input further cause the processing device to monitor one or more voice interactions between the user and the voice recognition system.

17. The system of claim 9, wherein the non-transitory, processor-readable storage medium further comprises one or more programming instructions that, when executed, cause the processing device to:

prompt the user to contact the call center operator; and

receive confirmation from the user in response to the prompt.

18. A vehicle component for determining user frustration, the vehicle component comprising:

a microphone configured to receive a voice input from a user, wherein;

the voice input corresponds to an interaction between the user and a voice recognition system, and

the voice input comprises one or more indicators of the user frustration; and

a processing device communicatively coupled to the microphone and configured to;

determine that the user is frustrated from the one or more indicators,

connect the user to a call center operator,

transmit data to a call center computing device associated with the call center operator, the data corresponding to one or more user inputs, one or more vehicle responses that resulted in the user frustration, and one or more vehicle system settings,

receive one or more commands from the call center computing device, and

execute one or more processes that correspond to the one or more commands.

19. The vehicle component of claim 18, wherein the processing device is further configured to receive supplemental information corresponding to the one or more indicators of the user frustration.

20. The vehicle component of claim 18, further comprising user interface hardware configured to:

prompt the user to contact the call center operator; and

receive confirmation from the user in response to the prompt.