Dynamic volume adjustment for virtual assistants

• US 10,705,789 B2
• Filed: 07/25/2018
• Issued: 07/07/2020
• Est. Priority Date: 07/25/2018
• Status: Active Grant

• Current Assignee: Sensory Incorporated
• Original Assignee: Sensory Incorporated
• Inventors: Mozer, Todd F.
• Primary Examiner(s): Zhu, Richard Z
• Application Number:
  US16/045,560
• Publication Number:
  US 20200034108A1
• Time in Patent Office: 713 Days
• Field of Search: None
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 50/265 Personal security, identity or safety
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H03G 3/02 Manually-operated control H03G3/001 and H03G3/002 take precedence
• CPC Class Codes: H03G 3/3005 in amplifiers suitable for low-frequencies, e.g. audio amplifiers H03G3/32, H03G3/34 take precedence
• CPC Class Codes: H03G 3/3089 Control of digital or coded signals
• CPC Class Codes: H03G 3/32 the control being dependent upon ambient noise level or sound level

Abstract

Techniques for implementing dynamic volume adjustment by a virtual assistant are provided. In one embodiment, the virtual assistant can receive a voice query or command from a user, recognize the content of the voice query or command, process the voice query or command based on the recognized content, and determine an auditory response to be output to the user. The virtual assistant can then identify a plurality of criteria for automatically determining an output volume level for the response, where the plurality of criteria including content-based criteria and environment-based criteria, calculate values for the plurality of criteria, and combine the values to determine the output volume level. The virtual assistant can subsequently cause the auditory response to be output to the user at the determined output volume level.

Claims

1. A method comprising:

receiving, by a virtual assistant, a first voice query or command from a user;

recognizing, by the virtual assistant, content of the first voice query or command;

processing, by the virtual assistant, the first voice query or command based on the recognized content;

subsequently to the processing, determining, by the virtual assistant, a first auditory response to be output to the user;

identifying, by the virtual assistant, a first plurality of criteria for automatically determining a first output volume level for the first auditory response, the first plurality of criteria including content-based criteria and environment-based criteria, the content-based criteria including an identity criterion that is based on an identity of the user;

calculating, by the virtual assistant, a first set of values for the first plurality of criteria;

combining, by the virtual assistant, the first set of values to determine the first output volume level;

causing, by the virtual assistant, the first auditory response to be output to the user at the first output volume level;

after the first auditory response has been output to the user at the first output volume level, receiving, by the virtual assistant, a verbal signal from the user that the first auditory response is too loud;

in response to receiving the verbal indication from the user that the auditory response is too loud, saving, by the virtual assistant, the first set of values for the first plurality of criteria and an indication of the verbal signal; and

upon receiving a second voice query or command from the user;

determining a second auditory response to be output to the user;

identifying a second plurality of criteria for automatically determining a second output volume level for the second auditory response;

calculating a second set of values for the second plurality of criteria; and

comparing the second set of values for the second plurality of criteria with the saved first set of values for the first plurality of criteria; and

if the second set of values is determined to be substantially similar to the first set of values;

setting the second output volume level to be lower than the first output volume level; and

causing the second auditory response to be output to the user at the second output volume level.

2. The method of claim 1 wherein the content-based criteria pertain to the content of the first auditory response.

3. The method of claim 2 wherein the content-based criteria include an importance criterion that is based on an importance of the content.

4. The method of claim 3 wherein calculating a value for the importance criterion comprises:

classifying the content as one of a plurality of predefined content types; and

identifying an importance level mapped to the classified content type.

5. The method of claim 4 wherein the importance level is defined by the user.

6. The method of claim 4 wherein content types relating to user privacy or financial transactions are mapped to relatively higher importance levels.

7. The method of claim 1 wherein the environment-based criteria pertain to current auditory conditions surrounding a device implementing the virtual assistant or the user.

8. The method of claim 7 wherein the environment-based criteria include a distance criterion that is based on a distance between the device and the user.

9. The method of claim 8 wherein the distance is determined using computer vision, echo-location, or radar.

10. The method of claim 7 wherein the environment-based criteria include an ambient noise criterion that is based on an amount of ambient noise around the device or the user.

11. The method of claim 1 wherein combining the first set of values for the first plurality of criteria to determine the first output volume level comprises:

identifying weights associated with the first plurality of criteria; and

combining the first set of values in a manner that takes into account the weights.

12. The method of claim 11 wherein the weights are defined by the user.

13. A non-transitory computer readable medium having stored thereon program code executable by a processor of a computing device, the program code comprising:

code that causes the processor to receive a first voice query or command from a user;

code that causes the processor to recognize content of the first voice query or command;

code that causes the processor to process the first voice query or command based on the recognized content;

code that causes the processor to determine a first auditory response to be output to the user;

code that causes the processor to identify a first plurality of criteria for automatically determining a first output volume level for the first auditory response, the first plurality of criteria including content-based criteria and environment-based criteria, the content-based criteria including an identity criterion that is based on an identity of the user;

code that causes the processor to calculate a first set of values for the first plurality of criteria;

code that causes the processor to combine the first set of values to determine the first output volume level;

code that causes the processor to output the first auditory response to the user at the first output volume level;

code that causes the processor to receive, after the first auditory response has been output to the user at the first output volume level, a verbal signal from the user that the first auditory response is too loud;

code that causes the processor to save, in response to receiving the verbal indication from the user that the auditory response is too loud, the first set of values for the plurality of criteria and an indication of the verbal signal; and

code that causes the processor to, upon receiving a second voice query or command from the user;

determine a second auditory response to be output to the user;

identify a second plurality of criteria for automatically determining a second output volume level for the second auditory response;

calculate a second set of values for the second plurality of criteria; and

compare the second set of values for the second plurality of criteria with the saved first set of values for the first plurality of criteria; and

if the second set of values is determined to be substantially similar to the first set of values;

set the second output volume level to be lower than the first output volume level; and

cause the second auditory response to be output to the user at the second output volume level.

14. The non-transitory computer readable medium of claim 13 wherein the content-based criteria pertain to the content of the auditory response, and wherein the environment-based criteria pertain to current auditory conditions surrounding the computing device or the user.

15. A computing device comprising:

a processor; and

a memory having stored thereon program code which, when executed by the processor, causes the processor to;

receive a first voice query or command from a user;

recognize content of the first voice query or command;

process the first voice query or command based on the recognized content;

determine a first auditory response to be output to the user;

identify a first plurality of criteria for automatically determining a first output volume level for the first auditory response, the first plurality of criteria including content-based criteria and environment-based criteria, the content-based criteria including an identity criterion that is based on an identity of the user;

calculate a first set of values for the first plurality of criteria;

combine the first set of values to determine the first output volume level;

output the first auditory response to the user at the first output volume level;

receive, after the first auditory response has been output to the user at the first output volume level, a verbal signal from the user that the first auditory response is too loud;

save, in response to receiving the verbal indication from the user that the auditory response is too loud, the first set of values for the plurality of criteria and an indication of the verbal signal; and

upon receiving a second voice query or command from the user;

determine a second auditory response to be output to the user;

identify a second plurality of criteria for automatically determining a second output volume level for the second auditory response;

calculate a second set of values for the second plurality of criteria; and

compare the second set of values for the second plurality of criteria with the saved first set of values for the first plurality of criteria; and

if the second set of values is determined to be substantially similar to the first set of values;

set the second output volume level to be lower than the first output volume level; and

cause the second auditory response to be output to the user at the second output volume level.

16. The computing device of claim 15 wherein the content-based criteria pertain to the content of the auditory response, and wherein the environment-based criteria pertain to current auditory conditions surrounding the computing device or the user.