Personalized and contextualized audio briefing

• US 10,402,450 B2
• Filed: 05/11/2017
• Issued: 09/03/2019
• Est. Priority Date: 05/13/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Goodman, Michael Andrew, Xu, Bibo
• Primary Examiner(s): Wozniak, James S
• Application Number:
  US15/593,236
• Publication Number:
  US 20170329848A1
• Time in Patent Office: 845 Days
• Field of Search: 704251, 704270, 7042701, 704275
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 1/181 Enclosures for portable computers G06F1/1613
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: H04L 65/1094 Inter-user-equipment sessions transfer or sharing
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04R 1/323 for loudspeakers H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 2201/028 Structural combinations of loudspeakers with built-in power amplifiers, e.g. in the same acoustic enclosure H04R2499/10 takes precedence; Single (sub)woofer with two or more satellite loudspeakers for mid- and high-frequency band reproduction driven via the (sub)woofer H04R2205/026
• CPC Class Codes: H04R 2227/005 : Audio distribution systems for home, i.e. multi-room use

Abstract

A method at an electronic device with an audio input device and an audio output device includes: receiving through the audio input device a verbal input from a user; transmitting information corresponding to the verbal input to a remote system; receiving from the remote system a response responsive to the verbal input, the response including information in accordance with one or more criteria; and outputting the response through the audio output device.

Claims

1. A method of providing a personalized audio briefing to a user, comprising:

at an electronic device comprising memory, one or more processors, an audio input device, and an audio output device;

receiving from the user identification of one or more information sources associated with the user, wherein each of the information sources has a content type selected from a plurality of content types;

receiving from the user an authorization to access the one or more identified information sources;

receiving from the user order preference information associated with a preferred order of the content types for presentation within the audio briefing;

transmitting to a remote system the identification of one or more information sources, the authorization, and the order preference information;

receiving through the audio input device a verbal input from the user requesting the audio briefing;

in response to the verbal input, transmitting to the remote system the verbal input and device information associated with a device type of the electronic device;

following the transmitting of the verbal input and device information associated with a device type of the electronic device;

receiving a response responsive to the verbal input generated by the remote system including information from the one or more information sources arranged in a modified order distinct from the preferred order, wherein the modified order is determined by the remote system based at least in part on the order preference information, the device information associated with the device type of the electronic device, and one or more real-time criteria associated with the user and obtained by the remote system in conjunction with the verbal input; and

outputting in the modified order the response through the audio output device.

2. The method of claim 1, wherein the one or more criteria comprise a time of day.

3. The method of claim 2, wherein:

the personalized audio briefing includes a default salutation; and

outputting in the modified order the response through the audio output device further comprises outputting a modified salutation distinct from the default salutation depending on the time of day.

4. The method of claim 1, wherein the audio briefing includes public information and personal information associated with the user.

5. The method of claim 1, wherein the one or more criteria comprise a location of the electronic device.

6. The method of claim 1, wherein the one or more criteria comprise a usage history of the electronic device by the user.

7. The method of claim 4, wherein the authorization to access the one or more information sources includes authorization to access the personal information associated with the user.

8. The method of claim 1, wherein the response is generated by the remote system in real-time.

9. The method of claim 1, wherein the response is streamed from the remote system to the electronic device in real-time, and receiving the response responsive to the verbal input comprises receiving the streamed response and outputting in the modified order the streamed response in real-time.

10. The method of claim 1, wherein:

the electronic device is communicatively connected to a client device of the user, the client device distinct from the electronic device; and

receiving from the user order preference information comprises receiving the order preference information from the client device.

11. An electronic device, comprising

an audio input device;

an audio output device;

one or more processors; and

memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for;

receiving from the user identification of one or more information sources associated with the user, wherein each of the information sources has a content type selected from a plurality of content types;

receiving from the user an authorization to access the one or more identified information sources;

receiving from the user order preference information associated with a preferred order of the content types for presentation within the audio briefing;

transmitting to a remote system the identification of one or more information sources, the authorization, and the order preference information;

receiving through the audio input device a verbal input from the user requesting the audio briefing;

in response to the verbal input, transmitting to a remote system the verbal input and device information associated with a device type of the electronic device;

to following the transmitting of the verbal input and device information associated with a device type of the electronic device;

receiving a response responsive to the verbal input generated by the remote system including information from the one or more information sources arranged in a modified order distinct from the preferred order, wherein the modified order is determined by the remote system based at least in part on the order preference information, the device information associated with the device type of the electronic device, and one or more real-time criteria associated with the user and obtained by the remote system in conjunction with the verbal input; and

outputting in the modified order the response through the audio output device.

12. The electronic device of claim 11, wherein the one or more criteria comprise a time of day.

13. The electronic device of claim 12, wherein the instructions for outputting in the modified order the response through the audio output device further comprises instructions for omitting one or more of the content types for presentation based on the time of day.

14. The electronic device of claim 11, wherein the one or more criteria comprise a location of the electronic device.

15. The electronic device of claim 11, wherein:

the device includes one or more visual indicators; and

outputting in the modified order the response further comprises displaying a status of the electronic device using the one or more visual indicators.

16. The electronic device of claim 11, wherein the one or more programs comprise instructions for:

while outputting the response, receiving a predefined command from the user; and

responsive to the predefined command, terminating the outputting.

17. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with an audio input device, an audio output device, and one or more processors, cause the electronic device to:

receive from the user identification of one or more information sources associated with the user, wherein each of the information sources has a content type selected from a plurality of content types;

receive from the user an authorization to access the one or more identified information sources;

receive from the user order preference information associated with a preferred order of the content types for presentation within the audio briefing;

transmit to a remote system the identification of one or more information sources, the authorization, and the order preference information;

receive through the audio input device a verbal input from the user requesting the audio briefing;

in response to the verbal input, transmit to a remote system the verbal input and device information associated with a device type of the electronic device;

following the transmitting of the verbal input and device information associated with a device type of the electronic device;

receive a response responsive to the verbal input generated by the remote system including information from the one or more information sources arranged in a modified order distinct from the preferred order, wherein the modified order is determined by the remote system based at least in part on the order preference information, the device information associated with the device type of the electronic device, and one or more real-time criteria associated with the user and obtained by the remote system in conjunction with the verbal input; and

output in the modified order the response through the audio output device.

18. The non-transitory computer readable storage medium of claim 17, wherein the one or more criteria comprise a time of day.

19. The non-transitory computer readable storage medium of claim 17, wherein the one or more criteria comprise a location of the electronic device.

20. The non-transitory computer readable storage medium of claim 17, wherein the information includes information associated with one or more categories specified by the user.