Reporting operational metrics in speech-based systems

• US 9,536,527 B1
• Filed: 06/30/2015
• Issued: 01/03/2017
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Carlson, Ty Loren
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US14/755,957
• Time in Patent Office: 553 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 43/14 using software, i.e. software packages network security related monitoring H04L63/1408
• CPC Class Codes: H04R 1/08 Mouthpieces; Microphones; Attachments therefor
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A speech-based system is configured to use its audio-based user interface to present various types of device status information such as wireless signal strengths, communication parameters, battery levels, and so forth. In described embodiments, the system is configured to understand spoken user requests for device and system status. For example, the user may speak a request to obtain the current wireless signal strength of the speech-based system. The speech-based system may respond by determining the signal strength and by playing speech or other sound informing the user of the signal strength. Furthermore, the system may monitor operational parameters to detect conditions that may degrade the user experience, and may report such conditions using generated speech or other sounds.

Claims

1. A system comprising:

a microphone;

a loudspeaker;

a network communications interface configured to provide data communications with a network-based service;

one or more audio processing components configured to produce a first signal using the microphone, the first signal representing user speech;

one or more processors;

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform actions comprising;

sending the first signal to the network-based service using the network communications interface, the network-based service being configured to perform automatic speech recognition to determine that the user speech comprises a request for a status of data communications with the network-based service;

receiving, from the network-based service, a query for the status of data communications with the network-based service;

determining the status of data communications with the network-based service;

sending data to the network-based service indicating the status of communications with the network-based service;

receiving a second signal from the network-based service, the second signal representing speech indicating the status of data communications with the network-based service; and

producing the speech represented by the second signal using the loudspeaker.

2. The system of claim 1, further comprising:

an operating system that provides an interface management application programming interface (API), the interface management API providing information regarding the network communications interface; and

wherein determining the status of data communications with the network-based service comprises calling the interface management API to obtain one or more parameters relating to the status of data communications with the network-based service.

3. The system of claim 1, wherein:

the network communications interface comprises a wireless communications interface configured to communicate wirelessly with the network-based service; and

the system further comprises an operating system that provides a wireless interface management application programming interface (API), the wireless interface management API providing information regarding the wireless communications interface; and

wherein determining the status of data communications with the network-based service comprises calling the wireless interface management API to obtain a received signal strength indicator (RSSI) corresponding to the wireless communications interface.

4. The system of claim 1, wherein:

the network communications interface comprises a wireless communications interface configured to communicate with the network-based service using a wireless signal;

the status of data communications with the network-based service comprises a signal strength of the wireless signal; and

the actions further comprise;

detecting a decrease in the signal strength;

sending data to the network-based service indicating the decrease in the signal strength;

receiving a third signal from the network-based service, the third signal representing speech indicating the decrease in the signal strength; and

producing the speech represented by the third signal using the loudspeaker.

5. A method performed by an audio device, comprising:

generating a first signal using a microphone, the first signal representing first speech;

sending the first signal to a network-based service for recognition of the first speech that is represented by the first signal, wherein the first speech comprises a request for status information;

sending data indicating the status information to the network-based service;

receiving a second signal from the network-based service, the second signal representing a sound indicating the status information; and

producing the sound using a speaker.

6. The method of claim 5, further comprising repeatedly sending data indicating the status information to the network-based service prior to sending the first signal.

7. The method of claim 5, further comprising:

receiving, from the network-based service, a query for one or more status parameters relating to the status information.

8. The method of claim 5, wherein:

the second signal represents second speech including the status information; and

producing the sound comprises producing the speech that states the status information.

9. The method of claim 5, wherein the sound comprises speech that states the status information.

10. The method of claim 5, wherein the status information comprises a performance parameter, and the method further comprising receiving an audio signal from the network-based service, wherein the audio signal represents speech that includes instructions regarding how to improve the performance parameter.

11. The method of claim 5, wherein:

the status information comprises a performance parameter; and

producing the sound comprises producing a tone that varies to indicate a value of the performance parameter.

12. The method of claim 5, wherein producing the sound comprises producing an earcon corresponding to the status information.

13. The method of claim 5, wherein:

the communicating comprises communicating over a wireless communication channel using a radio-frequency signal; and

the status information comprises a received signal strength indicator of the radio-frequency signal.

14. The method of claim 5, wherein the status information comprises one or more of:

a setting of the audio device;

a performance indicator of the audio device;

a network setting;

a wireless signal strength;

network status information;

a network performance indicator;

battery information;

status of communications by the audio device with a peripheral device;

status of communications with the network-based service;

status of the network-based service;

firmware or software version information;

or

diagnostic information.

22. The method of claim 5, further comprising:

detecting the status information; and

determining that the status information is below a threshold,

and wherein sending the data is based at least in part on determining that the status information is below the threshold.

15. A method, comprising:

detecting, by an audio device, a status of the audio device;

sending, by the audio device, to a network-based service, data indicating the status of the audio device;

receiving, by the audio device, a first audio signal from the network-based service, the first audio signal representing first speech that states information regarding the status; and

producing, by the audio device, the first speech using a speaker.

16. The method of claim 15, wherein:

the network connection comprises a wireless communication channel using a radio-frequency signal; and

detecting the status comprises detecting a decrease in a signal strength of the radio-frequency signal.

17. The method of claim 15, wherein:

detecting the status comprises detecting a decrease in a performance parameter; and

the first speech states instructions regarding how to improve the performance parameter.

18. The method of claim 15, wherein:

detecting the status comprises detecting a decrease in a performance parameter, the performance parameter indicating a quality of communications with the network-based service; and

the first speech states instructions regarding how to improve the performance parameter.

19. The method of claim 15, wherein detecting the status of the audio device comprises:

obtaining a performance parameter; and

comparing the performance parameter to a threshold.

20. The method of claim 15, further comprising sending a second audio signal to the network-based service for recognition of second speech that is represented by the second audio signal.

21. The method of claim 15, wherein the first speech indicates one or more of:

a setting of the audio device;

a performance indicator of the audio device;

a network setting;

a wireless signal strength;

network status information;

a network performance indicator;

battery information;

status of communications by the audio device with a peripheral device;

status of communications with the network-based service;

status of the network-based service;

firmware or software version information;

or

diagnostic information.

23. The method of claim 15, further comprising repeatedly sending, by the audio device, via the network connection, data indicating the status to the network-based service.