EXTERNAL VISUAL INTERACTIONS FOR SPEECH-BASED DEVICES

• US 20160162259A1
• Filed: 12/05/2014
• Published: 06/09/2016
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Zhao, Weidong, Caldwell, Gary, Bhargava, Ruchi
• Granted Patent: US 11,327,711 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58

Abstract

Examples are disclosed herein that are related to providing extended functionalities on-demand to an audio-based wearable device. One example provides a wearable computing device including an acoustic receiver configured to receive speech inputs, a speaker configured to present audio outputs, a communications subsystem configured to connect to an external device, a logic subsystem configured to execute instructions, and a storage subsystem having instructions executable by the logic subsystem to execute a program, connect to the external device via a wireless communications protocol, conduct an audio-based interaction of the program via the speech inputs received at the acoustic receiver and the audio outputs provided by the speaker, upon reaching a screen-based interaction of the program, notify a user via the speaker to interact with the external device, and provide image data to the external device for presentation via a screen of the external device.

Claims

1. A wearable computing device, comprising:

an acoustic receiver configured to receive speech inputs;

a speaker configured to present audio outputs;

a communications subsystem configured to connect to an external device;

a logic subsystem configured to execute instructions; and

a storage subsystem comprising instructions executable by the logic subsystem to

execute a program,

connect to the external device via a wireless communications protocol,

conduct an audio-based interaction of the program via the speech inputs received at the acoustic receiver and the audio outputs provided by the speaker,

upon reaching a screen-based interaction of the program, notify a user via the speaker to interact with the external device, and

provide image data to the external device for presentation via a screen of the external device.

2. The wearable computing device of claim 1, wherein the wireless communications protocol is a first wireless communications protocol, and wherein the instructions are executable by the logic subsystem to connect to the external device also via a second wireless communications protocol.

3. The wearable computing device of claim 2, wherein the instructions are executable by the logic subsystem to monitor one or more performance characteristics for each of the first wireless communications protocol and the second wireless communications protocol, and to communicate with the external device via one of the first wireless communications protocol and the second wireless communications protocol based upon the one or more performance characteristics.

4. The wearable computing device of claim 1, wherein the instructions are executable by the logic subsystem to connect to the external device by detecting a beacon signal emitted by the external device.

5. The wearable computing device of claim 1, wherein the instructions are executable by the logic subsystem to connect to one or more additional external devices.

6. The wearable computing device of claim 1, wherein the instructions are executable by the logic subsystem to receive from the external device information regarding one or more of an orientation, a frame rate, and a resolution of the screen of the external device.

7. The wearable computing device of claim 1, wherein the instructions are executable by the logic subsystem to receive user inputs for the program made via the external device.

8. The wearable computing device of claim 7, wherein the user inputs comprise position signals corresponding to touch inputs on the external device.

9. A wearable computing device comprising:

an acoustic receiver configured to receive speech inputs;

a speaker configured to present audio outputs;

a communications subsystem configured to connect to an external device and a remote service;

a logic subsystem configured to execute instructions; and

a storage subsystem comprising instructions executable by the logic subsystem to

connect to the remote service via a computer network,

connect to the external device,

execute a program locally on the wearable computing device,

conduct an audio-based interaction of the program via the speech inputs received at the acoustic receiver and the audio outputs provided by the speaker,

upon reaching a screen-based interaction of the program, notify a user to interact with the external device,

instruct the external device to obtain the image data from the remote service for the screen-based interaction, and

receive user input information from the external device during the screen-based interaction.

10. The wearable computing device of claim 9, wherein the instructions are executable by the logic subsystem to connect to the external device via a first wireless communications protocol and a second wireless communications protocol, to monitor one or more performance characteristics for each of the first wireless communications protocol and the second wireless communications protocol, and to communicate with the external device via one of the first wireless communications protocol and the second wireless communications protocol based upon the one or more performance characteristics.

11. The wearable computing device of claim 9, wherein the instructions are executable by the logic subsystem to receive the user input information from the external device via the remote service.

12. The wearable computing device of claim 9, wherein the instructions are executable by the logic subsystem to receive the user input information directly from the external device.

13. The wearable computing device of claim 12, wherein the instructions are executable by the logic subsystem to forward the user input information received directly from the external device to the remote service.

14. The wearable computing device of claim 9, wherein the instructions are executable by the logic subsystem to receive from the external device information regarding one or more of an orientation, a frame rate, and a resolution of a screen of the external device.

15. The wearable computing device of claim 9, wherein the user input information comprises position signals corresponding to touch inputs on the external device.

16. On a wearable computing device comprising an acoustic receiver and a speaker, a method comprising:

executing a program;

connecting to an external device via a wireless communications protocol;

conducting an audio-based interaction of the program via speech inputs received at the acoustic receiver and audio outputs provided by the speaker;

upon reaching a screen-based interaction of the program, notifying a user via the speaker to interact with the external device;

receiving from the external device orientation information regarding an orientation of a screen of the external device;

providing image data to the external device based upon the orientation information for presentation via the screen of the external device; and

receiving user input information from the external device during the screen-based interaction.

17. The method of claim 16, wherein the wireless communications protocol is a first wireless communications protocol, and further comprising connecting to the external device also via a second wireless communications protocol.

18. The method of claim 17, further comprising monitoring one or more performance characteristics for each of the first wireless communications protocol and the second wireless communications protocol, and communicating with the external device via one of the first wireless communications protocol and the second wireless communications protocol based upon the one or more performance characteristics.

19. The method of claim 16, wherein the user input information comprises position signals corresponding to touch inputs on the external device.

20. The method of claim 16, further comprising connecting a remote service via a computer network, wherein receiving the user input information from the external device comprises receiving the user input information via the remote service.