Coordinating the execution of a voice command across multiple connected devices

• US 10,255,917 B2
• Filed: 03/31/2017
• Issued: 04/09/2019
• Est. Priority Date: 03/31/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Carey, James E., Cradick, Ryan K., Kleinheksel, Cory J.
• Primary Examiner(s): Azad, Abul K
• Application Number:
  US15/475,772
• Publication Number:
  US 20180286391A1
• Time in Patent Office: 739 Days
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/0364 for improving intelligibility

Abstract

A computer-implemented method includes exchanging device data, associated with a first participating user device, with the one or more second participating user devices; receiving audio data associated with a voice command; exchanging the audio data with the one or more second participating user devices; identifying, by the first participating user device, a voice command based on exchanging the audio data; determining which one of the first participating user device or the one or more second participating user devices should respond to the voice command based on details of the voice command and the exchanging the device data; responding to the voice command based on determining that the first participating user device should respond to the voice command; and signaling a particular second participating user device to respond to the voice command based on determining that the particular second participating user device should respond to the voice command.

Claims

1. A computer-implemented method comprising:

exchanging, by a first participating user device, device data, associated with the first participating user device, with the one or more second participating user devices;

receiving, by the first participating user device, audio data associated with a voice command;

exchanging, by the first participating user device, the audio data with the one or more second participating user devices;

forming, by the first participating user device, a combined audio data object based on the exchanging the audio data, wherein the combined audio data object includes most audibly legible portions of audio data obtained by each of the first participating user devices and the one or more second participating user devices;

identifying, by the first participating user device, a voice command based on exchanging the audio data;

determining, by the first participating user device, which one of the first participating user device or the one or more second participating user devices should respond to the voice command based on details of the voice command and the exchanging the device data, the determining further comprising coordinating with the one or more second participating user devices to make a final determination as to which one of the first participating user device or the one or more second participating user devices, should respond to the voice command, wherein the making the final determination is based on scores representing which one of the first participating user device or the one or more second participating user devices, should respond to the voice command;

responding, by the first participating user device, to the voice command based on determining that the first participating user device should respond to the voice command; and

signaling, by the first participating user device, a particular second participating user device to notify the particular second participating user device to respond to the voice command based on determining that the particular second participating user device should respond to the voice command.

2. The method of claim 1, wherein the device data is selected from a group consisting of:

device activity information;

device keywords;

device capabilities; and

relationships and connections with other devices.

3. The method of claim 1, wherein the exchanging the device data includes providing the device data, associated with the first participating user device, to the one or more second participating users via a communications link and receiving, from each of the one or more second participating user devices, device data associated with each of the one or more second participating user devices.

4. The method of claim 1, wherein the exchanging the audio data includes providing the audio data, obtained by the first participating user device, to the one or more second participating users via a communications link and receiving, from each of the one or more second participating user devices, audio data obtained by each of the one or more second participating user devices.

5. The method of claim 1, further comprising establishing a communications link with the one or more second participating user devices, wherein the establishing the communications link comprises discovering the one or more second participating user devices through at least one selected from the group consisting of:

a local area network;

a personal area network; and

a near field communications network.

6. The method of claim 5, wherein the establishing the communications link is based on security policies permitting the establishing the communications link to be established with the one or more second user devices.

7. The method of claim 1, wherein a service provider at least one of creates, maintains, deploys and supports the computing device.

8. The method of claim 1, wherein the establishing the communications link, the exchanging the device data, the receiving the audio data, the exchanging the audio data, the forming the combined audio data object, the identifying the enhanced voice command, the identifying which one of the first participating user device or the one or more second participating user devices should respond to the enhanced voice command, the responding to the enhanced voice command, and the signaling the signaling the particular second participating user device to respond to the enhanced voice command are provided by a service provider on a subscription, advertising, and/or fee basis.

9. The method of claim 1, wherein the computing device includes software provided as a service in a cloud environment.

10. The method of claim 1, further comprising deploying a system for improving the performance of recognizing and executing voice commands, comprising providing a computer infrastructure operable to perform the establishing the communications link, the exchanging the device data, the receiving the audio data, the exchanging the audio data, the forming the combined audio data object, the identifying the enhanced voice command, the identifying which one of the first participating user device or the one or more second participating user devices should respond to the enhanced voice command, the responding to the enhanced voice command, and the signaling the signaling the particular second participating user device to respond to the enhanced voice command.

11. A computer program product for improving the performance of recognizing and executing voice commands, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a first participating user device to cause the first participating user device to:

establish a communications link with one or more second participating user devices;

exchange, via the communications link, device data, associated with the first participating user device, with the one or more second participating user devices;

form a combined audio data object based on the exchanging the audio data, wherein the combined audio data object includes most audibly legible portions of audio data obtained by each of the first participating user devices and the one or more second participating user devices;

receive audio data associated with a voice command;

determine which one of the first participating user device or the one or more second participating user devices should respond to the voice command based on details of the voice command and the exchanging the device data, the determining further comprising coordinating with the one or more second participating user devices to make a final determination as to which one of the first participating user device or the one or more second participating user devices, should respond to the voice command, wherein the making the final determination is based on scores representing which one of the first participating user device or the one or more second participating user devices, should respond to the voice command;

respond to the voice command based on determining that the first participating user device should respond to the enhanced voice command; and

signal a particular second participating user device to respond to the voice command based on determining that the particular second participating user device should responds to the voice command.

12. The computer program product of claim 11, wherein the device data is selected from a group consisting of:

device activity information;

device keywords;

device capabilities; and

relationships and connections with other devices.

13. The computer program product of claim 11, wherein the exchanging the device data includes providing the device data, associated with the first participating user device, to the one or more second participating users via the communications link and receiving, from each of the one or more second participating user devices, device data associated with each of the one or more second participating user devices.

14. The computer program product of claim 11, wherein the establishing the communications link with the one or more second participating user devices comprises discovering the one or more second participating user devices through at least one selected from the group consisting of:

a local area network;

a personal area network; and

a near field communications network.

15. The computer program product of claim 14, wherein the establishing the communications link is based on security policies permitting the establishing the communications link to be established with the one or more second user devices.

16. A system comprising:

a CPU, a computer readable memory and a computer readable storage medium associated with a first participating user device;

program instructions to establish a communications link with one or more second participating user devices;

program instructions to receive audio data associated with a voice command;

program instructions to exchange, via the communications link, the audio data with the one or more second participating user devices;

program instructions to form a combined audio data object based on the exchanging the audio data, wherein the combined audio data object includes most audibly legible portions of audio data obtained by each of the first participating user devices and the one or more second participating user devices;

program instructions to identify an enhanced voice command from the combined audio data object;

program instructions to determine which one of the first participating user device or the one or more second participating user devices should respond to the enhanced voice command based on details of the enhanced voice command;

program instructions to coordinate with the one or more second participating user devices to make a final determination as to which one of the first participating user device or the one or more second participating user devices, should respond to the voice command, wherein the making the final determination is based on scores representing which one of the first participating user device or the one or more second participating user devices, should respond to the voice command;

program instructions to respond to the enhanced voice command based on determining that the first participating user device should respond to the enhanced voice command; and

program instructions to signal a particular second participating user device to respond to the enhanced voice command based on determining that the particular second participating user device should responds to the enhanced voice command,

wherein the program instructions are stored on the computer readable storage medium for execution by the CPU via the computer readable memory.