DEVICE ARBITRATION FOR LISTENING DEVICES

• US 20160155443A1
• Filed: 04/01/2015
• Published: 06/02/2016
• Est. Priority Date: 11/28/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: KHAN, YASER, UZELAC, ALEKSANDAR, Hwang, Daniel J., Paolantonio, Sergio, Kam, Jenny, Kannan, Vishwac Sena, Mooney, II, Dennis James, Brush, Alice Jane Bernheim
• Granted Patent: US 9,812,126 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3209 Monitoring remote activity, e.g. over telephone lines or network connections
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

An electronic device in a topology of interconnected electronic devices can listen for a wake phrase and voice commands. The device can control when and how it responds so that a single device responds to voice commands. Per-task device preferences can be stored for a user. If a preferred device is not available, the task can still be performed on a device that has appropriate capabilities. Machine learning can determine a user'"'"'s preferences. Power conservation and effective user interaction can result.

Claims

1. A method of controlling which electronic device out of topology of interconnected electronic devices responds to a wake phrase, the method comprising:

in an electronic device of the topology of interconnected electronic devices configured to listen for voice commands, receiving a wake phrase from a microphone of the electronic device;

recognizing the wake phrase; and

responsive to recognizing the wake phrase, waking the electronic device and controlling which one electronic device out of the topology of interconnected electronic devices plays an audio prompt that indicates an active listening state in response to the wake phrase, wherein the controlling invokes response arbitration rules selecting a single electronic device out of the topology of interconnected electronic devices according to a user preference indicating a primary device designation for the interconnected electronic devices or recorded activity detected by one or more hardware sensors of the electronic device.

2. The method of claim 1 wherein:

the response arbitration rules specify rules of succession for devices when a primary device is not available.

3. The method of claim 1 wherein:

the recorded activity indicates physical activity near or at the electronic device; and

the controlling chooses an electronic device having most recent physical activity.

4. The method of claim 3 wherein:

the electronic device having most recent physical activity is chosen in response to determining that a primary device is not available.

5. The method of claim 1 wherein:

the controlling action chooses an electronic device designated as a primary device by the user preference.

6. The method of claim 1 further comprising:

determining whether the wake phrase was spoken by an authorized user;

the wake phrase is not recognized when not spoken by an authorized user.

7. The method of claim 1 wherein:

the recognizing is performed by an auxiliary processor of the electronic device while the electronic device is in a standby state.

8. The method of claim 7 wherein:

the controlling is performed by a main processor of the electronic device after the electronic device has transitioned out of the standby state; and

the electronic device consumes less power when in the standby state.

9. The method of claim 8 wherein:

waking the electronic device activates the main processor from an inactive state.

10. The method of claim 1 further comprising:

receiving a voice command with the microphone of the electronic device;

recognizing a task in the voice command received with the microphone; and

responsive to recognizing the task, controlling which one electronic device out of the topology of interconnected electronic devices performs the recognized task, wherein the controlling invokes task arbitration rules selecting a single electronic device out of the topology of interconnected electronic devices according to user device preference or stored capabilities of the interconnected electronic devices.

11. The method of claim 10 wherein:

the task arbitration rules specify rules of succession for devices when a preferred device for the task is not available.

12. The method of claim 10 further comprising:

recognizing repeated, consistent usage of a particular device for a particular task;

setting the user preference to indicate the particular device as associated with the particular task before listening;

wherein the recognized task comprises the particular task, and controlling which one electronic device performs the task comprises choosing the particular device for the particular task based on the user preference.

13. The method of claim 10 further comprising:

determining via the stored capabilities of the interconnected electronic devices that the recognized task is not performable at the electronic device; and

selecting an other electronic device to perform the recognized task.

14. The method of claim 13 further comprising:

handing the recognized task off to an other electronic device.

15. The method of claim 13 further comprising:

sending a wake command to the other electronic device.

16. The method of claim 13 wherein:

the selected single electronic device did not hear the voice command.

17. An electronic device configured to recognize a wake phrase when in a low-power standby state in an environment comprising a topology of interconnected electronic devices, the electronic device comprising:

a plurality of hardware sensors comprising a microphone;

a speaker;

a stored preference indicating a primary device designation for the topology of interconnected electronic devices; and

a stored device listening state, wherein the stored device listening state transitions from the low-power standby state upon recognition of the wake phrase as received by the microphone, and wherein the electronic device is configured to control which one electronic device out of the topology of interconnected electronic devices transitions to an active listening state and plays an audio prompt that indicates the active listening state in response to the wake phrase according to the stored preference indicating the primary device designation for the interconnected electronic devices or recorded activity detected by one or more of the hardware sensors of the electronic device.

18. The electronic device of claim 17 wherein the electronic device further comprises:

an auxiliary voice recognition subsystem configured to recognize the wake phrase and transition the electronic device out of the low-power standby state; and

a main voice recognition subsystem configured to control which one electronic device out of the topology of interconnected electronic devices transitions to an active listening state.

19. The electronic device of claim 17 further comprising:

a voice recognizer configured to recognize a voice command received by the microphone of the electronic device, wherein the voice command comprises a task;

a task arbiter configured to control which one electronic device out of the topology of interconnected electronic devices performs the task according to user preference or capabilities of the interconnected electronic devices.

20. One or more machine-readable media comprising machine-executable instructions causing an electronic device to perform a method controlling which electronic device out of topology of interconnected electronic devices responds to a wake phrase, the method comprising:

receiving a wake phrase of a virtual persona from a microphone of the electronic device;

recognizing the wake phrase of the virtual persona in the electronic device; and

responsive to recognizing the wake phrase of the virtual persona, waking the electronic device and controlling which one electronic device out of the topology of interconnected electronic devices plays an audio prompt that indicates a listening state in response to the wake phrase, wherein the controlling invokes arbitration rules selecting a single device out of the topology of interconnected electronic devices when a user preference indicating a primary device designation for the interconnected electronic devices indicates a primary device that is not available, based on recorded activity detected by one or more hardware sensors of the electronic device indicating that the single device is currently being used;

receiving a command phrase from the microphone of the electronic device;

recognizing a task in the command phrase;

via a timeout, determining that a preferred device for the task is not available;

responsive to determining that a preferred device for the task is not available, determining a fallback device for the task; and

handing the task off to the fallback device.