MANAGEMENT LAYER FOR MULTIPLE INTELLIGENT PERSONAL ASSISTANT SERVICES

• US 20180301147A1
• Filed: 04/13/2017
• Published: 10/18/2018
• Est. Priority Date: 04/13/2017
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Kim, Seon Man
• Granted Patent: US 10,748,531 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Performing speech recognition in a multi-device system includes receiving a first audio signal that is generated by a first microphone in response to a verbal utterance, and a second audio signal that is generated by a second microphone in response to the verbal utterance; dividing the first audio signal into a first sequence of temporal segments; dividing the second audio signal into a second sequence of temporal segments; comparing a sound energy level associated with a first temporal segment of the first sequence to a sound energy level associated with a first temporal segment of the second sequence; based on the comparing, selecting, as a first temporal segment of a speech recognition audio signal, one of the first temporal segment of the first sequence and the first temporal segment of the second sequence; and performing speech recognition on the speech recognition audio signal.

Claims

1. A non-transitory computer-readable storage medium including instructions that, when executed by one or more processors, configure the one or more processors to perform speech recognition performing the steps of:

detecting an activation phrase within a first audio signal;

based on the activation phrase, selecting an intelligent personal assistant (IPA) that is included in a group of candidate IPAs;

extracting query content from the first audio signal;

generating a second audio signal based on the query content; and

transmitting the second audio signal to the IPA.

2. The non-transitory computer-readable medium of claim 1, further comprising instructions that, when executed by one or more processors, configure the one or more processors to perform the steps of:

detecting an additional activation phrase in the first audio signal; and

based on the additional activation phrase, selecting an additional IPA that is included in the group of candidate IPAs.

3. The non-transitory computer-readable medium of claim 2, further comprising instructions that, when executed by one or more processors, configure the one or more processors to perform the step of transmitting the second audio signal to the additional IPA.

4. The non-transitory computer-readable medium of claim 2, further comprising instructions that, when executed by one or more processors, configure the one or more processors to perform the steps of:

extracting additional query content from the first audio signal that is associated with the additional activation phrase;

generating a third voice input based on the additional query content; and

transmitting the third voice input to the additional IPA.

5. The non-transitory computer-readable medium of claim 1, further comprising instructions that, when executed by one or more processors, configure the one or more processors to perform the steps of:

receiving a first response audio signal from the IPA;

causing the first response audio signal to be played;

while the first response audio signal is being played, receiving a second response audio signal from an additional IPA that is included in the group of candidate IPAs; and

beginning to play the second response audio signal after the first response audio signal has completed playing.

6. The non-transitory computer-readable medium of claim 5, wherein the first response audio signal is received in response to transmitting the second audio signal.

7. The non-transitory computer-readable medium of claim 1, further comprising instructions that, when executed by one or more processors, configure the one or more processors to perform the steps of:

receiving a first response audio signal from the IPA;

storing the first response audio signal;

after storing the first response audio signal, receiving a second response audio signal from an additional IPA that is included in the group of candidate IPA; and

causing the second response audio signal to begin to play before causing the first response audio signal to begin to play.

8. The non-transitory computer-readable medium of claim 1, further comprising instructions that, when executed by one or more processors, configure the one or more processors to perform the steps of:

detecting a change listener command in the first audio signal;

determining an additional IPA based on the change listener command; and

transmitting the second audio signal to the additional IPA.

9. The non-transitory computer-readable medium of claim 8, wherein detecting a change listener command comprises detecting a word or phrase that indicates a user who uttered the activation phrase is directing voice commands to a second IPA included in the group of candidate IPAs.

10. A system comprising:

a memory storing an intelligent personal assistant (IPA) management application; and

one or more processors that are coupled to the memory and, when executing the IPA management application, are configured to;

detect an activation phrase within a first audio signal;

based on the activation phrase, select an intelligent personal assistant (IPA) that is included in a group of candidate IPAs;

extract query content from the first audio signal;

generate a second audio signal based on the query content; and

transmit the second audio signal to the IPA.

11. The system of claim 10, wherein the one or more processors are further configured to perform the steps of:

detecting a change listener command in the first audio signal;

determining an additional IPA based on the change listener command;

extracting additional query content from the first audio signal;

generating a third audio signal based on the additional query content; and

transmitting the third audio signal to the second IPA.

12. The system of claim 11, wherein the additional query content includes a voice command.

13. The system of claim 11, wherein determining the additional IPA based on the change listener command comprises determining the additional IPA based on at least one of a previously issued voice command, a previously issued voice query, and which process is currently being controlled by an IPA.

14. The system of claim 11, wherein determining the additional IPA based on the previously issued voice command comprises determining that the additional IPA is an IPA that received the previously issued voice command via a second audio signal.

15. The system of claim 11, wherein determining the additional IPA based on which process is currently being controlled by the IPA service comprises determining that the additional IPA is controlling the process.

16. The system of claim 10, wherein the one or more processors are further configured to perform the steps of:

detecting an additional activation phrase in the first audio signal; and

based on the additional activation phrase, selecting an additional IPA that is included in the group of candidate IPAs.

17. The system of claim 16, wherein the one or more processors are further configured to perform the step of transmitting the second audio signal to the additional IPA.

18. The system of claim 16, wherein the one or more processors are further configured to perform the steps of:

extracting additional query content from the first audio signal that is associated with the additional activation phrase;

generating a third voice input based on the additional query content; and

transmitting the third voice input to the additional IPA

19. A method for performing speech recognition, the method comprising:

detecting an activation phrase within a first audio signal;

based on the activation phrase, selecting an intelligent personal assistant (IPA) that is included in a group of candidate IPAs;

extracting query content from the first audio signal;

generating a second audio signal based on the query content; and

transmitting the second audio signal to the IPA.

20. The method of claim 19, further comprising:

detecting an additional activation phrase in the first audio signal; and

based on the additional activation phrase, selecting an additional IPA that is included in the group of candidate IPAs.