CONTEXT-BASED ENDPOINT DETECTION

• US 20160358598A1
• Filed: 09/04/2015
• Published: 12/08/2016
• Est. Priority Date: 06/07/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: WILLIAMS, Shaun E., MASON, Henry G., KRISHNAMOORTHY, Mahesh, AGRAWAL, Neha, UGUROGLU, Selen, PAULIK, Matthias, KAJAREKAR, Sachin S., MOHAMED, Ali S.
• Granted Patent: US 10,186,254 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

The present disclosure generally relates to context-based endpoint detection in user speech input. A method for identifying an endpoint of a spoken request by a user may include receiving user input of natural language speech including one or more words; identifying at least one context associated with the user input; generating a probability, based on the at least one context associated with the user input, that a location in the user input is an endpoint; determining whether the probability is greater than a threshold; and in accordance with a determination that the probability is greater than the threshold, identifying the location in the user input as the endpoint.

Claims

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device, cause the electronic device to:

receive user input comprising natural language speech including one or more words;

identify at least one context associated with the user input;

generate a probability, based on the at least one context associated with the user input, that a location in the user input is an endpoint;

determine whether the probability is greater than a threshold; and

in accordance with a determination that the probability is greater than the threshold, identify the location in the user input as the endpoint.

2. The non-transitory computer-readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the one or more processors of the electronic device, cause the device to:

record the user input until the endpoint is identified.

3. The non-transitory computer-readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the one or more processors of the electronic device, cause the device to:

divide the user input into a plurality of audio frames; and

generate a probability that each frame of the plurality of frames includes the endpoint.

4. The non-transitory computer-readable storage medium of claim 3, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the one or more processors of the electronic device, cause the device to:

record the user input until the endpoint is identified;

wherein in accordance with a determination that the probability of an endpoint for a frame is greater than the threshold;

stop recording the user input; and

process the user input; and

wherein in accordance with a determination that the probability of an endpoint for a frame is not greater than the threshold;

continue recording the user input.

5. The non-transitory computer-readable storage medium of claim 3, wherein the non-transitory computer-readable storage medium comprises instructions, which when executed by the one or more processors of the electronic device, cause the device to:

further in accordance with a determination that the probability of an endpoint for a frame is greater than the threshold, output data associated with end of speech; and

further in accordance with a determination that the probability of an endpoint for a frame is not greater than the threshold, output data associated with continuing speech.

6. The non-transitory computer-readable storage medium of claim 5, wherein the data associated with end of speech is an end of speech tag.

7. The non-transitory computer-readable storage medium of claim 5, wherein in accordance with a determination that the probability of an audio frame of the plurality of audio frames is not greater than the threshold, the data associated with continuing speech is a continuing speech tag.

8. The non-transitory computer-readable storage medium of claim 5, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the one or more processors of the electronic device, cause the device to:

determine whether each frame includes audio information associated with user input;

in accordance with a determination that a frame includes audio information associated with user input, determine whether a threshold number of frames of silence follow;

in accordance with a determination that the threshold number of frames of silence follow, cease recording user input; and

in accordance with a determination that the threshold number of frames of silence do not follow, continue recording user input.

9. The non-transitory computer-readable storage medium of claim 8, wherein the threshold number of frames of silence is associated with the duration of the determination whether the probability is greater than the threshold.

10. The non-transitory computer-readable storage medium of claim 1, wherein the at least one context includes the content of the one or more words.

11. The non-transitory computer-readable storage medium of claim 1, wherein the at least one context includes the device context.

12. The non-transitory computer-readable storage medium of claim 11, wherein the device context includes the location of the device.

13. The non-transitory computer-readable storage medium of claim 1, wherein the at least one context includes a rate of speech.

14. The non-transitory computer-readable storage medium of claim 1, wherein the at least one context includes information associated with a user.

15. The non-transitory computer-readable storage medium of claim 14, wherein the information associated with a user includes the identity of the user.

16. The non-transitory computer-readable storage medium of claim 14, wherein the information associated with a user includes information about the speech patterns of the user.

17. The non-transitory computer-readable storage medium of claim 1, further comprising a digital assistant operable through the device, wherein the at least one context includes the state of the digital assistant.

18. The non-transitory computer-readable storage medium of claim 17, wherein the at least one context includes user input relative to the state of the digital assistant.

19. The non-transitory computer-readable storage medium of claim 1, further comprising an application operable through the device, wherein the at least one context includes application context.

20. A method for identifying an endpoint of a spoken request by a user, comprising:

at a device with one or more processors and memory;

receiving user input comprising natural language speech including one or more words;

identifying at least one context associated with the user input;

generating a probability, based on the at least one context associated with the user input, that a location in the user input is an endpoint;

determining whether the probability is greater than a threshold; and

in accordance with a determination that the probability is greater than the threshold, identifying the location in the user input as the endpoint.

21. An electronic device, comprising:

a display;

a memory;

a processor coupled to the display and the memory;

and programs stored in the memory to be executed by the processor, the programs comprising instructions for;

receiving user input comprising natural language speech including one or more words;

identifying at least one context associated with the user input;

generating a probability, based on the at least one context associated with the user input, that a location in the user input is an endpoint;

determining whether the probability is greater than a threshold; and

in accordance with a determination that the probability is greater than the threshold, identifying the location in the user input as the endpoint.