Routing natural language commands to the appropriate applications

• US 9,734,839 B1
• Filed: 06/20/2012
• Issued: 08/15/2017
• Est. Priority Date: 06/20/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Adams, Jeffrey P.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Shin, Seong Ah A
• Application Number:
  US13/528,529
• Time in Patent Office: 1,882 Days
• Field of Search: 704 1- 9, 704275, 704231
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

A device is configured with multiple applications that each respond to various commands. The correct application to receive a natural language command is identified by consideration of how well the command matches functions of the application. A target application to receive the command may additionally be selected by consideration of which application is most likely to receive a command. The likelihood of an application to receive a command may be determined by considering context. The command may be a voice input that is analyzed by speech recognition technology to determine word strings representing possible commands. Thus, the selection of a target application to receive the command may be based on any or all of the word strings from the natural language input, a closeness of fit between the command and an application, and the likelihood an application is the target for the next incoming command.

Claims

1. A voice controlled system comprising:

one or more processors;

computer-readable media accessible by the one or more processors;

a first application and a second application stored on the computer-readable media to be executed by the one or more processors;

a microphone to receive audio input;

a speech recognition module to identify first data from a signal representing the audio input, the first data including text representing one or more words; and

a command router to

determine, using second data that is different from the first data, a first application probability of the first application being a recipient of a next command, wherein the second data is available to the command router prior to identification of the first data,

determine, using the second data, a second application probability of the second application being a recipient of the next command,

provide, to the first application, the text,

receive, from the first application, a first matching probability indicating a degree of matching between the one or more words and a command which the first application can interpret,

provide, to the second application, the text,

receive, from the second application, a second matching probability indicating a degree of matching between the one or more words and a command which the second application can interpret, and

select, based at least in part on the first application probability, the second application probability, the first matching probability, and the second matching probability, the first application to receive the command in the one or more words and to perform at least one operation associated with the next command.

2. The voice controlled system of claim 1, wherein the one or more words are represented by the speech recognition module as a lattice of multiple word strings and the first matching probability is based at least in part on a probability that a command which the first application can interpret is a word string in the lattice.

3. The voice controlled system of claim 1, wherein the first application probability is based on at least one of a command history, a user profile, or an environmental context of the voice controlled device.

4. The voice controlled system of claim 1, wherein the first application probability is a Bayesian prior.

5. The voice controlled system of claim 1, wherein the first matching probability is a Bayesian probability.

6. The voice controlled system of claim 1, wherein the first matching probability is determined at least in part as a fuzzy match between the one or more words and a plurality of entries in a commands database associated with the first application.

7. The voice controlled system of claim 1, wherein the first matching probability is determined at least in part by a statistical parser.

8. The voice controlled system of claim 1, wherein the first matching probability is determined at least in part by grammar rules.

9. The voice controlled system of claim 1, wherein the first matching probability is determined at least in part by applying an n-gram model.

10. The voice controlled system of claim 1, wherein the first matching probability is determined based at least in part on a context specific to the first application, the context including at least one of a state of the first application, a history of commands implemented by the first application, or a user profile associated with the first application.

11. The voice controlled system of claim 1, wherein the first matching probability is determined at least in part on a command from a previous audio input.

12. The voice controlled system of claim 1, wherein the selection of either the first application or the second application to receive the command in the one or more words is based at least in part on a product of the first application probability and the first matching probability.

13. The voice controlled system of claim 1, wherein the command router determines that the one or more words are likely to be associated with the next command based at least in part on a degree of matching between individual words of the one or more words and a lattice of commands associated with the first application and the second application.

14. A computer-implemented method comprising:

receiving, at a command router, a natural language input captured at a microphone;

identifying first data from the natural language input, the first data including text representing one or more words;

determining that the one or more words may be associated with a command;

determining, using second data that is different from the first data, a first application score, the first application score indicating a likelihood that the command is associated with a first application;

determining, using the second data that is different from the first data, a second application score, the second application score indicating a likelihood that the command is associated with a second application, wherein the second data is available to access prior to identification of the first data;

providing the text to the first application;

receiving, from the first application, a first matching probability score indicating a degree of matching between the one or more words of the natural language input and the command associated with the first application;

providing the text to the second application;

receiving, from the second application, a second matching probability score indicating a degree of matching between the one or more words of the natural language input and the command associated with the second application; and

causing the first application to process the command based at least partly on the first application score, the second application score, the first matching probability score and the second matching probability score.

15. The computer-implemented method of claim 14, wherein the natural language input comprises a lattice of multiple word strings.

16. The computer-implemented method of claim 15, wherein the first matching probability score is based at least in part on a probability that a word string of multiple possible word strings in the lattice corresponds to the command.

17. The computer-implemented method of claim 14, wherein the first matching probability score comprises a fuzzy match score that represents a degree of matching between the natural language input and a command recognized by the first application.

18. The computer-implemented method of claim 14, wherein the first matching probability score is determined at least in part by a statistical parser.

19. The computer-implemented method of claim 14, wherein the first matching probability score is based at least in part on a command in a previous natural language input.

20. The computer-implemented method of claim 14, wherein causing the first application to process the command comprises causing the first application to process the command in response to determining that a product of the first application score and the first matching probability score exceed a threshold value.

21. The computer-implemented method of claim 14, wherein the first application score is a first probability and the first matching probability score is a second probability.

22. The computer-implemented method of claim 14, wherein the determining the one or more words are likely to be associated with the next command is based at least in part on a degree of matching between individual words of the one or more words and a lattice of commands associated with the first application and the second application.

23. One or more non-transitory computer-readable storage media storing instructions that, when executed on one or more processors, causes the one or more processors to perform acts comprising:

identifying first data from natural language input, the first data including text representing one or more words;

determining, using second data that is different from the first data, a first application score indicating a likelihood that a first application will be a recipient of a subsequent command, wherein the second data is available to access before identifying the first data;

determining, using the second data that is different from the first data, a second application score indicating a likelihood that a second application will be a recipient of the subsequent command;

receiving, from the first application, a first matching probability indicating a first degree of matching between the subsequent command and a first command which the first application is configure to interpret;

receiving, from the second application, a second matching probability indicating a second degree of matching between the subsequent command and a second command which the second application is configure to interpret; and

responsive to receiving the subsequent command, causing one or more applications to process the command based at least in part on the first application score, the second application score, the first matching probability and the second matching probability.

24. The one or more non-transitory computer-readable storage media of claim 23, wherein determining the first application score and the second application score occurs prior to receiving the subsequent command.

25. The one or more non-transitory computer-readable storage media of claim 23, wherein the first application score comprises a Bayesian prior probability.

26. The one or more non-transitory computer-readable storage media of claim 23, wherein the first application score is based at least in part on at least one of a command history, a user profile, or one or more environmental factors.

27. The one or more non-transitory computer-readable storage media of claim 23, wherein the one or more applications include the first application.

28. The one or more non-transitory computer-readable storage media of claim 23, wherein the first matching probability is based at least in part on a lattice of multiple word strings.

29. The one or more non-transitory computer-readable storage media of claim 23, wherein the first matching probability is based at least in part on a word string of multiple word strings.

30. The one or more non-transitory computer-readable storage media of claim 23, wherein identifying the one or more applications is based at least in part on a product of the first application score and the first matching probability and a product of the second application score and the second matching probability.

31. The one or more non-transitory computer-readable storage media of claim 23, wherein identifying the one or more applications comprises selecting an application of the one or more applications having a largest product of an application score and a matching probability.

32. The one or more non-transitory computer-readable storage media of claim 23, wherein identifying the one or more applications comprises identifying two or more applications.

33. The one or more non-transitory computer-readable storage media of claim 23, wherein the one or more applications include the second application.