Systems and methods for spoken dialog service arbitration

• US 9,396,727 B2
• Filed: 12/02/2013
• Issued: 07/19/2016
• Est. Priority Date: 07/10/2013
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Tzirkel-Hancock, Eli, Lindemann, Greg T., Sims, Robert D., Tsimhoni, Omer
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US14/094,110
• Publication Number:
  US 20150019219A1
• Time in Patent Office: 960 Days
• Field of Search: 704/244, 704/235, 704/231, 704/240, 340/438, 340/539.11
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Systems and methods for arbitrating spoken dialog services include determining a capability catalog associated with a plurality of devices accessible within an environment. The capability catalog includes a list of the plurality of devices mapped to a list of spoken dialog services provided by each of the plurality of devices. The system arbitrates between the plurality of devices and the spoken dialog services in the capability catalog to determine a selected device and a selected dialog service.

Claims

1. A method for arbitrating spoken dialog services, the method comprising:

determining, with an arbitration module that includes a processor, a capability catalog associated with a plurality of devices accessible within an environment, the capability catalog including a list of the plurality of devices mapped to a list of spoken dialog services provided by each of the plurality of devices;

receiving, with a speech understanding module that includes a processor, a spoken utterance from a user within the environment;

arbitrating, with an arbitration module that includes a processor, between the plurality of devices and the spoken dialog services in the capability catalog to determine a selected device and a selected dialog service; and

processing the spoken utterance with the selected spoken dialog service on the selected device.

2. The method of claim 1, wherein the arbitrating includes:

classifying the spoken utterance to determine a set of candidate devices from the plurality of devices and a set of spoken dialog services based on the capability catalog; and

determining the selected device from the set of candidate devices and the selected spoken dialog service from the list of candidate spoken dialog services based on a verification criterion.

3. The method of claim 2, wherein the verification criterion comprises a confidence level associated with the classification of the spoken utterance.

4. The method of claim 2, further comprising performing the classifying of the spoken dialog service in conjunction with at least one of training data associated with past classification performance and user behavior.

5. The method of claim 1, further including determining the selected device and the selected spoken dialog service based on supplemental information requested from the user.

6. The method of claim 1, wherein the environment is a vehicle environment, and the plurality of devices includes at least one of:

a mobile device removeably located within the vehicle environment, a device incorporated into the vehicle environment, and a device comprising a network server remote from the vehicle environment.

7. The method of claim 2, further including:

forwarding the spoken utterance to a first spoken dialog service and a second spoken dialog service within the capability catalog;

receiving, from the first spoken dialog service, a first confidence score associated with recognition of the spoken utterance;

receiving, from the second spoken dialog service, a second confidence score associated with recognition of the spoken utterance; and

determining the verification criterion based at least in part on the first and second confidence scores.

8. The method of claim 1, further including:

specifying default settings associated with the plurality of devices and the spoken dialog services; and

modifying the default settings based on user behavior.

9. A system for spoken dialog arbitration, the system comprising:

a speech understanding module, including a processor, configured to process a speech utterance received from a user within an environment;

an arbitration module, including a processor, communicatively coupled to the speech understanding module, the arbitration module configured to determine a capability catalog associated with a plurality of devices accessible within the environment, the capability catalog including a list of the plurality of devices mapped to a list of spoken dialog services provided by each of the plurality of devices;

receive the spoken utterance;

arbitrate between the plurality of devices and the spoken dialog services in the capability catalog to determine a selected device and a selected dialog service; and

processing the spoken utterance with the selected spoken dialog service on the selected device.

10. The system of claim 9, wherein the arbitration module is configured to:

classify the spoken utterance to determine a set of candidate devices from the plurality of devices and a set of spoken dialog services based on the capability catalog; and

determine the selected device from the set of candidate devices and the selected spoken dialog service from the list of candidate spoken dialog services based on a verification criterion.

11. The system of claim 10, wherein the verification criterion comprises a confidence level associated with the classification of the spoken utterance.

12. The system of claim 10, wherein the arbitration module performs the classifying of the spoken dialog service in conjunction with at least one of training data associated with past classification performance and user behavior.

13. The system of claim 10, wherein the arbitration module is configured to:

forward the spoken utterance to a first spoken dialog service and a second spoken dialog service within the capability catalog;

receive, from the first spoken dialog service, a first confidence score associated with recognition of the spoken utterance;

receive, from the second spoken dialog service, a second confidence score associated with recognition of the spoken utterance; and

determine the verification criterion based at least in part on the first and second confidence scores.

14. The system of claim 9, wherein the arbitration module is configured to:

specify default settings associated with the plurality of devices and the spoken dialog services; and

modify the default settings based on user behavior.

15. An arbitration method comprising:

determining, with an arbitration module that includes a processor, a capability catalog associated with a plurality of devices accessible within a vehicle, the capability catalog including a list of the plurality of devices mapped to a list of spoken dialog services provided by each of the plurality of devices;

receiving, with a speech understanding module that includes a processor, the spoken utterance;

arbitrating, with an arbitration module that includes a processor, between the plurality of devices and the spoken dialog services in the capability catalog to determine a selected device and a selected dialog service, wherein the arbitrating includes classifying the spoken utterance to determine a set of candidate devices from the plurality of devices and a set of spoken dialog services based on the capability catalog, and determining the selected device from the set of candidate devices and the selected spoken dialog service from the list of candidate spoken dialog services based on a verification criterion; and

processing the spoken utterance with the selected spoken dialog service on the selected device.

16. The method of claim 15, wherein the steps of receiving the first confidence score from the first spoken dialog service and receiving the second confidence score from the second spoken dialog service is followed by one or more steps of prompting the user for additional input and providing the additional user utterances to both services concurrently, and selecting one of the services when the confidence returned by that service is sufficiently higher than that of the other service.

17. The method of claim 15, wherein the verification criterion comprises a confidence level associated with the classification of the spoken utterance.

18. The method of claim 15, further comprising performing the classifying of the spoken dialog service in conjunction with at least one of training data associated with past classification performance and user behavior.

19. The method of claim 15, further including:

specifying default settings associated with the plurality of devices and the spoken dialog services; and

modifying the default settings based on user behavior.

20. The method of claim 15, further including:

forwarding the spoken utterance to a first spoken dialog service and a second spoken dialog service within the capability catalog;

receiving, from the first spoken dialog service, a first confidence score associated with recognition of the spoken utterance;

receiving, from the second spoken dialog service, a second confidence score associated with recognition of the spoken utterance; and

determining the verification criterion based at least in part on the first and second confidence scores.