Records disambiguation in a multimodal application operating on a multimodal device

• US 9,349,367 B2
• Filed: 04/24/2008
• Issued: 05/24/2016
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Agapi, Ciprian, Bodin, William K., Cross, Charles W. Jr., Mansey, Pradeep P.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US12/109,167
• Publication Number:
  US 20090271199A1
• Time in Patent Office: 2,952 Days
• Field of Search: 704/10, 704231-257, 704270-271, 704E15001-E1505, 348 1401- 1416, 348 61-161, 379 671- 8828, 379419-440
• US Class Current: 1/1
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Methods, apparatus, and products are disclosed for record disambiguation in a multimodal application operating on a multimodal device, the multimodal device supporting multiple modes of interaction including at least a voice mode and a visual mode, that include: prompting, by the multimodal application, a user to identify a particular record among a plurality of records; receiving, by the multimodal application in response to the prompt, a voice utterance from the user; determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records; generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance; and selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction.

Claims

1. A computer-implemented method of record disambiguation in a voice- and text-supported application operating on a voice- and text-supported device, the method comprising:

receiving, by the voice- and text-supported application, a voice input from a user;

determining, by the voice- and text-supported application, that the voice input matches record attribute values of at least one particular record attribute type for multiple records of a plurality of records and thereby ambiguously identifies more than one of the plurality of records, the multiple records of the plurality of records that match the voice input comprising a matching set of records;

generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set by identifying at least one additional record attribute type of the plurality of records in the matching set for which each record of the plurality of records in the matching set has a different record attribute value;

prompting the user, based on the identified at least one additional record attribute type, by;

outputting, via voice output, a name of the identified at least one additional record attribute type to the user in the user interaction, and

outputting, for each record of the plurality of records in the matching set, a record attribute value of the record for the identified at least one additional record attribute type to the user in the user interaction; and

selecting, by the voice- and text-supported application for further processing, one of the multiple records ambiguously identified by the voice input in dependence upon the user interaction.

2. The method of claim 1, wherein determining, by the voice- and text-supported application, that the voice input ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, one or more recognition results in dependence upon the voice input and a grammar; and

determining that the one or more recognition results match a first record attribute value of a first record of the plurality of records and a second record attribute value of a second record of the plurality of records, wherein the first record attribute value and the second record attribute value comprise at least one homophonic set of different words that have a same pronunciation.

3. The method of claim 1, wherein determining, by the voice- and text-supported application, that the voice input ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, a predefined recognition result in dependence upon the voice input and a grammar; and

executing a semantic interpretation script for the predefined recognition result specifying that the voice input ambiguously identifies more than one of the plurality of records.

4. The method of claim 1, wherein generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set further comprises executing an ECMAScript to create a VoiceXML form.

5. The method of claim 1, wherein generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set further comprises:

creating a disambiguation grammar in dependence upon record attribute values of the multiple records in the matching set;

prompting the user to disambiguate the multiple records in the matching set;

receiving, in response to prompting the user, a second voice input from the user; and

determining, using a speech recognition engine, a recognition result in dependence upon the second voice input and the disambiguation grammar,

wherein selecting, by the voice- and text-supported application for further processing, one of the multiple records ambiguously identified by the voice input in dependence upon the user interaction further comprises identifying one of the multiple records ambiguously identified by the voice input in dependence upon the recognition result.

6. The method of claim 5, wherein creating a disambiguation grammar in dependence upon the record attribute values of the multiple records in the matching set comprises:

identifying a record attribute type of the plurality of records in the matching set that specifies a unique value for each record of the plurality of records in the matching set; and

creating the disambiguation grammar in dependence upon the identified record attribute type.

7. A voice- and text-supported device comprising:

a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, perform acts of;

receiving, by a voice- and text-supported application operating on the voice- and text-supported device, a voice input from a user;

determining, by the voice- and text-supported application, that the voice input matches record attribute values of at least one particular record attribute type for multiple records of a plurality of records and thereby ambiguously identifies more than one of the plurality of records, the multiple records of the plurality of records that match the voice input comprising a matching set of records;

generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set by identifying at least one additional record attribute type of the plurality of records in the matching set for which each record of the plurality of records in the matching set has a different record attribute value;

prompting the user, based on the identified at least one additional record attribute type, by;

outputting, via voice output, a name of the identified at least one additional record attribute type to the user in the user interaction, and

outputting, for each record of the plurality of records in the matching set, a record attribute value of the record for the of the identified at least one additional record attribute type to the user in the user interaction; and

selecting, by the voice- and text-supported application for further processing, one of the multiple records ambiguously identified by the voice input in dependence upon the user interaction.

8. The voice- and text-supported device of claim 7, wherein determining, by the voice- and text-supported application, that the voice input ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, one or more recognition results in dependence upon the voice input and a grammar; and

determining that a record attribute value of more than one of the plurality of records is a homophone of the one or more recognition results.

9. The voice- and text-supported device of claim 7, wherein determining, by the voice- and text-supported application, that the voice input ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, a predefined recognition result in dependence upon the voice input and a grammar; and

executing a semantic interpretation script for the predefined recognition result specifying that the voice input ambiguously identifies more than one of the plurality of records.

10. The voice- and text-supported device of claim 7, wherein generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set further comprises executing an ECMAScript to create a VoiceXML form.

11. The voice- and text-supported device of claim 7, wherein generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set further comprises:

creating a disambiguation grammar in dependence upon record attribute values of the multiple records in the matching set;

prompting the user to disambiguate the multiple records in the matching set;

receiving, in response to prompting the user, a second voice input from the user; and

determining, using a speech recognition engine, a recognition result in dependence upon the second voice input and the disambiguation grammar,

wherein selecting, by the voice- and text-supported application for further processing, one of the multiple records ambiguously identified by the voice input in dependence upon the user interaction further comprises identifying one of the multiple records ambiguously identified by the voice input in dependence upon the recognition result.

12. The voice- and text-supported device of claim 11, wherein creating a disambiguation grammar in dependence upon the record attribute values of the multiple records in the matching set comprises:

identifying a record attribute type of the plurality of records in the matching set that specifies a unique value for each record of the plurality of records in the matching set; and

creating the disambiguation grammar in dependence upon the identified record attribute type.

13. The voice- and text-supported device of claim 7, wherein identifying the at least one additional record attribute type of the plurality of records in the matching set comprises traversing each record attribute type of a plurality of record attribute types of the plurality of records in the matching set in an order determined by at least one priority value associated with each record attribute type of the plurality of record attribute types.

14. The voice- and text-supported device of claim 7, wherein outputting the identified at least one additional record attribute type to the user in the user interaction comprises outputting names of a plurality of identified additional record attribute types to the user in the user interaction.

15. At least one non-transitory recordable medium having disposed thereon a computer program product for use with a voice- and text-supported application operating on a voice- and text-supported device, the computer program product disposed upon a computer readable medium, the computer program product comprising computer program instructions that, when executed, perform acts of:

receiving, by the voice- and text-supported application, a voice input from a user;

determining, by the voice- and text-supported application, that the voice input matches record attribute values of at least one particular record attribute type for multiple records of a plurality of records and thereby ambiguously identifies more than one of the plurality of records, the multiple records of the plurality of records that match the voice input comprising a matching set of records;

generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set by identifying at least one additional attribute type of the plurality of records in the matching set for which each record of the plurality of records in the matching set has a different record attribute value;

prompting the user, based on the identified at least one additional record attribute type, by;

outputting, via voice output, a name of the identified at least one additional record attribute type to the user in the user interaction, and

outputting, for each record of the plurality of records in the matching set, a record attribute value of the record for the identified at least one additional record attribute type to the user in the user interaction; and

selecting, by the voice- and text-supported application for further processing, one of the multiple records ambiguously identified by the voice input in dependence upon the user interaction.

16. The at least one non-transitory recordable medium of claim 15, wherein determining, by the voice- and text-supported application, that the voice input ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, one or more recognition results in dependence upon the voice input and a grammar; and

determining that a record attribute value of more than one of the plurality of records is a homophone of the one or more recognition results.

17. The at least one non-transitory recordable medium of claim 15, wherein determining, by the voice- and text-supported application, that the voice input ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, a predefined recognition result in dependence upon the voice input and a grammar; and

executing a semantic interpretation script for the predefined recognition result specifying that the voice input ambiguously identifies more than one of the plurality of records.

18. The at least one non-transitory recordable medium of claim 15, wherein generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set further comprises executing an ECMAScript to create a VoiceXML form.

19. The at least one non-transitory recordable medium of claim 15, wherein generating, by the voice- and text-supported application, a user interaction to disambiguate the multiple records in the matching set further comprises:

creating a disambiguation grammar in dependence upon record attribute values of the multiple records in the matching set;

prompting the user to disambiguate the multiple records in the matching set;

receiving, in response to prompting the user, a second voice input from the user; and

determining, using a speech recognition engine, a recognition result in dependence upon the second voice input and the disambiguation grammar,

wherein selecting, by the voice- and text-supported application for further processing, one of the multiple records ambiguously identified by the voice input in dependence upon the user interaction further comprises identifying one of the multiple records ambiguously identified by the voice input in dependence upon the recognition result.

20. The at least one non-transitory recordable medium of claim 19, wherein creating a disambiguation grammar in dependence upon the record attribute values of the multiple records in the matching set comprises:

identifying a record attribute type of the plurality of records in the matching set that specifies a unique value for each record of the plurality of records in the matching set; and

creating the disambiguation grammar in dependence upon the identified record attribute type.