Records Disambiguation In A Multimodal Application Operating On A Multimodal Device

• US 20090271199A1
• Filed: 04/24/2008
• Published: 10/29/2009
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines SA (International Business Machines Corporation)
• Inventors: Cross, Charles W. JR., Bodin, William K., Agapi, Ciprian, Mansey, Pradeep P.
• Granted Patent: US 9,349,367 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Methods, apparatus, and products are disclosed for record disambiguation in a multimodal application operating on a multimodal device, the multimodal device supporting multiple modes of interaction including at least a voice mode and a visual mode, that include: prompting, by the multimodal application, a user to identify a particular record among a plurality of records; receiving, by the multimodal application in response to the prompt, a voice utterance from the user; determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records; generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance; and selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction.

Claims

1. A computer-implemented method of record disambiguation in a multimodal application operating on a multimodal device, the multimodal device supporting multiple modes of interaction including at least a voice mode and a visual mode, the method comprising:

prompting, by the multimodal application, a user to identify a particular record among a plurality of records;

receiving, by the multimodal application in response to the prompt, a voice utterance from the user;

determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records;

generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance; and

selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction.

2. The method of claim 1 wherein determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, one or more recognition results in dependence upon the voice utterance and a grammar; and

determining that a record attribute of more than one of the plurality of records is a homophone of the recognition results.

3. The method of claim 1 wherein determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, a predefined recognition result in dependence upon the voice utterance and a grammar; and

executing a semantic interpretation script for the predefined recognition result specifying that the voice utterance ambiguously identifies more than one of the plurality of records.

4. The method of claim 1 wherein generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance further comprises executing an ECMAScript to create a VoiceXML form.

5. The method of claim 1 wherein:

generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance further comprises;

creating a disambiguation grammar in dependence upon the record attributes of the records ambiguously identified by the voice utterance, prompting the user to disambiguate the records ambiguously identified by the voice utterance, receiving, in response to the disambiguation prompt, a second voice utterance from the user, and determining, using a speech recognition engine, a recognition result in dependence upon the second voice utterance and the disambiguation grammar; and

selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction further comprises identifying one of the records ambiguously identified by the voice utterance in dependence upon the recognition result.

6. The method of claim 4 wherein creating a disambiguation grammar in dependence upon the record attributes of the records ambiguously identified by the voice utterance further comprises:

identifying an attribute type for the records ambiguously identified by the voice utterance that specifies a unique value for each of the records ambiguously identified by the voice utterance; and

creating the disambiguation grammar in dependence upon the record attributes of the identified attribute type for the records ambiguously identified by the voice utterance.

7. A multimodal device for record disambiguation in a multimodal application operating on the multimodal device, the multimodal device supporting multiple modes of interaction including at least a voice mode and a visual mode, the multimodal device comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions capable of:

prompting, by the multimodal application, a user to identify a particular record among a plurality of records;

receiving, by the multimodal application in response to the prompt, a voice utterance from the user;

determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records;

generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance; and

selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction.

8. The multimodal device of claim 7 wherein determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, one or more recognition results in dependence upon the voice utterance and a grammar; and

determining that a record attribute of more than one of the plurality of records is a homophone of the recognition results.

9. The multimodal device of claim 7 wherein determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, a predefined recognition result in dependence upon the voice utterance and a grammar; and

executing a semantic interpretation script for the predefined recognition result specifying that the voice utterance ambiguously identifies more than one of the plurality of records.

10. The multimodal device of claim 7 wherein generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance further comprises executing an ECMAScript to create a VoiceXML form.

11. The multimodal device of claim 7 wherein:

generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance further comprises;

creating a disambiguation grammar in dependence upon the record attributes of the records ambiguously identified by the voice utterance, prompting the user to disambiguate the records ambiguously identified by the voice utterance, receiving, in response to the disambiguation prompt, a second voice utterance from the user, and determining, using a speech recognition engine, a recognition result in dependence upon the second voice utterance and the disambiguation grammar; and

selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction further comprises identifying one of the records ambiguously identified by the voice utterance in dependence upon the recognition result.

12. The multimodal device of claim 11 wherein creating a disambiguation grammar in dependence upon the record attributes of the records ambiguously identified by the voice utterance further comprises:

identifying an attribute type for the records ambiguously identified by the voice utterance that specifies a unique value for each of the records ambiguously identified by the voice utterance; and

creating the disambiguation grammar in dependence upon the record attributes of the identified attribute type for the records ambiguously identified by the voice utterance.

13. A computer program product for record disambiguation in a multimodal application operating on a multimodal device, the multimodal device supporting multiple modes of interaction including at least a voice mode and a visual mode, the computer program product disposed upon a computer readable medium, the computer program product comprising computer program instructions capable of:

prompting, by the multimodal application, a user to identify a particular record among a plurality of records;

receiving, by the multimodal application in response to the prompt, a voice utterance from the user;

determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records;

generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance; and

selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction.

14. The computer program product of claim 13 wherein determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, one or more recognition results in dependence upon the voice utterance and a grammar; and

determining that a record attribute of more than one of the plurality of records is a homophone of the recognition results.

15. The computer program product of claim 13 wherein determining, by the multimodal application, that the voice utterance ambiguously identifies more than one of the plurality of records further comprises:

determining, using a speech recognition engine, a predefined recognition result in dependence upon the voice utterance and a grammar; and

executing a semantic interpretation script for the predefined recognition result specifying that the voice utterance ambiguously identifies more than one of the plurality of records.

16. The computer program product of claim 13 wherein generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance further comprises executing an ECMAScript to create a VoiceXML form.

17. The computer program product of claim 13 wherein:

generating, by the multimodal application, a user interaction to disambiguate the records ambiguously identified by the voice utterance in dependence upon record attributes of the records ambiguously identified by the voice utterance further comprises;

creating a disambiguation grammar in dependence upon the record attributes of the records ambiguously identified by the voice utterance, prompting the user to disambiguate the records ambiguously identified by the voice utterance, receiving, in response to the disambiguation prompt, a second voice utterance from the user, and determining, using a speech recognition engine, a recognition result in dependence upon the second voice utterance and the disambiguation grammar; and

selecting, by the multimodal application for further processing, one of the records ambiguously identified by the voice utterance in dependence upon the user interaction further comprises identifying one of the records ambiguously identified by the voice utterance in dependence upon the recognition result.

18. The computer program product of claim 17 wherein creating a disambiguation grammar in dependence upon the record attributes of the records ambiguously identified by the voice utterance further comprises:

identifying an attribute type for the records ambiguously identified by the voice utterance that specifies a unique value for each of the records ambiguously identified by the voice utterance; and

creating the disambiguation grammar in dependence upon the record attributes of the identified attribute type for the records ambiguously identified by the voice utterance.

19. The computer program product of claim 13 wherein the computer readable medium comprises a recordable medium.

20. The computer program product of claim 13 wherein the computer readable medium comprises a transmission medium.