Method of enhancing voice interactions using visual messages

• US 20040236574A1
• Filed: 05/20/2003
• Published: 11/25/2004
• Est. Priority Date: 05/20/2003
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Ativanichayaphong, Soonthorn, Jaramillo, David, McCobb, Gerald, Wilson, Leslie R.
• Granted Patent: US 7,966,188 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method for enhancing voice interactions within a portable multimodal computing device using visual messages. A multimodal interface can be provided that includes an audio interface and a visual interface. A speech input can then be received and a voice recognition task can be performed upon at least a portion of the speech input. At least one message within the multimodal interface can be visually presented, wherein the message is a prompt for the speech input and/or a confirmation of the speech input.

Claims

1. A method for enhancing voice interactions within a multimodal computing device using visual messages comprising the steps of:

providing a multimodal interface that includes an audio interface and a visual interface;

visually presenting at least one message within said multimodal interface, wherein said message is at least one of a prompt for a speech input and a confirmation of said speech input;

receiving said speech input; and

,

performing a voice recognition task upon at least a portion of said speech input.

3. The method of claim 1, further comprising the step of:

audibly presenting said at least one message within said multimodal interface.

4. The method of claim 1, further comprising the steps of:

allowing a user of said multimodal interface to establish at least one interface preference; and

,

determining whether to visually present at least one of said messages based upon said interface preference.

5. The method of claim 4, wherein said interface preference is associated with establishing visual prompts for speech input, said method further comprising the steps of:

if visual prompting is affirmatively indicated by said interface preference, visually presenting within said multimodal interface said prompt for said speech input; and

,

if visual prompting is negatively indicated by said interface preference, audibly prompting for said speech input.

6. The method of claim 4, wherein said interface preference is associated with establishing visual confirmation of speech input, said method further comprising the steps of:

if visual confirmation is affirmatively indicated by said interface preference, providing visual confirmation of said speech input via said multimodal interface; and

,

if visual confirmation is negatively indicated by said interface preference, providing audible confirmation of said speech input via said multimodal interface.

7. The method of claim 4, further comprising the steps of:

determining a user identity; and

,

retrieving said interface preferences from a data store based upon said user identity.

8. The method of claim 4, further comprising the steps of:

establishing an exception against an item within said multimodal interface, wherein said exception overrides said at least one interface preference with regards to said item; and

,

determining whether to present at least one of said visual messages and said audio messages responsive to said exception.

9. The method of claim 1, further comprising the steps of:

ascertaining an acoustic characteristic of an environment in which said multimodal interface is presented; and

,

responsive to said ascertaining step, determining whether to visually present at least one of said messages.

2. The method of said 1, further comprising the steps of:

before said receiving step, visually presenting within said multimodal interface said prompt for said speech input; and

,

after said receiving step, visually presenting within said multimodal interface said confirmation of said speech input.

10. A system for enhancing voice interactions within a multimodal computing device using visual messages comprising the steps of:

means for providing a multimodal interface that includes an audio interface and a visual interface;

means for visually presenting at least one message within said multimodal interface, wherein said message is at least one of a prompt for a speech input and a confirmation of said speech input

means for receiving said speech input; and

,

means for performing a voice recognition task upon at least a portion of said speech input.

11. A machine-readable storage having stored thereon, a computer program having a plurality of code sections, said code sections executable by a machine for causing the machine to perform the steps of:

providing a multimodal interface that includes an audio interface and a visual interface;

visually presenting at least one message within said multimodal interface, wherein said message is at least one of a prompt for a speech input and a confirmation of said speech input;

receiving said speech input; and

,

performing a voice recognition task upon at least a portion of said speech input.

12. The machine-readable storage of claim 11, further comprising the steps of:

before said receiving step, visually presenting within said multimodal interface said prompt for said speech input; and

,

after said receiving step, visually presenting within said multimodal interface said confirmation of said speech input.

13. The machine-readable storage of claim 11, further comprising the step of:

audibly presenting said at least one message within said multimodal interface.

14. The machine-readable storage of claim 11, further comprising the steps of:

allowing a user of said multimodal interface to establish at least one interface preference; and

,

determining whether to visually present at least one of said messages based upon said interface preference.

15. The machine-readable storage of claim 14, wherein said interface preference is associated with establishing visual prompts for speech input, said method further comprising the steps of:

if visual prompting is affirmatively indicated by said interface preference, visually presenting within said multimodal interface said prompt for said speech input; and

,

if visual prompting is negatively indicated by said interface preference, audibly prompting for said speech input.

16. The machine-readable storage of claim 14, wherein said interface preference is associated with establishing visual confirmation of speech input, said method further comprising the steps of:

if visual confirmation is affirmatively indicated by said interface preference, providing visual confirmation of said speech input via said multimodal interface; and

,

if visual confirmation is negatively indicated by said interface preference, providing audible confirmation of said speech input via said multimodal interface.

17. The machine-readable storage of claim 14, further comprising the steps of:

determining a user identity; and

,

retrieving said interface preferences from a data store based upon said user identity.

18. The machine-readable storage of claim 14, further comprising the steps of:

establishing an exception against an item within said multimodal interface, wherein said exception overrides said at least one interface preference with regards to said item; and

,

determining whether to present at least one of said visual messages and said audio messages responsive to said exception.

19. The machine-readable storage of claim 11, further comprising the steps of:

ascertaining an acoustic characteristic of an environment in which said multimodal interface is presented; and

,

responsive to said ascertaining step, determining whether to visually present at least one of said messages.