Multi-modal messaging

• US 20040019487A1
• Filed: 03/11/2003
• Published: 01/29/2004
• Est. Priority Date: 03/11/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Maes, Stephane H., Sedivy, Jan, Labsky, Martin, Kleindienst, Jan
• Granted Patent: US 7,315,613 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/74 with voice recognition means
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

Systems and methods for multi-modal messaging that enable a user to compose, send and retrieve messages, such as SMS, MMS, IM or ordinary e-mail messages, for example, using one or more I/O (input/output) modalities (e.g., speech I/O and/or GUI I/O). A method for composing messages combines the advantages of a multi-modal interface (e.g., grammar-based speech and touchscreen or similar input devices) and message templates, which allows a user to construct a message with significantly less effort in a fraction of the time required by conventional methods. The user can dictate his/her messages using speech and/or GUI input, for example, based on a library of message templates which can be personalized by the user to fit his/her social interaction needs.

Claims

1. A method for providing multi-modal messaging, comprising the steps of:

receiving user input through a multi-modal user interface;

composing a message responsive to the user input using one or more message templates; and

forwarding the message to a target recipient using a messaging protocol.

2. The method of claim 1, wherein the multi-modal user input comprises speech input that is recognized using automatic speech recognition.

3. The method of claim 2, wherein the automatic speech recognition is based on grammars.

4. The method of claim 2, wherein the automatic speech recognition is based on prompts.

5. The method of claim 1., further comprising the steps of selecting a message format of the message as specified by the user input.

6. The method of claim 1, wherein the multi-modal user interface comprises a multi-modal browser.

7. The method of claim 1, comprising the step of maintaining a library of message templates that can be accessed by the user to edit a predefined message template or generate a new message template.

8. The method of claim 1, comprising the step of grouping the message templates into one or more categories or subcategories, wherein each category and subcategory comprises one or more keywords, each keyword being associated with a text string that is used to compose a message, and wherein the step of composing a message comprises selecting one or more categories, subcategories, and keywords in response to the user input to select a text string.

9. The method of claim 8, wherein the text string comprises a field that is associated with a grammar which is used to fill the field based on speech input from the user.

10. The method of claim 1, wherein the step of composing a message comprises composing a SMS (short message service) message.

11. The method of claim 1, comprising the step of translating the message to a language of the target recipient.

12. The method of claim 11, wherein the step of translating comprises selecting the target recipient from an address book and translating the message to the language associated with the recipient as pre-specified in the address book.

13. A method for authoring message templates for use with a messaging system, the method comprising the steps of:

defining a category that is associated with one or more related template messages;

defining a keyword for each template message associated with the category, wherein each keyword is a unique identifier for the associated template message;

specifying one or more properties associated with the category, said one or more properties including an identification property that is used for logically associating the category with a same category in a different language.

14. The method of claim 13, wherein the step of specifying one or more properties comprises specifying a language property that specifies a language of subcategories and templates associated with the category.

15. The method of claim 13, wherein the step of specifying one or more properties comprises specifying a name property which is used to represent the category visually on a display or which is used for recognizing spoken input.

16. The method of claim 13, wherein the step of specifying one or more properties comprises specifying an icon property that specifies a name of an image file that represents the category.

17. The method of claim 13, further comprising the step of defining character data that defines a message template associated with the category.

18. The method of claim 17, further comprising the step of defining one or more fill tags that enables a user to add character data to a message template.

19. The method of claim 18, further comprising the step of defining a grammar property for a fill tag, wherein the grammar property specifies a grammar that is used to add character data to the fill tag based on spoken input.

20. A multi-modal messaging system, comprising:

a multi-modal user interface for receiving user input, said user input comprising speech input;

an automatic speech recognition system for recognizing the speech input;

a messaging application for composing a message responsive to the user input using one or more message templates.

21. The system of claim 20, wherein the multi-modal user interface comprises a multi-modal browser.

22. The system of claim 20, wherein the multi-modal user interface executes on a mobile device.

23. The system of claim 20, wherein the messaging application comprises a SMS (short message service) application.

24. The system of claim 20, wherein the messaging system provides support for multiple languages and automatic translation of a message to a language of a target recipient.

25. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for providing multi-modal messaging, the method steps comprising:

receiving user input through a multi-modal user interface;

composing a message responsive to the user input using one or more message templates; and

forwarding the message to a target recipient using a messaging protocol.

26. The program storage device of claim 25, wherein the multi-modal user input comprises speech input that is recognized using automatic speech recognition.

27. The program storage device of claim 26, wherein the automatic speech recognition is based on grammars.

28. The program storage device of claim 26, wherein the automatic speech recognition is based on prompts.

29. The program storage device of claim 25, further comprising instructions for selecting a message format of the message as specified by the user input.

30. The program storage device of claim 25, wherein the multi-modal user interface comprises a multi-modal browser.

31. The program storage device of claim 25, comprising instructions for maintaining a library of message templates that can be accessed by the user to edit a predefined message template or generate a new message template.

32. The program storage device of claim 25, comprising instructions for grouping the message templates into one or more categories or subcategories, wherein each category and subcategory comprises one or more keywords, each keyword being associated with a text string that is used to compose a message, and wherein composing a message comprises instructions for selecting one or more categories, subcategories, and keywords in response to the user input to select a text string.

33. The program storage device of claim 32, wherein the text string comprises a field that is associated with a grammar which is used to fill the field based on speech input from the user.

34. The program storage device of claim 25, wherein the instructions for composing a message comprise instructions for composing a SMS (short message service) message.

35. The program storage device of claim 25, comprising instructions for translating the message to a language of the target recipient.

36. The program storage device of claim 35, wherein the instructions for translating comprise instructions for selecting the target recipient from an address book and translating the message to the language associated with the recipient as pre-specified in the address book.