Multi-modal messaging

• US 7,315,613 B2
• Filed: 03/11/2003
• Issued: 01/01/2008
• Est. Priority Date: 03/11/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Labsky, Martin, Maes, Stephane H., Sedivy, Jan, Kleindienst, Jan
• Primary Examiner(s): Gauthier; Gerald
• Application Number:
  US10/386,247
• Publication Number:
  US 20040019487A1
• Time in Patent Office: 1,757 Days
• Field of Search: 379/88.17, 379/88.13, 379/88.18, 379/265.01, 379/265.02, 379/265.09, 379/88.16, 379/67.1, 379/88.01, 707/3, 715/511
• US Class Current: 379/88.13
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

Systems and methods for multi-modal messaging that enable a user to compose, send and retrieve messages, such as SMS, MMS, IM or ordinary e-mail messages, for example, using one or more I/O (input/output) modalities (e.g., speech I/O and/or GUI I/O). A method for composing messages combines the advantages of a multi-modal interface (e.g., grammar-based speech and touchscreen or similar input devices) and message templates, which allows a user to construct a message with significantly less effort in a fraction of the time required by conventional methods. The user can dictate his/her messages using speech and/or GUI input, for example, based on a library of message templates which can be personalized by the user to fit his/her social interaction needs.

Claims

1. A method for providing multi-modal messaging, comprising:

maintaining a library of message templates, wherein each message template comprises a predefined text string, wherein at least one message template includes a fillable text field, wherein the message templates are grouped into one or more categories or subcategories, wherein each category and subcategory comprises one or more keywords associated therewith, and wherein each keyword for a given category or subcategories is associated with one message template within the group of message templates for the given category or subcategory;

receiving user input through a multi-modal user interface;

composing a message responsive to the user input using one or more message templates that are maintained in the library of message templates, wherein composing a message comprises accessing a target message template in the library in response to the user input by selecting a category or subcategory in which the target message template is grouped and then selecting the keyword within the category or subcategory associated with the target message template, and

forwarding the message to a target recipient using a messaging protocol.

2. The method of claim 1, wherein the multi-modal user input comprises speech input that is recognized using automatic speech recognition.

3. The method of claim 2, wherein the automatic speech recognition is based on grammars.

4. The method of claim 2, wherein the automatic speech recognition is based on prompts.

5. The method of claim 1, further comprising selecting a message format of the message as specified by the user input.

6. The method of claim 1, wherein the multi-modal user interface comprises a multi-modal browser.

7. The method of claim 1, wherein the library of message templates can be accessed by the user to edit a predefined message template or generate a new message template.

8. The method of claim 1, wherein composing the message further comprises filling a fillable field in the target message template in response to user input.

9. The method of claim 8, wherein the fillable field of the target template is associated with a grammar which is activated and used to fill the fillable field of the target message template based on speech input from the user.

10. The method of claim 1 wherein composing a message comprises composing a SMS (short message service) message.

11. The method of claim 1, comprising translating the message to a language of the target recipient.

12. The method of claim 11, wherein translating comprises selecting, the target recipient from an address book and translating, the message to the language associated with the recipient as pre-specified in the address book.

13. A multi-modal in messaging system, comprising:

a multi-modal user interface for receiving user input, said use input comprising speech input;

an automatic speech recognition system for recognizing the speech input;

a messaging application for composing a message responsive to the user input using one or more message templates,

wherein the messaging application maintain a library of message templates, wherein each message template comprises a predefined text string, wherein at least one message template includes a fillable text field, wherein the message templates are grouped into one or more categories or subcategories, wherein each category and subcategory comprises one or more keywords associated therewith, and wherein each keyword for a given category or subcategory is associated with one message template within the group of message templates for the given category or subcategory, and

wherein a user interacts with the messaging application to compose a message by accessing a target message template in the library in response to the user spoken input to select a category or subcategory in which the target message template is grouped and then select the keyword within the category or subcategory associated with the target message template in response to the user spoken input.

14. The system of claim 13, wherein the multi-modal user interface comprises a multi-modal browser.

15. The system of claim 13, wherein the multi-modal user interface executes on a mobile device.

16. The system of claim 13, wherein the messaging application comprises a SMS (short message service) application.

17. The system of claim 13, wherein the messaging system provides support the multiple languages and automatic translation of a message to a language of a target recipient.

18. The system of claim 13, wherein the user interacts with the messaging application to compose a message by filling a fillable field in the target message template in response to user spoken input wherein the fillable field of the target template is associated with a grammar which is activated and used to fill the fillable field of the target message template based on the spoken input from the user.

19. A program storage device readable by a computer tangibly embodying a program of instructions executable by the computer to perform method steps for providing multi-modal messaging, the method steps comprising:

maintaining a library of message templates, wherein each message template comprises a predetined text string, wherein at least one message template includes a fillable text field, wherein the message templates are grouped into one or more categories or subcategories, wherein each category and subcategory comprises one or more keywords associated therewith, and wherein each keyword for a given category or subcategory is associated with one message template within the group of message templates for the given category or subcategory;

receiving user input through a multi-modal user interface;

composing a message responsive to the user input using one or more message templates that are maintained in the library of message templates, wherein composing a message comprises accessing a target message template in the library in response to the user input by selecting a category or subcategory in which the target message template is grouped and then selecting the keyword within the category or subcategory associated with the target message template; and

forwarding the message to a target recipient using a messaging protocol.

20. The program storage device of claim 19, wherein the multi-modal user input comprises speech input that is recognized using automatic speech recognition.

21. The program storage device of claim 20, wherein the automatic speech recognition is based on grammars.

22. The program storage device of claim 20, wherein the automatic speech recognition is based on prompts.

23. The program storage device of claim 19, further comprising instructions for selecting a message format of the message as specified by the user input.

24. The program storage device of claim 19, wherein the multi-modal user interface comprises a multi-modal browser.

25. The program storage device of claim 19, comprising instructions for maintaining a library of message templates that can be accessed by the user to edit a predefined message template or generate a new message template.

26. The program storage device of claim 19, wherein the instruction for composing the message further comprise instructions for filling a fillable field in the target message template in response to user input.

27. The program storage device of claim 26, wherein the fillable field of the target template is associated with a grammar which is activated and used to fill the tillable field of the target message template based on speech input from the user.

28. The program storage device of claim 19, wherein the instructions for composing a message comprise instructions for composing a SMS (short message service) message.

29. The program storage device of claim 19, comprising instructions for translating the message to a language of the target recipient.

30. The program storage device of claim 29, wherein the instructions for translating comprise instructions for selecting the target recipient from an address book and translating the message to the language associated with the recipient as pre-specified in the address book.