Method and system for multi-modal communication

• US 8,601,096 B2
• Filed: 05/14/2002
• Issued: 12/03/2013
• Est. Priority Date: 05/14/2002
• Status: Expired due to Fees

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola Mobility LLC (Lenovo Group Ltd.)
• Inventors: Balasuriya, Senaka
• Primary Examiner(s): MIRZA, ADNAN M
• Application Number:
  US10/145,304
• Publication Number:
  US 20030217161A1
• Time in Patent Office: 4,221 Days
• Field of Search: 709/310, 709/101, 709/105, 709/202, 709/217, 709/208, 709/220, 709/221, 709/245, 709/218, 709/230, 709/254, 709/246, 709/231, 709/228, 709/227
• US Class Current: 709/218
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications

Abstract

A method and system for multi-modal communication has a terminal (102) with at least one input component (108) and at least one output component (112) coupled to a gateway (104) having a dialog manager (114). The system and method further has a content server (106) coupled to the dialog manager (114) wherein the content server (106) provides the dialog manager (114) with an encoded control command. Moreover, at least one input component (108) and at least one output component (112) are coupled to a control unit (110). The dialog manager (114) decodes the encoded control command, producing and providing a decoded control command to the control unit (110). In response to the decoded control command, the control unit (110) modifies at least one of the input components (108) and/or at least one of the output components (112).

Claims

1. A method for multi-modal communication comprising:

receiving a control command for an end user multimodal terminal from a content server;

decoding the control command by the multimodal terminal;

providing the decoded control command to a control unit of the multimodal terminal; and

in response to the decoded control command received from the content server, the multimodal terminal modifying at least one input component or at least one output component of the multimodal terminal such that the end user need not modify the at least one input or output component in response to the decoded control command.

2. The method of claim 1 wherein the step of modifying further comprises enabling or disabling at least one of the following:

at least one input component and at least one output component of the multimodal terminal.

3. The method of claim 1 further comprising:

prior to receiving the control command, receiving an information fetch request for requested information and fetching the requested information from a content server, wherein the control command is encoded within the requested information and controls an input or output component of the multimodal terminal.

4. A method for multi-modal communication comprising:

receiving an information fetch request for requested information;

accessing a content server to retrieve the requested information;

receiving a control command by an end user multimodal terminal wherein the control command is either from the content server and the control command is encoded within the requested information, or the control command is received from a control server using a reference indicator encoded within the requested information; and

providing the control command to a control unit of the end user multimodal terminal and in response to the control command, modifying at least one input component or at least one output component of the multimodal terminal such that the end user need not modify the at least one input or output component in response to the control command.

5. The method of claim 4 further comprising:

in response to the control command, enabling an input encoder such that an input command is encoded in a first encoded scheme.

6. The method of claim 4 wherein modifying comprises:

in response to the control command, disabling a speech detector/controller from detecting a barge-in voice command while an audio output is being provided to a speaker within an audio subsystem.

7. The method of claim 4 wherein modifying comprise:

in response to the control command, disabling an output on a display of at least one keypad entry and enabling an output of non-recognition characters on the display wherein a non-recognition character corresponds to each of the keypad entries.

8. A system for multi-modal communication comprising:

at least one dialog manager;

a content server operably coupled to the at least one dialog manager;

a control unit of an end user multimodal terminal operably coupled to the dialog manager wherein the dialog manager receives a control command from the content server, decodes the control command from the content server and provides the control command to the control unit of the end user multimodal terminal;

at least one input component operably coupled to the control unit; and

at least one output component operably coupled to the control unit wherein the control unit, in response to the control command, either enables the at least one input component, enables the at least one output component, disables the at least one input component, or disables the at least one output component such that the end user need not modify the at least one input or output component in response to the decoded control command.

9. The system of claim 8 wherein the control command is encoded in a mark-up language.

10. The system of claim 8 wherein the input component is at least one of the following:

a microphone disposed within an audio subsystem, a speech detector/controller, a speech recognition engine, a keypad, an input encoder, touchscreen, and a handwriting engine.

11. The system of claim 8 wherein the output component is at least one of the following:

a speaker disposed within the audio subsystem and a display.

12. A system for multi-modal communication comprising:

an end user multimodal terminal having at least one input component and at least one output component;

a gateway having a dialog manager operably coupled to the at least one input component and the at least one output component of the multimodal terminal;

a content server operably coupled to the dialog manager wherein the content server provides the dialog manager with a control command for the multimodal terminal; and

wherein the multimodal terminal controls modification of at least one input component or one output component of the end user multimodal terminal in response to the control command such that the end user need not modify the at least one input or output component in response to the control command.

13. The system of claim 12 wherein the at least one input component and the at least one output component are operably coupled to a terminal control unit disposed within the multimodal terminal and the terminal control unit is operably coupled to a gateway control unit disposed within the gateway.

14. The system of claim 13 wherein the dialog manager decodes the encoded control command and thereupon provides the control command to the gateway control unit.

15. The system of claim 14 wherein the gateway control unit provides the terminal control unit the control command and the terminal session control unit, in response to the control command, does at least one of the following:

enables the at least one input component, enables the at least one output component, disables the at least one input component, and disables the at least one output component.

16. The system of claim 15 wherein the input component is at least one of the following:

a microphone disposed within an audio subsystem, a speech detector/controller, a speech recognition engine, a keypad, an input encoder, touchscreen, and a handwriting engine.

17. The system of claim 15 wherein the output component is at least one of the following:

a speaker disposed within the audio subsystem and a display.

18. A system for multi-modal communication comprising:

an end user multimodal terminal having at least one input component, at least one output component, and a dialog manager operably coupled to the at least one input component and the at least one output component;

a content server operably coupled to the dialog manager, wherein the content server provides the dialog manager of the end user multimodal terminal with a control command;

wherein the at least one input component, the at least one output component and the dialog manager are operably coupled to a control unit disposed within the terminal and wherein the dialog manager decodes the encoded control command to generate a decoded control command and provides the decoded control command to the control unit; and

wherein the control unit, in response to the decoded control command, either enables the at least one input component, enables the at least one output component, disables the at least one input component, or disables the at least one output component such that the end user need not modify the at least one input or output component in response to the decoded control command.

19. The system of claim 18 wherein the input component is at least one of the following:

a microphone disposed within an audio subsystem, a speech detector/controller, a speech recognition engine, a keypad, an input encoder, touchscreen, and a handwriting engine and wherein the output component is at least one of the following;

a speaker disposed within the audio subsystem and a display.

20. A multimodal terminal comprising:

at least one input component, at least one output component;

a dialogue manager operatively coupled to the at least one input component and the at least one output component;

a control unit, operatively coupled to the input component and the output component and the dialogue manager, wherein the dialogue manager decodes an encoded control command received from a content server to generate a decoded control command and wherein the control unit, in response to the decoded control command, either enables at least one input component, enables at least one output component, disables at least one input component, or disables at least one output component;

wherein the encoded control command is encoded in requested information from the content server such that the end user need not modify the at least one input or output component in response to the decoded control command.

21. The method of claim 20 further comprising:

in response to the control command, enabling an input encoder such that an input command is encoded in a first encoded scheme.

22. The method of claim 20 further comprising:

in response to the control command, disabling a speech detector/controller from detecting a barge-in voice command while an audio output is being provided to a speaker within an audio subsystem.

23. The method of claim 20 further comprising:

in response to the control command, disabling an output on a display of at least one keypad entry and enabling an output of non-recognition characters on the display wherein a non-recognition character corresponds to each of the keypad entries.

24. A method for multi-modal communication comprising:

receiving a control command for a multimodal terminal from a content server;

decoding the control command received from the content server by the multimodal terminal;

providing the decoded control command to a control unit of the multimodal terminal;

in response to the decoded control command, modifying at least one input component or at least one output component of the multimodal terminal such that the end user need not modify the at least one input or output component in response to the decoded control command;

wherein the step of modifying further comprises enabling or disabling at least one of the following;

at least one input component and at least one output component of the multimodal terminal; and

prior to receiving the control command, receiving an information fetch request for requested information and fetching the requested information from a content server, wherein the control command is encoded within the requested information and controls an input or output component of the multimodal terminal.

25. A method for multi-modal communication comprising:

receiving a control command for a mobile multimodal terminal from a content server wherein the control command is encoded in a mark-up language;

decoding the control command received from the content server by the mobile multimodal terminal;

providing the decoded control command to a control unit in the mobile multimodal terminal;

in response to the decoded control command, modifying at least one input component or at least one output component of the mobile multimodal terminal such that the end user need not modify the at least one input or output component in response to the decoded control command;

wherein the input component is at least one of the following;

a microphone disposed within an audio subsystem, a speech detector/controller, a speech recognition engine, a keypad, an input encoder, touchscreen, and a handwriting engine; and

wherein the output component is at least one of the following;

a speaker disposed within the audio subsystem and a display.