System and method for concurrent multimodal communication
First Claim
Patent Images
1. A method for multimodal communication comprising:
- obtaining modality specific instructions for a plurality of user agent programs that operate in different input modalities with respect to each other; and
during a session, synchronizing output from the plurality of user agent programs based on the modality specific instructions.
4 Assignments
0 Petitions
Accused Products
Abstract
A multimodal network element facilitates concurrent multimodal communication sessions through differing user agent programs on one or more devices. For example, a user agent program communicating in a voice mode, such as a voice browser in a voice gateway that includes a speech engine and call/session termination, is synchronized with another user agent program operating in a different modality, such as a graphical browser on a mobile device. The plurality of user agent programs are operatively coupled with a content server during a session to enable concurrent multimodal interaction.
363 Citations
23 Claims
-
1. A method for multimodal communication comprising:
-
obtaining modality specific instructions for a plurality of user agent programs that operate in different input modalities with respect to each other; and
during a session, synchronizing output from the plurality of user agent programs based on the modality specific instructions. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A multimodal network element comprising:
-
an information fetcher operative to obtain modality specific instructions for a plurality of user agent programs that operate in different input modalities with respect to each other during a same session; and
a concurrent multimodal synchronization coordinator, operatively coupled to the information fetcher and operative to, during the session, synchronize output from the plurality of user agent programs based on the modality specific instructions. - View Dependent Claims (11)
-
-
12. A method for multimodal communication comprising:
-
sending a request for concurrent multimodal input information corresponding to multiple input modalities associated with a plurality of user agent programs operating during a same session; and
fusing received concurrent multimodal input information sent from the plurality of user agent programs sent in response to the request for concurrent different multimodal information. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19)
prior to sending the request for concurrent multimodal input information, determining a proxy for each different modality associated with each application associated with a different modality for a given session; and
synchronizing the request for concurrent different multimodal input information to the plurality of user agent programs using the proxies determined for each different modality.
-
-
14. The method of claim 12 including waiting a period of time to determine whether the concurrent multimodal input information was suitably received for fusion.
-
15. The method of claim 14 including waiting a different period of time depending on a modality of each user agent program.
-
16. The method of claim 12 wherein sending the request for concurrent multimodal input information corresponding to multiple input modalities includes sending modality specific instructions for a plurality of user agent programs that operate in different modalities with respect to each other to request concurrent entry of information in different modalities.
-
17. The method of claim 12 wherein the step of sending the request for concurrent multimodal input information corresponding to multiple input modalities includes sending a first modality based markup language form to a device and sending a second modality markup language based form to one or more devices to request concurrent entry of information in different modalities.
-
18. The method of claim 12 including the step of combining received multimodal input information sent from one of the plurality of user agent programs sent in response to the request for concurrent different multimodal information based on a time received and based on confidence values of individual results received.
-
19. The method of claim 12 including transcoding modality specific instructions to include data identifying modality specific instructions for another user agent program operating in a different modality.
-
20. A multimodal network element comprising:
-
a plurality of proxies that each send a request for concurrent multimodal input information corresponding to multiple input modalities associated with a plurality user agent programs operating during a same session; and
a multimodal fusion engine, operatively responsive to received concurrent multimodal input information sent from the plurality of user agent programs sent in response to the request for concurrent different multimodal information and operative to fuse the different multimodal input information sent from the plurality of user agent programs to provide concurrent multimodal communication from differing user agent programs during a same session. - View Dependent Claims (21, 22, 23)
an information fetcher operative to obtain modality specific instructions for the plurality user agent programs that operate in different modalities with respect to each other during a same session; and
a concurrent multimodal synchronization coordinator, operatively coupled to the information fetcher and the plurality of proxies, operative to synchronize the received concurrent multimodal input information output from the plurality of user agent programs, during a session.
-
-
22. The multimodal network element of claim 21 including:
-
a concurrent multimodal session persistence controller operatively coupled to the plurality user agent programs, to maintain, during non-session conditions and on a per user basis, concurrent multimodal session status information of the plurality user agent programs configured for different concurrent modality communication during a session and to re-establish a concurrent multimodal session in response to accessing the concurrent multimodal session status information; and
memory, operatively coupled to the concurrent multimodal session persistence controller, containing the concurrent multimodal session status information.
-
-
23. The multimodal network element of claim 20 including a markup language interpreter and a transcoder, operatively coupled to the markup language interpreter, wherein the transcoder transcodes modality specific instructions to include data identifying modality specific instructions for another user agent program operating in a different modality.
Specification