Multimodal information services

• US 20010049603A1
• Filed: 03/08/2001
• Published: 12/06/2001
• Est. Priority Date: 03/10/2000
• Status: Active Grant

• Current Assignee: Amobee, Inc. (Nexxen International Ltd.)
• Original Assignee: Webversa, Inc. (Nexxen International Ltd.)
• Inventors: Sravanapudi, Ajay P., Day, Richard D.
• Granted Patent: US 7,167,830 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context
• CPC Class Codes: H04L 69/14 Multichannel or multilink protocols
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 88/06 adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals

Abstract

A system for multimodal information services includes a multimodal information service mechanism for providing the multimodal information services. An information user connects to a first network through at least one communication device, the first network connecting to the multimodal information service mechanism, the information user requesting and receiving the multimodal information services from the multimodal information service mechanism through the device. An information source connects to the multimodal information service mechanism via a second network, the information source providing multimodal information to the multimodal information service mechanism to facilitate the multimodal information services.

Claims

1. A system for multimodal information services, comprising:

a multimodal information service mechanism for providing said multimodal information services;

an information user connecting to a first network through at least one communication device, said first network connecting to said multimodal information service mechanism, said information user requesting and receiving said multimodal information services from said multimodal information service mechanism through said device; and

an information source connecting to said multimodal information service mechanism via a second network, said information source providing multimodal information to said multimodal information service mechanism to facilitate said multimodal information services.

2. The system according to

claim 1

, wherein said communication device includes at least one of:

a wireless phone;

a PDA;

a pager;

a facsimile device;

or

an e-mail mechanism.

3. The system according to

claim 1

, wherein said first network and said second network include at least one of:

the Internet;

an intranet;

a wireless network;

or

a public service telephony network.

4. The system according to

claim 1

, wherein said information source includes at least one of:

a database;

a web site; and

an application.

5. The system according to

claim 1

, wherein said multimodal information service mechanism comprises:

a multimodal platform; and

at least one multimodal information service application, each of said at least one multimodal information service application providing a service to said information user and being constructed based on said multimodal platform.

6. A system for a multimodal platform, comprising:

a multimodal connection mechanism for switching multimodal data through appropriate channels; and

a multimodal interaction enabling mechanism for facilitating multimodal data rendering.

7. The system according to

claim 6

, wherein said multimodal connection mechanism comprises:

at least one data channel for transferring data in different modalities; and

a channel switching mechanism for routing multimodal data to said at least one data channel.

8. The system according to

claim 6

, wherein said multimodal interaction enabling mechanism comprises:

a data adapter for rendering data in a source modality into data in a destination modality, said data in said source modality being obtained from an information source;

an adapted data storage for storing data in said destination modality, converted by said data adapter; and

a multimodal server for facilitating said data adapter to perform said rendering said data in source modality to said data in destination modality.

9. The system according to

claim 8

, wherein said data adapter comprises:

an application interface mechanism for performing interaction between said multimodal platform and a multimodal information service application;

a data retrieval mechanism for accessing said data in source modality from said information source; and

a multimodal data rendering mechanism for rendering said data in source modality into said data in destination modality wherein said destination modality being specified by said application interface mechanism based on the interaction between said multimodal platform and said multimodal information service application.

10. The system according to

claim 9

, wherein said multimodal data rendering mechanism comprises at least one of:

an electronic mail renderer for rendering said data in source modality into an electronic mail;

a facsimile renderer for rendering said data in source modality into a facsimile;

a paging renderer for rendering said data in source modality into a paging message;

a text-to-speech renderer for rendering data in text modality into data in speech modality; and

a speech-to-text renderer for rendering data in speech modality into data in text modality.

11. The system according to

claim 10

, wherein said multimodal server comprises at least one of:

a mail server for facilitating said electronic mail renderer;

a facsimile server for facilitating said facsimile renderer;

a short message service for facilitating said paging renderer;

a text-to-speech engine for facilitating text-to-speech renderer by generating audio speech data according to text data;

an automatic speech recognizer for facilitating speech-to-text renderer by generating text data based on audio speech data using automated speech recognition; and

a telephony server for facilitating said text-to-speech engine and said automatic speech recognizer.

12. A system of a multimodal information service application for a voice-enabled Internet access service, comprising:

a dialog unit for performing a voice based dialog with an information user;

a voice clipboard for storing the content of said dialog;

a request queue for storing a service request issued by said information user during said voice based dialog, said service request requesting, from said voice-enabled Internet access service, Internet information to be delivered to said information user in a destination modality, said request being accessed by an application interface mechanism of a multimodal platform to generate requested Internet information in said destination modality; and

a result queue for storing said Internet information in said destination modality, generated by said multimodal platform according to said request.

13. The system according to

claim 12

, further comprising a statistics collection unit for gathering statistics based on said voice based dialog.

14. A system of a multimodal information service application for multimodal e-business alert service, comprising:

a multimodal e-business alert configuration that dictates different aspects of how said multimodal e-business alert service is to behave; and

a multimodal e-business alert unit for generating a multimodal e-business alert based on a multimodal e-business alert specification contained in an event issued by an e-business provider and said multimodal e-business alert configuration, said multimodal e-business alert, once generated, being sent to an e-business consumer according to said multimodal e-business alert specification.

15. The system according to

claim 14

, wherein said multimodal e-business alert unit comprises:

at least one incoming request queue for queuing an event representing an alert request received from said e-business provider;

an e-business alert service for constructing said e-business alert request based on said alert request and said multimodal e-business alert specification, for sending said e-business alert request to a multimodal platform, and for tracking the performance of said multimodal e-business alert service via various queues; and

at least one outgoing request queue for queuing said e-business alert request, said at least one outgoing alert queue connecting to a multimodal data rendering mechanism of a multimodal platform, each of said at least one outgoing request queue corresponding to a different modality, said e-business alert request queued in said at least outgoing queue being accessed, by said multimodal data rendering mechanism of said multimodal platform to generate an e-business alert in an destination modality according to said e-business alert specification and to send said e-business alert to said e-business consumer.

16. The system according to

claim 15

, wherein said at least one incoming request queue comprises at least one of:

an incoming ASAP queue for queuing an event, received from said e-business provider, that is to be converted into a multimodal e-business alert and that is specified to be sent to said e-business consumer as soon as possible; and

an incoming timed queue for queuing an event, received from said e-business provider, that is specified to be sent as a multimodal e-business alert to said e-business consumer at a given time.

17. The system according to

claim 15

, wherein said at least one outgoing request queue comprises at least one of:

a facsimile queue for queuing an e-business alert request that corresponds to an e-business alert that is to be sent to said e-business consumer via facsimile;

an electronic mail queue for queuing an e-business alert request that corresponds to an e-business alert that is to be sent to said e-business consumer via an electronic mail;

a pager queue for queuing an e-business alert request that corresponds to an e-business alert that is to be sent to said e-business consumer via a pager; and

a voice queue for queuing an e-business alert request that corresponds to an e-business alert that is to be sent to said e-business consumer via a voice channel.

18. The system according to

claim 15

, further comprising:

a response queue for storing an outcome of sending an e-business alert to said e-business consumer;

at least one status queue for storing information related to the performance of said e-business alert service;

a data recorder for recording information from said at least one status queue; and

a history database for storing information related to the history of sending e-business alerts to and receiving responses from said e-business consumer.

29. The method according to

claim 15

and

claim 28

, wherein said multimodal e-business alert specification comprises:

an e-business history; and

a push.

30. The method according to

claim 29

, wherein said push comprises at least one of:

at least one criterion under which a multimodal e-business alert is to be sent to said e-business consumer;

a default means by which said multimodal e-business alert is to be sent to said e-business consumer;

or

zero or more alternative means to send said multimodal e-business alert when an attempt to send said e-business alert using said default means fails, said zero or more alternative means being applied when a condition is satisfied.

31. The method according to

claim 30

, wherein said default means and said zero or more alternative means include at least one of:

electronic mail;

facsimile;

pager;

wired phone;

or

wireless phone.

32. The method according to

claim 30

, wherein said condition includes a situation in which a phone, to where an e-business alert is sent using said default means, is busy.

33. The method according to

claim 30

, wherein said condition includes the situation in which a phone, to where an e-business alert is sent via one of said zero or more alternative means, is not answered.

34. The method according to

claim 30

, wherein said sending a multimodal e-business alert comprises:

receiving, by said multimodal information service application, said event from said e-business provider;

retrieving said e-business alert configuration associated with said e-business consumer;

processing said multimodal e-business alert specification to generate said e-business alert request;

pushing said e-business alert request into one of a plurality of outgoing request queues;

generating, by said multimodal platform, said e-business alert based on said e-business alert request; and

sending said e-business alert to said e-business consumer using said default means through a multimodal connection mechanism in said multimodal platform.

35. The method according to

claim 34

, further comprising:

examining an response yielded from said sending to see whether said sending is successful, said examining generating an outcome of either negative or positive;

updating an e-business history, if said outcome is positive;

repeating the acts starting from said processing, if said alternative means is defined in said multimodal e-business alert specification.

19. A method for multimodal information services, comprising:

receiving, by a multimodal information service application in a multimodal information service mechanism, a service request from an information user via a first network, said request being issued through a communication device and requesting a service offered by said multimodal information service application;

identifying an information source according to said request, from where the requested information desired by said information user can be retrieved;

determining a source modality in which said requested information is represented and a destination modality in which said requested information is to be delivered to said information user;

retrieving, by a multimodal platform in said multimodal information service mechanism, said requested information from said information source in said source modality to generate source information;

generating destination information from said source information by rendering said source information into said destination information, if said source information is not in said destination modality;

switching said destination information into a channel that is appropriate for said destination modality; and

sending, by said multimodal information service application, said destination information to said information user from said channel to said communication device via a second network.

20. The method according to

claim 19

, wherein said communication device includes at least one of:

a wireless phone;

a PDA;

a pager;

a facsimile device;

or

a electronic mailing mechanism.

21. The method according to

claim 19

, wherein said first network and said second network include at least one of:

the Internet;

a wireless network;

or

a public service telephony network.

22. The method according to

claim 19

, wherein said information source includes at least one of:

a database;

a web site; and

an application.

23. The method according to

claim 19

, further comprising storing said destination information, rendered by said rendering, in an adapted data storage.

24. A multimodal information service application method for voice-enabled Internet access service, comprising:

conducting, by a dialog unit in said multimodal information service application, a conversation with an information user via a communication device;

updating a clipboard according to said conversation;

processing said conversation to extract a request for information, specified by said information user, and a destination modality in which said information is to be sent to said information user;

sending said request and said destination modality to a multimodal platform that facilitates data rendering among different modalities;

generating, by said multimodal platform, destination information by rendering said information into said destination modality according to said request;

receiving, by said multimodal information service application from said multimodal platform, said destination information; and

sending said destination information to said information user.

25. The method according to

claim 24

, wherein said conversation comprises at least one dialog, each of said at least one dialog including a prompt and a response.

26. The method according to

claim 25

, wherein said clipboard stores the content of said at least one dialog according to a schema defined in a standard language.

27. The method according to

claim 26

, wherein said standard language includes Voice Extensible Markup Language.

28. A method of multimodal information service application for multimodal e-business alert service, comprising:

constructing a multimodal e-business alert configuration that dictates how said multimodal e-business alert service is to behave;

issuing an event, by an e-business provider, to said multimodal e-business alert service, said event requesting said multimodal e-business alert service to send an e-business alert to an e-business consumer, said event including a multimodal e-business alert specification that instructs said multimodal e-business alert service how said e-business alert is to be sent to said e-business consumer;

sending, by said multimodal information service application, a multimodal e-business alert to said e-business consumer based on said multimodal e-business alert configuration and said multimodal e-business alert specification contained in said event, said multimodal e-business alert being rendered by a multimodal platform in a multimodal information service mechanism.

36. A computer-readable medium encoded with a program for multimodal information services, said program comprising:

receiving, by a multimodal information service application in a multimodal information service mechanism, a service request from an information user via a first network, said request being issued through a communication device and requesting a service offered by said multimodal information service application;

identifying an information source according to said request, from where the requested information desired by said information user can be retrieved;

determining a source modality in which said requested information is represented and a destination modality in which said requested information is to be delivered to said information user;

retrieving, by a multimodal platform in said multimodal information service mechanism, said requested information from said information source in said source modality to generate source information;

generating destination information from said source information by rendering said source information into said destination information, if said source information is not in said destination modality;

switching said destination information into a channel that is appropriate for said destination modality; and

sending, by said multimodal information service application, said destination information to said information user from said channel to said communication device via a second network.

37. The medium according to

claim 36

, said program further comprising storing said destination information, rendered by said rendering, in an adapted data storage.

38. A computer-readable medium encoded with a program for voice-enabled Internet access service, said program comprising:

conducting, by a dialog unit in said multimodal information service application, a conversation with an information user via a communication device;

updating a clipboard according to said conversation;

processing said conversation to extract a request for information, specified by said information user, and a destination modality in which said information is to be sent to said information user;

sending said request and said destination modality to a multimodal platform that facilitates data rendering among different modalities;

generating, by said multimodal platform, destination information by rendering said information into said destination modality according to said request;

receiving, by said multimodal information service application from said multimodal platform, said destination information; and

sending said destination information to said information user.

39. A computer-readable medium encoded with a program for multimodal e-business alert service, said program comprising:

constructing a multimodal e-business alert configuration that dictates how said multimodal e-business alert service is to behave;

issuing an event, by an e-business provider, to said multimodal e-business alert service, said event requesting said multimodal e-business alert service to send an e-business alert to an e-business consumer, said event including a multimodal e-business alert specification that instructs said multimodal e-business alert service how said e-business alert is to be sent to said e-business consumer;

sending, by said multimodal information service application, a multimodal e-business alert to said e-business consumer based on said multimodal e-business alert configuration and said multimodal e-business alert specification contained in said event, said multimodal e-business alert being rendered by a multimodal platform in a multimodal information service mechanism.

40. The medium according to

claim 39

, wherein said sending a multimodal e-business alert comprises:

receiving, by said multimodal information service application, said event from said e-business provider;

retrieving said e-business alert configuration associated with said e-business consumer;

processing said multimodal e-business alert specification to generate said e-business alert request;

pushing said e-business alert request into one of a plurality of outgoing request queues;

generating, by said multimodal platform, said e-business alert based on said e-business alert request; and

sending said e-business alert to said e-business consumer using said default means through a multimodal connection mechanism in said multimodal platform.

41. The medium according to

claim 40

, said program further comprising:

examining an response yielded from said sending to see whether said sending is successful, said examining generating an outcome of either negative or positive;

updating an e-business history, if said outcome is positive;

repeating the acts starting from said processing, if said alternative means is defined in said multimodal e-business alert specification.