Multi-modal information retrieval system

• US 20040172254A1
• Filed: 01/14/2004
• Published: 09/02/2004
• Est. Priority Date: 01/14/2003
• Status: Active Grant

• Current Assignee: xAd, Inc
• Original Assignee: xAd, Inc
• Inventors: Kumar, Sunil, Sharma, Dipanshu, Kholia, Chandra
• Granted Patent: US 7,054,818 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: A61K 8/411 Aromatic amines, i.e. where the amino group is directly linked to the aromatic nucleus
• CPC Class Codes: A61K 8/4913 having five membered rings, e.g. pyrrolidone carboxylic acid
• CPC Class Codes: A61Q 5/10 Preparations for permanently dyeing the hair
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Multi-modal system which enables documents to be interpreted as either or both of voice browser-based documents and/or visual browser-based documents for a thin client such as a portable telephone. Special techniques and additions are added into the document to enable the document to be converted between voice markup and visual markup languages. In addition, the document can be simultaneously viewed in both of the voice markup and the visual markup languages. Special techniques are used to allow keeping track of the browsing position within this document.

Claims

1. A method, comprising:

receiving a document in a structured language which includes tags associated with portions of the document;

using said tags to provide a speech mark-up language version from said document; and

using said tags, and the same said document, to provide a visual mark-up language version from said document.

2. A method as in claim 1, further comprising playing speech corresponding to said speech mark-up language version.

3. A method as in claim 1, further comprising switching from one of said versions to the other of said versions.

4. A method as in claim 1, further comprising providing a document which can be used in both speech mark-up language mode and visual mark-up language mode at the same time.

5. A method as in claim 1, further comprising allowing the user to browse a document in said visual mark-up language, and accepting a command to provide additional content in said speech mark-up language version at the same time as providing said visual mark-up language version.

6. A method as in claim 1, further comprising allowing browsing a document in said speech mark-up language and accepting a command to provide additional content in said visual markup language at the same time.

7. A method as in claim 1, wherein said visual mark-up language is one which can be executed over a WAP browser.

8. A method as in claim 1, wherein said document is one which relates to e-mails.

9. A method as in claim 1, further comprising receiving a document with a SWITCH tag that allows a user to switch content from one of said mark-up versions to the other said mark-up versions.

10. A method as in claim 1, wherein said determining a need for visual mark-up language comprises determining if an initiating terminal has visual mark-up language capability.

11. A method as in claim 10, wherein said determining if the initiating terminal has visual mark-up capability comprises sending a message to the initiating terminal and determining if the initiating terminal completes registration based on the message.

12. A method as in claim 10, wherein the initiating terminal is a handheld telephone.

13. A method as in claim 10, wherein said determining if the initiating terminal has visual mark-up capability comprises informing the terminal to access a predetermined website which requires visual mark-up capabilities, and determining if the terminal has access to said website.

14. A method as in claim 13, further comprising using said determining to determine if the terminal has short message service capabilities.

15. A method as in claim 1, further comprising a SHOW tag which commands the terminal to show both voice mark-up mode and visual mark-up mode at the same time.

16. A method as in claim 15, wherein said show tag requests a server to send both voice mark-up information and visual mark-up information.

17. A method as in claim 14, further comprising using said determining to determine if a message should be sent using SMS or SMTP.

18. A method as in claim 1, wherein said visual mark-up language is one of WML, XHTML, or cHTML.

19. A method as in claim 3, further comprising switching by determining a page that the user is currently using, and providing content and providing content for the same page in the other mark-up language.

20. A method as in claim 3, further comprising repeating currently viewed information during said switching.

21. A method as in claim 3, wherein said switching is carried out by initiating a special switching tag.

22. A method as in claim 3, wherein said switching out is carried out by initiating a link within the content.

23. A method as in claim 1, further comprising allowing the content to be browsed partly in visual form and partly in voice form.

24. A method as in claim 23, further comprising dividing the source documents into multiple parts, and converting each of the multiple parts into either or both of visual mark-up language and/or voice mark-up language.

25. A system, comprising:

an information storage unit which stores a document in structured language that includes tags associated with portions of the document; and

a conversion server which allows converting said tags to provide both information in a voice mark-up language and information in a visual mark-up language, based on the same document.

26. A system as in claim 25, wherein said voice mark-up language is in VoiceXML.

27. A system as in claim 25, wherein said voice mark-up language is in a web compatible language.

28. A system as in claim 25, further comprising a portable terminal, which receives information from said conversion server, and displays said information from said conversion server.

29. A system as in claim 25, wherein said conversion server is operative responsive to a command, to convert said information between voice mark-up language and visual mark-up language.

30. A system as in claim 29, wherein said information can be converted to either or both voice mark-up language and visual mark-up language.

31. A method a system as in claim 29, wherein said conversion server provides documents which have different units, each unit being separately marked as one or both of voice mark-up language and/or visual mark-up language.

32. A system as in claim 28, wherein said portable terminal is a handheld telephone.

33. A system as in claim 32, wherein said portable telephone keeps track of a currently browsed portion by unit numbers, and allows changing a currently browsed format.

34. A system as in claim 32, wherein said conversion server provides a duplicate version of the currently browsed portion responsive to said changing.

35. A document, comprising:

a structured format including text attributes and tag attributes, at least one of said text attributes being convertible into both a voice mark-up language and a visual mark-up language, and providing information which can be used in both said voice mark-up language and said visual mark-up language.

36. A document as in claim 35, wherein said voice mark-up language is VoiceXML.

37. A document as in claim 35, wherein said visual mark-up language is in a WAP compatible language.

38. A document as in claim 35, further comprising at least one tag enabling switching between format of contents.

39. A document as in claim 35, further comprising at least one tag enabling listening to contents during a visual browsing.

40. A document as in claim 35, further comprising at least one tag enabling showing of visual contents during a voice browsing session.

41. A document as in claim 35, further comprising tags which delineate different sections, wherein each section is separately delineated as being one or both of voice browsing or visual browsing.

42. A method, comprising:

providing first portion of a document on a portable phone in a voice mark-up language; and

providing a second portion of the same document on the portable phone, during the same browsing session, in a visual mark-up language.

43. A method as in claim 42, further comprising accepting a request from the portable phone to switch a mark-up language which is being displayed, and switching the mark-up language between visual mark-up language and speech mark-up language.

44. A method as in claim 42, further comprising accepting a request from the portable phone to display additional language, and displaying both speech mark-up language and visual mark-up language at the same time.

45. A method, comprising:

allowing a user to browse a document in either of voice XML or compact visual XML;

keeping track of the position of browsing in a current language; and

allowing switching from a current language to another language at a position related to said position of browsing.