Retrieval and presentation of network service results for mobile device using a multimodal browser

• US 8,073,700 B2
• Filed: 06/05/2006
• Issued: 12/06/2011
• Est. Priority Date: 09/12/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Jaramillo, David, White, Marc
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US11/422,093
• Publication Number:
  US 20070061146A1
• Time in Patent Office: 2,010 Days
• Field of Search: 704/270.1
• US Class Current: 704/270.1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04M 2201/39 using speech synthesis speech synthesis per se G10L13/00
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 7/0036 where the data service is an information service

Abstract

A method of obtaining information using a mobile device can include receiving a request including speech data from the mobile device, and querying a network service using query information extracted from the speech data, whereby search results are received from the network service. The search results can be formatted for presentation on a display of the mobile device. The search results further can be sent, along with a voice grammar generated from the search results, to the mobile device. The mobile device then can render the search results.

Claims

1. A method carried out by at least one computer, the method comprising acts of:

receiving a request comprising speech data from a mobile device;

querying a network service using query information obtained from the speech data, whereby search results are received from the network service;

formatting the search results for presentation on a display of the mobile device;

generating a voice grammar based at least in part on the search results; and

sending the search results and the voice grammar generated from the search results to the mobile device.

2. The method of claim 1, further comprising acts of:

selecting a term from the speech data; and

dynamically building a query and including the term selected from the speech data within the query, wherein the act of querying further comprises sending the query to the network service.

3. The method of claim 2, wherein the speech data comprises digitized audio, the method further comprising an act of converting the speech data into a textual representation of the speech data.

4. The method of claim 1, wherein the act of sending further comprises resending, to the mobile device, a different voice grammar from a prior transaction with the mobile device, with the search results and the voice grammar.

5. The method of claim 1, wherein the act of generating the voice grammar comprises selecting a term from the search results and dynamically generating the voice grammar to include the term selected from the search results, and wherein formatting the search results comprises creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device, wherein the voice grammar is part of the multi-modal markup language document.

6. The method of claim 5, further comprising acts of:

receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;

building a subsequent query using the term specified in the speech data of the subsequent request; and

sending the subsequent query to the network service.

7. The method of claim 2, wherein the speech data comprises a textual representation of user speech captured by the mobile device.

8. The method of claim 5, wherein, upon execution of the multi-modal markup language document by the mobile device, a user utterance comprising the term selected from the search results causes the mobile device to initiate a request for further information regarding the term selected from the search results.

9. The method of claim 1, wherein the at least one computer comprises a proxy server.

10. The method of claim 1, wherein the search results represent a subset of information queried

11. An article of manufacture, comprising:

at least one computer-readable non-transitory storage medium having encoded thereon instructions which, when executed by at least one processor, perform a method comprising acts of;

receiving a request comprising speech data from a mobile device;

querying a network service using query information obtained from the speech data, whereby search results are received from the network service;

formatting the search results for presentation on a display of the mobile device;

generating a voice grammar based at least in part on the search results; and

sending the search results and the voice grammar generated from the search results to the mobile device.

12. The article of manufacture of claim 11, wherein the method further comprises acts of:

selecting a term from the speech data; and

dynamically building a query and including the term selected from the speech data within the query, wherein the act of querying further comprises sending the query to the network service.

13. The article of manufacture of claim 12, wherein the speech data comprises digitized audio, and wherein the method further comprises an act of converting the speech data into a textual representation of the speech data.

14. The article of manufacture of claim 12, whrein the speech data comprises a textual respresentation of user speech captured by the mobile device.

15. The article of manufacture of claim 11, wherein the act of sending further comprises resending, to the mobile device, a different voice grammar from a prior transaction with the mobile device, with the search results and the voice grammar.

16. The article of manufacture of claim 11, wherein the act of generating the voice grammar comprises selecting a term from the search results and dynamically generating the voice grammar to include the term selected from the search results, and wherein formatting the search results comprises creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device, wherein the voice grammar is part of the multi-modal markup language document.

17. The article of manufacture of claim 16, wherein the method further comprises acts of:

receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;

building a subsequent query using the term specified in the speech data of the subsequent request; and

sending the subsequent query to the network service.

18. The article of manufacture of claim 16, wherein, upon execution of the multi-modal markup language document by the mobile device, a user utterance comprising the term selected from the search results causes the mobile device to initiate a request for further information regarding the term selected from the search results.

19. The article of manufacture of claim 11, wherein the search results represent a subset of information queried when querying the network service.

20. A system comprising at least one proxy server programmed to:

receive a request comprising speech data from a mobile device;

query a network service using query information obtained from the speech data, whereby search results are received from the network service;

format the search results for presentation on a display of the mobile device;

generate a voice grammar based at least in part on the search results; and

send the search results and the voice grammar generated from the search results to the mobile device.

21. The system of claim 20, wherein the at least one proxy server is further programmed to:

select a term from the speech data;

dynamically build a query and include the term selected from the speech data within the query; and

send the query to the network service.

22. The system of claim 21, wherein the speech data comprises digitized audio, and wherein the at least one proxy server is further programmed to convert the speech data into a textual representation of the speech data.

23. The system of claim 21, wherein the speech data comprises a textual representation of user speech captured by the mobile device.

24. The system of claim 20, wherein the at least one proxy server is further programmed to send to the mobile device a different voice grammar from a prior transaction with the mobile device.

25. The system of claim 20, wherein the at least one proxy server is further programmed to generate the voice grammar at least in part by selecting a term from the search results and dynamically generating the voice grammar to include the term selected from the search results, and wherein the at least one proxy server is further programmed to format the search results at least in part by creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device, wherein the voice grammar is part of the multi-modal markup language document.

26. The system of claim 25, wherein the at least one proxy server is further programmed to:

receive a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;

build a subsequent query using the term specified in the speech data of the subsequent request; and

send the subsequent query to the network service.

27. The system of claim 25, wherein, upon execution of the multi-modal markup language document by the mobile device, a user utterance comprising the term selected from the search results causes the mobile device to initiate a request for further information regarding the term selected from the search results.

28. The system of claim 20, wherein the search results represent a subset of information queried when querying the network service.