Retrieval and presentation of network service results for mobile device using a multimodal browser

• US 8,781,840 B2
• Filed: 01/31/2013
• Issued: 07/15/2014
• Est. Priority Date: 09/12/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Jaramillo, David, White, Marc
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US13/756,073
• Publication Number:
  US 20130158994A1
• Time in Patent Office: 530 Days
• Field of Search: 704/270.1
• US Class Current: 704/270.1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04M 2201/39 using speech synthesis speech synthesis per se G10L13/00
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 7/0036 where the data service is an information service

Abstract

A method of obtaining information using a mobile device can include receiving a request including speech data from the mobile device, and querying a network service using query information extracted from the speech data, whereby search results are received from the network service. The search results can be formatted for presentation on a display of the mobile device. The search results further can be sent, along with a voice grammar generated from the search results, to the mobile device. The mobile device then can render the search results.

Claims

1. A method comprising acts of:

sending from a device, via at least one communication medium, a request to obtain information;

receiving at the device, via the at least one communication medium, a response to the request to obtain information, the response comprising one or more search results and a voice grammar dynamically generated based on the one or more search results;

presenting via the device at least some of the one or more search results to a user of the device; and

using the voice grammar dynamically generated based on the one or more search results to process user speech spoken by the user subsequent to the at least some of the one or more search results being presented to the user.

2. The method of claim 1, wherein the request to obtain information is a first request, and the act of using the voice grammar dynamically generated based on the one or more search results to process the user speech comprises determining that the user speech comprises an indication that the user desires to select at least one search result of the one or more search results, and wherein the method further comprises an act of:

in response to determining that the user speech comprises an indication that the user desires to select the at least one search result, sending a second request via the at least one communication medium, the second request requesting information relating to the at least one search result.

3. The method of claim 2, wherein the response to the first request is a first response and the voice grammar dynamically generated based on the one or more search results is a first voice grammar, and wherein the method further comprises acts of:

receiving at the device, via the at least one communication medium, a second response in response to the second request, the second response comprising one or more pieces of information relating to the at least one search result and a second voice grammar dynamically generated based on the one or more pieces of information;

presenting via the device at least some of the one or more pieces of information to the user; and

using the second voice grammar dynamically generated based on the one or more pieces of information to process user speech spoken by the user subsequent to the at least some of the one or more pieces of information being presented to the user.

4. The method of claim 1, wherein the act of presenting the at least some of the one or more search results to the user comprises:

providing an indication to the user that the user is allowed to select at least one search result of the one or more search results by speaking one or more words associated with the at least one search result.

5. The method of claim 4, wherein providing an indication to the user comprises providing a visual indication that distinguishes the one or more words associated with the at least one search result from other words presented to the user.

6. The method of claim 1, wherein the request to obtain information is a subsequent request and the voice grammar dynamically generated based on the one or more search results is a subsequent voice grammar, and wherein the method further comprises acts of:

sending from the device an initial request via the at least one communication medium;

receiving at the device, via the at least one communication medium, an initial response to the initial request, the initial response comprising an initial voice grammar; and

using the initial voice grammar to process user speech spoken by the user.

7. The method of claim 6, further comprising an act of:

generating the subsequent request as a result of using the initial voice grammar to process user speech.

8. A system comprising at least one processor configured to:

send, via at least one communication medium, a request to obtain information;

receive, via the at least one communication medium, a response to the request to obtain information, the response comprising one or more search results and a voice grammar dynamically generated based on the one or more search results;

present via the device at least some of the one or more search results to a user of the device; and

use the voice grammar dynamically generated based on the one or more search results to process user speech spoken by the user subsequent to the at least some of the one or more search results being presented to the user.

9. The system of claim 8, wherein the request to obtain information is a first request, and using the voice grammar dynamically generated based on the one or more search results to process the user speech comprises determining that the user speech comprises an indication that the user desires to select at least one search result of the one or more search results, and wherein the at least one processor is further configured to:

in response to determining that the user speech comprises an indication that the user desires to select the at least one search result, send a second request via the at least one communication medium, the second request requesting information relating to the at least one search result.

10. The system of claim 9, wherein the response to the first request is a first response and the voice grammar dynamically generated based on the one or more search results is a first voice grammar, and wherein the at least one processor is further configured to:

receive, via the at least one communication medium, a second response in response to the second request, the second response comprising one or more pieces of information relating to the at least one search result and a second voice grammar dynamically generated based on the one or more pieces of information;

present at least some of the one or more pieces of information to the user; and

use the second voice grammar dynamically generated based on the one or more pieces of information to process user speech spoken by the user subsequent to the at least some of the one or more pieces of information being presented to the user.

11. The system of claim 8, wherein the at least one processor is further configured to present the at least some of the one or more search results to the user at least in part by:

providing an indication to the user that the user is allowed to select at least one search result of the one or more search results by speaking one or more words associated with the at least one search result.

12. The system of claim 11, wherein providing an indication to the user comprises providing a visual indication that distinguishes the one or more words associated with the at least one search result from other words presented to the user.

13. The system of claim 8, wherein the request to obtain information is a subsequent request and the voice grammar dynamically generated based on the one or more search results is a subsequent voice grammar, and wherein the at least one processor is further configured to:

send an initial request via the at least one communication medium;

receive, via the at least one communication medium, an initial response to the initial request, the initial response comprising an initial voice grammar; and

use the initial voice grammar to process user speech spoken by the user.

14. The system of claim 13, wherein the at least one processor is further configured to:

generate the subsequent request as a result of using the initial voice grammar to process user speech.

15. At least one computer-readable storage device having encoded thereon instructions that, when executed by at least one processor of a device, perform a method comprising acts of:

sending from the device, via at least one communication medium, a request to obtain information;

receiving at the device, via the at least one communication medium, a response to the request to obtain information, the response comprising one or more search results and a voice grammar dynamically generated based on the one or more search results;

presenting via the device at least some of the one or more search results to a user of the device; and

using the voice grammar dynamically generated based on the one or more search results to process user speech spoken by the user subsequent to the at least some of the one or more search results being presented to the user.

16. The at least one computer-readable storage device of claim 15, wherein the request to obtain information is a first request, and the act of using the voice grammar dynamically generated based on the one or more search results to process the user speech comprises determining that the user speech comprises an indication that the user desires to select at least one search result of the one or more search results, and wherein the method further comprises an act of:

in response to determining that the user speech comprises an indication that the user desires to select the at least one search result, sending a second request via the at least one communication medium, the second request requesting information relating to the at least one search result.

17. The at least one computer-readable storage device of claim 16, wherein the response to the first request is a first response and the voice grammar dynamically generated based on the one or more search results is a first voice grammar, and wherein the method further comprises acts of:

receiving at the device, via the at least one communication medium, a second response in response to the second request, the second response comprising one or more pieces of information relating to the at least one search result and a second voice grammar dynamically generated based on the one or more pieces of information;

presenting via the device at least some of the one or more pieces of information to the user; and

using the second voice grammar dynamically generated based on the one or more pieces of information to process user speech spoken by the user subsequent to the at least some of the one or more pieces of information being presented to the user.

18. The at least one computer-readable storage device of claim 15, wherein the act of presenting the at least some of the one or more search results to the user comprises:

providing an indication to the user that the user is allowed to select at least one search result of the one or more search results by speaking one or more words associated with the at least one search result.

19. The at least one computer-readable storage device of claim 18, wherein providing an indication to the user comprises providing a visual indication that distinguishes the one or more words associated with the at least one search result from other words presented to the user.

20. The at least one computer-readable storage device of claim 15, wherein the request to obtain information is a subsequent request and the voice grammar dynamically generated based on the one or more search results is a subsequent voice grammar, and wherein the method further comprises acts of:

sending from the device an initial request via the at least one communication medium;

receiving at the device, via the at least one communication medium, an initial response to the initial request, the initial response comprising an initial voice grammar; and

using the initial voice grammar to process user speech spoken by the user.