Retrieval and Presentation of Network Service Results for Mobile Device Using a Multimodal Browser

• US 20070061146A1
• Filed: 06/05/2006
• Published: 03/15/2007
• Est. Priority Date: 09/12/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: White, Marc, Jaramillo, David
• Granted Patent: US 8,073,700 B2
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04M 2201/39 using speech synthesis speech synthesis per se G10L13/00
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 7/0036 where the data service is an information service

Abstract

A method of obtaining information using a mobile device can include receiving a request including speech data from the mobile device, and querying a network service using query information extracted from the speech data, whereby search results are received from the network service. The search results can be formatted for presentation on a display of the mobile device. The search results further can be sent, along with a voice grammar generated from the search results, to the mobile device. The mobile device then can render the search results.

Claims

1. A method of obtaining information using a mobile device comprising:

receiving a request comprising speech data from the mobile device;

querying a network service using query information extracted from the speech data, whereby search results are received from the network service;

formatting the search results for presentation on a display of the mobile device; and

sending the search results and a voice grammar generated from the search results to the mobile device.

2. The method of claim 1, further comprising:

extracting a term from the speech data; and

dynamically building a query and including the term extracted from the speech data within the query, wherein querying further comprises sending the query to the network service.

3. The method of claim 2, wherein the speech data is digitized audio, the method further comprising first converting the speech data into a textual representation of the speech data.

4. The method of claim 1, wherein sending further comprises resending, to the mobile device, a different voice grammar from a prior transaction with the mobile device, with the search results and the voice grammar.

5. The method of claim 1, wherein formatting the search results further comprises:

selecting a term from the search results;

creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device; and

dynamically generating the voice grammar and including the term from the search results within the voice grammar, wherein the voice grammar is part of the multi-modal markup language document.

6. The method of claim 5, further comprising:

receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;

building a subsequent query using the term specified in the speech data of the subsequent request; and

sending the subsequent query to the network service.

7. A method of obtaining information using a mobile device comprising:

receiving a request comprising speech data from the mobile device;

dynamically building a query using the speech data;

sending the query to a network service and, in response, receiving search results from the network service;

dynamically creating a voice grammar from the search results; and

creating a multi-modal markup language document comprising the search results and the voice grammar and sending the multi-modal markup language document to the mobile device.

8. The method of claim 7, wherein the speech data is a textual representation of user speech, the method further comprising:

selecting at least one term from the speech data; and

including the selected term within the query.

9. The method of claim 8, wherein the speech data is digitized audio, the method further comprising first speech recognizing the speech data.

10. The method of claim 7, further comprising including within the multi-modal markup language document at least one different voice grammar that was previously sent to the mobile device in a prior transaction.

11. The method of claim 7, further comprising:

selecting at least one term from the search results;

formatting the term within the multi-modal markup language document for presentation upon a display of the mobile device; and

including the selected term within the voice grammar of the multi-modal markup language document.

12. The method of claim 11, wherein, upon execution of the multi-modal markup language document within the mobile device, a detected user spoken utterance specifying the term from the voice grammar initiates a request for further information regarding the spoken term.

13. The method of claim 12, further comprising:

receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;

building a subsequent query using the term specified in the speech data of the subsequent request; and

sending the subsequent query to the network service.

14. A machine readable storage having stored thereon a computer program having a plurality of code sections comprising:

code for receiving a request comprising speech data from a mobile device;

code for dynamically building a query using the speech data;

code for sending the query to a network service and, in response, receiving search results from the network service;

code for dynamically creating a voice grammar from the search results; and

code for creating a multi-modal markup language document comprising the search results and the voice grammar and sending the multi-modal markup language document to the mobile device.

15. The machine readable storage of claim 14, wherein the speech data is a textual representation of user speech, the machine readable storage further comprising:

code for selecting at least one term from the speech data; and

code for including the selected term within the query.

16. The machine readable storage of claim 15, wherein the speech data is digitized audio, the machine readable storage further comprising code for first speech recognizing the speech data.

17. The machine readable storage of claim 14, further comprising code for including within the multi-modal markup language document at least one different voice grammar that was previously sent to the mobile device in a prior transaction.

18. The machine readable storage of claim 14, further comprising:

code for selecting at least one term from the search results;

code for formatting the term within the multi-modal markup language document for presentation upon a display of the mobile device; and

code for including the selected term within the voice grammar of the multi-modal markup language document.

19. The machine readable storage of claim 18, wherein, upon execution of the multi-modal markup language document within the mobile device, a detected user spoken utterance specifying the term from the voice grammar initiates a request for further information regarding the spoken term.

20. The machine readable storage of claim 19, further comprising:

code for receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;

code for building a subsequent query using the term specified in the speech data of the subsequent request; and

code for sending the subsequent query to the network service.