VOICE-ENABLED DIALOG INTERACTION WITH WEB PAGES

• US 20150149168A1
• Filed: 11/27/2013
• Published: 05/28/2015
• Est. Priority Date: 11/27/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: MELAMED, I. Dan, STENT, Amanda Joy, JUNG, Hyuckchul, NIRAULA, Nobal Bikram
• Granted Patent: US 9,690,854 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Voice enabled dialog with web pages is provided. An Internet address of a web page is received including an area with which a user of a client device can specify information. The web page is loaded using the received Internet address of the web page. A task structure of the web page is then extracted. An abstract representation of the web is then generated. A dialog script, based on the abstract representation of the web page is then provided. Spoken information received from the user is converted into text and the converted text is inserted into the area.

Claims

1. A method of enabling voice interaction with a web page, comprising:

receiving, at a processor, an Internet address of the web page, the web page including an area with which a user of a client device can specify information;

loading the web page using the Internet address;

extracting a task structure of the web page;

generating an abstract representation of the web page; and

providing a dialog script based on the abstract representation to provide voice interaction with the user,

wherein spoken information of the user is converted into text and the text is inserted into the area.

2. The method of claim 1, further comprising:

extracting forms, form constraints, and dependencies from the web page using a browser object model and a document object model.

3. The method of claim 1, further comprising:

determining that the client device does not have an alphabetic input device.

4. The method of claim 1, further comprising:

generating a browser object model of the web page.

5. The method of claim 4, wherein the browser object model comprises a state of hypertext markup language elements as the hypertext markup language elements would appear in a web browser.

6. The method of claim 4, further comprising:

generating a document object model of the web page.

7. The method of claim 6, wherein the document object model comprises a tree of hypertext markup language elements.

8. The method of claim 1, further comprising:

detecting that the user of the client device prefers to specify information in the web page by voice.

9. The method of claim 1, further comprising:

determining that the client device is a portable communications device.

10. The method of claim 9, wherein the determining is based upon a received user agent string in a hypertext transfer protocol address header received with the Internet address.

11. A system for enabling voice interaction with a web page, comprising:

a processor;

a memory that stores executable instructions that, when executed by the processor, cause the processor to perform operations comprising;

receiving an Internet address of the web page, the web page including an area with which a user of a client device can specify information;

loading the web page using the Internet address;

extracting a task structure of the web page;

generating an abstract representation of the web page; and

providing a dialog script based on the abstract representation to provide voice interaction with the user, and

wherein spoken information of the user is converted into text and the text is inserted into the area.

12. A tangible computer readable storage medium encoded with an executable computer program that enables voice interaction with a web page and that, when executed by a processor, causes the processor to perform operations comprising:

receiving an Internet address of the web page, the web page including an area with which a user of a client device can specify information;

loading the web page using the Internet address;

extracting a task structure of the web page;

generating an abstract representation of the web page; and

providing a dialog script based on the abstract representation to provide voice interaction with the user,

wherein spoken information of the user is converted into text and the text is inserted into the area.

13. The tangible computer readable storage medium of claim 12, wherein the operations further comprise:

extracting forms, form constraints, and dependencies from the web page using a browser object model and a document object model.

14. The tangible computer readable storage medium of claim 12, wherein the operations further comprise:

determining that the client device does not have an alphabetic input device.

15. The tangible computer readable storage medium of claim 12, wherein the operations further comprise:

generating a browser object model of the web page.

16. The tangible computer readable storage medium of claim 15, wherein the browser object model comprises a state of hypertext markup language elements as the hypertext markup language elements would appear in a web browser.

17. The tangible computer readable storage medium of claim 15, wherein the operations further comprise:

generating a document object model of the web page.

18. The tangible computer readable storage medium of claim 12, wherein the operations further comprise:

detecting that the user of the client device prefers to specify information in the web page by voice.

19. The tangible computer readable storage medium of claim 12, wherein the operations further comprise:

determining that the client device is a portable communications device.

20. The tangible computer readable storage medium of claim 12, wherein the determining is based upon a received user agent string in a hypertext transfer protocol address header received with the Internet address.