System and method for voice activating web pages

• US 9,202,467 B2
• Filed: 06/07/2004
• Issued: 12/01/2015
• Est. Priority Date: 06/06/2003
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Charney, Michael L., Starren, Justin
• Primary Examiner(s): SPOONER, LAMONT M
• Application Number:
  US10/863,577
• Publication Number:
  US 20050143975A1
• Time in Patent Office: 4,194 Days
• Field of Search: 704/270, 704/271, 704/275, 715728-802, 345/1.1, 345/1.3, 709/246
• US Class Current: 1/1
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method for processing a voice request for data specifying a web page, the request including a rule-based grammar statement, the method comprising identifying a grammar associated with the rule-based grammar statement, determining whether a first connection identified as being associated with the grammar is specified in a data structure identifying one or more connections, and processing the request in at least a portion of a window capable of presenting the web page and associated with the first connection if a first connection identified as being associated with the grammar is specified in the data structure.

Claims

1. A method for processing a voice request for data specifying a web page, the request including at least a portion of, and invoking, a rule-based grammar statement, the method comprising:

responsive to the voice request;

identifying which one of a plurality of grammars is associated with the rule-based grammar statement;

determining whether a first connection identified as being associated with the grammar is specified in a data structure identifying a plurality of connections and, for each of the plurality of connections, a respective grammar and a respective window in which a web page is presentable, wherein each of the identified grammars is processable by a processor to interpret a received voice request; and

if the first connection is specified in the data structure, interpreting the request using the identified grammar, to request the specified webpage and display the specified webpage in at least a portion of the window associated with the first connection.

2. The method of claim 1, wherein the rule-based grammar statement includes a phrase portion, a command portion, and a tag portion.

3. The method of claim 1, the method further comprising processing the request in at least a portion of a new window capable of presenting the specified web page and associated with a second connection if the first connection identified as being associated with the grammar is not specified in the data structure.

4. The method of claim 3, the method further comprising adding the second connection to the data structure and associating the second connection with the grammar if the second connection is not specified in the data structure.

5. The method of claim 3, the method further comprising associating the new window with a connection if the data structure does not specify any connection associated with the grammar.

6. The method of claim 5, wherein the new window comprises a window selected according to a user preference.

7. The method of claim 5, wherein the new window comprises a last window loaded by a computing device associated with the second connection.

8. The method of claim 5, wherein the new window comprises an active window at a computing device associated with the second connection.

9. The method of claim 1, wherein identifying a grammar comprises identifying a grammar according to a user preference.

10. The method of claim 1, wherein identifying a grammar comprises identifying a grammar associated with a last window loaded by a computing device associated with a connection.

11. The method of claim 1, wherein identifying a grammar comprises identifying a grammar associated with an active window at a computing device associated with a connection.

12. The method of claim 1, wherein the window comprises a browser window.

13. The method of claim 1, wherein at least a portion of the window comprises a frame.

14. The method of claim 1, wherein the request comprises a request for complex data.

15. The method of claim 14, wherein the request comprises a request for complex data embedded in a web page.

16. The method of claim 14, the method further comprising examining a header or a footer associated with the request to determine a type of data requested.

17. The method of claim 16, wherein examining a header or a footer comprises examining a header or a footer specifying a mime type to determine a type of data requested.

18. The method of claim 14, the method further comprising examining a filename or a file extension associated with the request to determine a type of data requested.

19. The method of claim 1, the method further comprising removing identification of a connection from the data structure if a window associated with that connection is closed.

20. The method of claim 1, wherein each of the plurality of grammars is associable with a respective plurality of ruled-based grammar statements.

21. The method of claim 1, wherein, for each of the grammars specified in the data structure, a respective page mark-up is received, the page mark-up including both first components parsed into a respective visual web page and second components parsed into the respective grammar and not contributing to generation of the respective visual web page, the respective grammar being generated irrespective of content of the first components and being loaded into memory in association with the respective visual web page.

22. The method of claim 1, wherein the request for the specified webpage is made to a web server.

23. A system for processing a voice request for data specifying a web page, the request including at least a portion of, and invoking, a rule-based grammar statement, the system comprising:

means for, responsive to the voice request;

identifying which one of a plurality of grammars is associated with the rule-based grammar statement;

determining whether a first connection identified as being associated with the grammar is specified in a data structure identifying a plurality of connections and, for each of the plurality of connections, a respective grammar and a respective window in which a web page is presentable, wherein each of the identified grammars is processable by a processor to interpret a received voice request; and

if the first connection is specified in the data structure, interpreting the request using the identified grammar, to request the specified webpage and display the specified webpage in at least a portion of the window associated with the first connection.

24. The system of claim 23, wherein the rule-based grammar statement comprises a phrase portion, a command portion, and a tag portion.

25. The system of claim 23, the system further comprising means for processing the request in at least a portion of a new window capable of presenting the specified web page and associated with a second connection if the first connection identified as being associated with the grammar is not specified in the data structure.

26. The system of claim 25, the system further comprising means for adding the second connection to the data structure and associating the second connection with the grammar if the second connection is not specified in the data structure.

27. The system of claim 25, the system further comprising means for associating the new window with the connection if the connection is not specified in the data structure.

28. The system of claim 23, wherein the request for data specifying a web page comprises a request for complex data.

29. The system of claim 23, wherein the request for the specified webpage is made to a web server.

30. A non-transitory computer usable medium or media storing program code which, when executed on one or more computerized devices, causes the one or more computerized devices to execute a method for processing a voice request for data specifying a web page, the request including at least a portion of, and invoking, a rule-based grammar statement, the method comprising:

responsive to the voice request;

identifying which one of a plurality of grammars is associated with the rule-based grammar statement;

determining whether a first connection identified as being associated with the grammar is specified in a data structure identifying a plurality of connections and, for each of the plurality of connections, a respective grammar and a respective window in which a web page is presentable, wherein each of the identified grammars is processable by a processor to interpret a received voice request; and

if the first connection is specified in the data structure, interpreting the request using the identified grammar, to request the specified webpage display the specified webpage in at least a portion of the window associated with the first connection.

31. The non-transitory computer usable medium or media of claim 30, wherein the rule-based grammar statement comprises a phrase portion, a command portion, and a tag portion.

32. The non-transitory computer usable medium or media of claim 30, the method further comprising processing the request in at least a portion of a new window capable of presenting the specified web page and associated with a second connection if the first connection identified as being associated with the grammar is not specified in the data structure.

33. The non-transitory computer usable medium or media of claim 32, the method further comprising adding the second connection to the data structure and associating the second connection with the grammar if the second connection is not specified in the data structure.

34. The non-transitory computer usable medium or media of claim 32, the method further comprising associating the new window with the connection if the connection is not specified in the data structure.

35. The non-transitory computer usable medium or media of claim 30, wherein the request for data specifying a web page comprises a request for complex data.

36. The non-transitory computer usable medium or media of claim 30, wherein the request for the specified webpage is made to a web server.

37. A method for processing a voice request for data specifying a first web page and invoking a rule-based grammar statement, the voice request including a phrase, the method comprising:

prior to receiving the voice request, receiving data including first data specifying a second web page for display and second data defining the rule-based grammar statement, the first and second data being associated with a connection between a speech-related data processing module and a web browser;

responsive to the voice request, determining whether the connection is specified in a data structure identifying one or more connections;

selecting a grammar associated with the connection from a data structure identifying one or more grammars; and

processing the rule-based grammar statement using the selected grammar in at least a portion of a first window capable of presenting the first web page and associated with the connection if the connection is specified in the data structure identifying one or more connections, the processing including, in response to the phrase, executing a command (a) for navigating from the second web page to the specified first web page and (b) referenced by the rule-based grammar statement, text of the phrase being different than text of each of all visual links in the second web page.

38. The method of claim 37, the method further comprising selecting a new grammar associated with the connection from a data structure specifying one or more grammars and processing the rule-based grammar statement using the selected grammar in at least a portion of a new window associated with the connection if the connection is not specified in the data structure specifying one or more connections.

39. The method of claim 38, the method further comprising adding the connection to the data structure specifying one or more connections and associating the connection with the new grammar in the data structure specifying one or more grammars.

40. The method of claim 37, wherein:

the received data defines the rule-based grammar statement as including a phrase portion, a command portion, and a tag portion;

the phrase of the voice request corresponds to at least a portion of the phrase portion; and

the command portion references the executed command.

41. The method of claim 37, wherein the second web page does not include any visual link to the first web page.

42. A system for processing a voice request for data specifying a web page, the voice request including at least a portion of, and invoking, a rule-based grammar statement, the system comprising at least one processor configured to process the voice request by:

identifying a grammar associated with the rule-based grammar statement;

determining whether a connection is associated with the identified grammar;

if the grammar is determined to be a connection associated grammar;

accessing a data structure identifying a plurality of connections to find a first connection which the data structure specifies is associated with the grammar, wherein the data structure identifies respective grammars associated with the plurality of connections, and each of the identified grammars is processable by a processor to interpret a received voice request; and

interpreting the voice request using the grammar identified as being associated with the rule-based grammar statement to request the web page and present the web page in at least a portion of a window capable of presenting the web page and associated with the first connection; and

otherwise interpreting the voice request using the grammar identified as being associated with the rule-based grammar statement to request the web page from a web server and present the web page in at least a portion of a new window capable of presenting the web page and associated with a second connection.

43. The system of claim 42, wherein the processing of the voice request further comprises:

if the grammar is not determined to be a connection associated grammar, adding the second connection to the data structure and associating the second connection with a new grammar.

44. The system of claim 42, wherein the processing of the voice request further comprises, if the grammar is not determined to be a connection associated grammar, modifying the data structure to include an association of the new window or the portion of the new window with the second connection.

45. The system of claim 42, wherein the system:

receives a page mark-up that includes both first components parsed into a visual web page and second components parsed into the identified grammar and not contributing to generation of the visual web page, the grammar being generated irrespective of content of the first components; and

loads the grammar into memory in association with the visual web page.

46. The system of claim 42, wherein the request for the specified webpage is made to a web server.

47. A computer-implemented method for providing web pages, comprising:

receiving a first data, the first data including a first web page and a first grammar based on which a voice command is parseable, wherein the first grammar is associated with the first web page;

responsive to the receipt of the first data;

displaying in a first display window at least a portion of the first web page;

storing a reference to the first grammar in a data structure that associates the first grammar with the first display window;

responsive to receipt of second data associated with a second web page;

displaying in a second display window at least a portion of the second web page in accordance with the second data; and

for a second grammar, based on which a voice command is parseable, and that is obtained via the receipt of the second data, storing a reference to the second grammar in the data structure, the data structure associating the second grammar with the second display window; and

responsive to an input voice command;

determining with which of the first and second grammars the voice command accords;

parsing the input voice command according to the determined one of the first and second grammars;

accessing the data structure to determine with which of the first and second display windows the determined one of the first and second grammars is associated; and

displaying a new data in the determined one of the first and second display windows.

48. The method of claim 47, wherein the method further comprises:

for each of the first and second display windows, opening a respective one of a first and a second connection with a web browser, wherein the data displayed in the first and second display windows is provided to the web browser for generating the display over the first and second connections, respectively.

49. The method of claim 48, wherein the first and second connections are persistent connections, and at least one of (a) are closed in response to a closing of a respective one of the first and second display windows, and (b) are closed in response to a loading into the respective one of the first and second display windows of a different web page not associated with a grammar.

50. The method of claim 49, wherein the method further comprises:

responsive to the closing, deleting from the data structure the respective one of the first and second grammars.

51. The method of claim 50, wherein the data displayed in the determined one of the first and second display windows is a subset of data received in response to a request for a third web page, the request transmitted in response to the voice command, and the data received in response to the data request includes a third grammar, the method further comprising:

updating the data structure to include a reference to the third grammar in association with the determined one of the first and second display windows.

52. The method of claim 47, wherein each of the first and second grammars defines rule based grammar structures, each of the rule based grammar structures including a phrase portion, a command portion, and a tag portion.

53. The method of claim 47, wherein the method further comprises:

responsive to another voice command;

determining that the another voice command is equally associable with the first and second grammars;

selecting one of the first and second display windows; and

displaying data in the selected one of the first and second display windows.

54. The method of claim 53, wherein the selecting is in accordance with a user input.

55. The method of claim 53, wherein whichever of the first and second display windows was last loaded is selected in the selecting step.

56. The method of claim 53, wherein whichever of the first and second display windows is currently active is selected in the selecting step.

57. The method of claim 47, wherein the method further comprises:

responsive to a command, determining which of (a) a request for a new web page and (b) a request for embedded data of a current web page the command includes;

if it is determined that the command includes the request for the embedded data, transmitting the request to a web server; and

otherwise

determining whether the command is a voice command.

58. The method of claim 57, wherein the determination of which of the request for the new page and the request for the embedded data the command includes is performed by examining one of a header and a footer associated with the command.

59. The method of claim 58, wherein the examining includes examining a specification, in the one of the header and the footer, of a mime type.

60. The method of claim 57, wherein the determination of which of the request for the new page and the request for the embedded data the command includes is performed by examining one of a file name and a file extension associated with the command.

61. The method of claim 47, wherein:

the first web page is a visual web page;

the first data includes a first page-markup that includes both first components parseable into the first visual web page and second components parseable into the first grammar irrespective of content of the first components, the second components being specified by a speech markup language and not used for generating the first visual web page; and

the second data includes a second page-markup that includes both first components parseable into the second visual web page and second components parseable into the second grammar irrespective of content of the first components of the second page-markup, the second components of the second page-markup being specified by a speech markup language and not used for generating the second visual web page.

62. The method of claim 47, wherein, in the accessing step, the determined one of the first and second display windows is not in focus and the other of the first and second display windows is in focus.

63. A method for processing a voice request for data specifying a web page, the request including at least a portion of, and invoking, a rule-based grammar statement, the method comprising:

responsive to the voice request;

identifying which one of a plurality of grammars is associated with the rule-based grammar statement;

determining whether a first connection identified as being associated with the grammar is specified in a data structure identifying a plurality of connections and, for each of the plurality of connections, a respective grammar and a respective window in which a web page is presentable; and

if the first connection is specified in the data structure, processing the request using the identified grammar, to display the specified webpage in at least a portion of the window associated with the first connection;

wherein the request includes a command (a) to be run, responsive to the request, in the window associated with the first connection and (b) that is independent of all commands previously run in any of the windows identified in the data structure.