Accessing network-based electronic information through scripted online interfaces using spoken input

• US 6,513,063 B1
• Filed: 03/14/2000
• Issued: 01/28/2003
• Est. Priority Date: 01/05/1999
• Status: Expired due to Term

• Current Assignee: IPA Technologies Inc. (Wi-LAN Inc.)
• Original Assignee: SRI International, Inc.
• Inventors: Julia, Luc, Voutsas, Dimitris, Cheyer, Adam
• Primary Examiner(s): WILEY, DAVID ARMAND
• Application Number:
  US09/524,868
• Time in Patent Office: 1,050 Days
• Field of Search: 709/219, 709/227
• US Class Current: 709/219
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 9/465 Distributed object oriented systems remote method invocation [RMI] G06F9/548
• CPC Class Codes: G06N 5/043 Distributed expert systems; Blackboards

Abstract

A system, method, and article of manufacture are provided for navigating an electronic data source that has a scripted online interface by means of spoken input. When a spoken request is received from a user, it is interpreted. A navigation query is constructed based on the interpretation of the speech input and a template extracted by scraping an online scripted interface to the data source. The resulting interpretation of the request is thereupon used to automatically construct an operational navigation query to retrieve the desired information from one or more electronic network data sources, which is then transmitted to a client device of the user.

Claims

1. A method for speech-based navigation of an electronic data source, the electronic data source having a scripted online interface and being located at one or more network servers located remotely from a user, comprising the steps of:

(a) receiving a spoken request for desired information from the user;

(b) rendering an interpretation of the spoken request;

(c) dynamically scraping the online scripted interface to the data source, to extract an input template;

(d) constructing a navigation query based upon the interpretation of step (b) and the template of step (c);

(e) using the navigation query to select a portion of the electronic data source; and

(f) transmitting the selected portion of the electronic data source from the network server to a client device of the user.

2. The method of claim 1, wherein the online scripted interface is a web site.

3. The method of claim 1, wherein the input template is extracted from an online form of the scripted interface.

4. The method of claim 1, wherein the step of rendering an interpretation further includes deriving linguistic information by using a speech recognition engine and a linguistic parser.

5. The method of claim 1, wherein at least one of the steps of rendering an interpretation and constructing a navigation query are performed, at least in part, on a computing device located locally with the user.

6. The method of claim 1, wherein at least one of the steps of rendering an interpretation and constructing a navigation query are performed, at least in part, on a network computing device located remotely from the user.

7. The method of claim 1, further comprising the step of (g) soliciting additional input in response to one or more deficiencies encountered during the step of constructing the navigation query.

8. The method of claim 7, wherein the deficiencies include unresolved words of the spoken request.

9. The method of claim 7, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken request.

10. The method of claim 7, wherein the step of soliciting additional input is performed in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query constructed in step (d).

11. The method of claim 10, wherein the deficiencies include existence of more than one data record within the data source responsive to the navigation query.

12. The method of claim 10, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.

13. The method of claim 7, further comprising the steps of:

(h) receiving the additional input from the user;

(i) refining the navigational query based on the additional input;

(j) determining whether the navigational query is deemed adequate;

(k) soliciting additional input if the navigational query is not deemed adequate; and

repeating steps (h)-(j) until the navigational query is deemed adequate.

14. The method of claim 7, wherein a determination that deficiencies exist in the navigational query is at least partially based on a user-input statement that additional information is required.

15. The method of claim 7, wherein the additional input is solicited by presenting a menu to the user on the client device of the user.

16. The method of claim 7, wherein the additional input is solicited by presenting a textual request for the additional input.

17. The method of claim 7, wherein the additional input is solicited by outputting an audible request for the additional input.

18. The method of claim 7, wherein the additional input is solicited by presenting a list of portions of the electronic data source that match the navigational query.

19. The method of claim 7, wherein the additional input received from the user is at least partially speech based.

20. The method of claim 7, wherein the additional input received from the user includes no spoken input.

21. The method of claim 1, wherein the method is performed with respect to a plurality of simultaneous users and corresponding client devices.

22. The method of claim 1, further including the step of selecting the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken request.

23. The method of claim 1, wherein at least a portion of a communications link between the client device and the data source is the Internet.

24. A system for speech-based navigation of an electronic data source, the electronic data source having a scripted online interface and being located at one or more network servers located remotely from a user, the system comprising:

(a) a portable microphone operable to receive a spoken request for desired information from the user;

(b) language processing logic, operable to render an interpretation of the spoken request;

(c) template extraction logic, operable to dynamically scrape the online scripted interface to the data source, to extract an input template;

(d) query construction logic, operable to construct a navigation query based upon the interpretation rendered by the language processing logic and the template extracted by the template extraction logic;

(e) navigation logic, operable to select a portion of the electronic data source using the navigation query; and

(f) electronic communications infrastructure for transmitting the selected portion of the electronic data source from the network server to a client device of the user.

25. The system of claim 24, wherein the online scripted interface is a web site.

26. The system of claim 24, wherein the input template is extracted from an online form of the scripted interface.

27. The system of claim 24, wherein the language processing logic includes speech recognition logic and linguistic parsing logic for deriving linguistic information.

28. The system of claim 24, wherein at least a portion of the language processing logic is hosted on a computing device located locally with the user.

29. The system of claim 24, further comprising user interaction logic for soliciting additional input in response to one or more deficiencies encountered during the construction of the navigation query.

30. The system of claim 29, wherein the deficiencies include unresolved words of the spoken request.

31. The system of claim 29, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken request.

32. The system of claim 29, wherein the user interaction logic solicits additional input in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query constructed by the query construction logic.

33. The system of claim 32, wherein the deficiencies include existence of more than one data record within the data source responsive to the navigation query.

34. The system of claim 32, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.

35. The system of claim 24 wherein the navigation logic selects the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken request.

36. The system of claim 24, wherein at least a portion of the electronic communications infrastructure is the Internet.

37. A computer program embodied on a computer readable medium for speech-based navigation of an electronic data source, the electronic data source having a scripted online interface and being located at one or more network servers located remotely from a user, comprising:

(a) a code segment that receives a spoken request for desired information from the user;

(b) a code segment that renders an interpretation of the spoken request;

(c) a code segment that dynamically scrapes the online scripted interface to the data source, to extract an input template;

(d) a code segment that constructing a navigation query based upon the interpretation of step (b) and the template of step (c);

(e) a code segment that uses the navigation query to select a portion of the electronic data source; and

(f) a code segment that transmits the selected portion of the electronic data source from the network server to a client device of the user.

38. The computer program of claim 37, wherein the online scripted interface is a web site.

39. The computer program of claim 37, wherein the input template is extracted from an online form of the scripted interface.

40. The computer program of claim 37, further comprising a code segment that derives linguistic information by using a speech recognition engine and a linguistic parser.

41. The computer program of claim 37, wherein the rendering of the interpretation and construction of the navigation query are performed, at least in part, on a computing device located locally with the user.

42. The computer program of claim 37, wherein the rendering of the interpretation and construction of the navigation query are performed, at least in part, on a network computing device located remotely from the user.

43. The computer program of claim 37, further comprising (g) a code segment that solicits additional input in response to one or more deficiencies encountered during construction of the navigation query.

44. The computer program of claim 43, wherein the deficiencies include unresolved words of the spoken request.

45. The computer program of claim 43, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken request.

46. The computer program of claim 43, wherein the code segment that solicits additional input solicits the additional input in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query.

47. The computer program of claim 46, wherein the deficiencies include existence of more than one data record within the data source responsive to the navigation query.

48. The computer program of claim 46, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.

49. The computer program of claim 43, further comprising the steps of:

(h) a code segment that receives the additional input from the user

(i) a code segment that refines the navigational query based on the additional input;

(j) a code segment that determines whether the navigational query is deemed adequate;

(k) a code segment that solicits additional input if the navigational query is not deemed adequate and repeating steps (h)-(j) until the navigational query is deemed adequate.

50. The computer program of claim 43, wherein a determination that deficiencies exist in the navigational query is at least partially based on a user-input statement that additional information is required.

51. The computer program of claim 43, wherein the additional input is solicited by presenting a menu to the user on the dient device of the user.

52. The computer program of claim 43, wherein the additional input is solicited by presenting a textual request for the additional input.

53. The computer program of claim 43, wherein the additional input is solicited by outputting an audible request for the additional input.

54. The computer program of claim 43, wherein the additional input is solicited by presenting a list of portions of the electronic data source that match the navigational query.

55. The computer program of claim 43, wherein the additional input received from the user is at least partially speech based.

56. The computer program of claim 43, wherein the additional input received from the user includes no spoken input.

57. The computer program of claim 37, wherein the code segments of the computer program operate with respect to a plurality of simultaneous users and corresponding client devices.

58. The computer program of claim 37, further including a code segment that selects the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken request.

59. The computer program of claim 37, wherein at least a portion of a communications link between the client device and the data source is the Internet.

60. A method for utilizing spoken natural language for navigating an electronic data source, the electronic data source having a scripted online interface and being located at one or more network servers located remotely from a user, comprising the steps of:

(a) receiving a spoken natural language (“

NL”

) request for desired information from the user;

(b) rendering an interpretation of the spoken request;

(c) dynamically scraping the online scripted interface to the data source, to extract an input template;

(d) constructing a navigation query based upon the interpretation of step (b) and the template of step (c);

(e) using the navigation query to select a portion of the electronic data source; and

(f) transmitting the selected portion of the electronic data source from the network server to a client device of the user.

61. The method of claim 60, wherein the online scripted interface is a web site.

62. The method of claim 60, wherein the input template is extracted from an online form of the scripted interface.

63. The method of claim 60, wherein the step of rendering an interpretation further includes deriving linguistic information by using a speech recognition engine and an NL parser.

64. The method of claim 60, wherein at least one of the steps of rendering an interpretation and constructing a navigation query are performed, at least in part, on a computing device located locally with the user.

65. The method of claim 60, wherein at least one of the steps of rendering an interpretation and constructing a navigation query are performed, at least in part, on a network computing device located remotely from the user.

66. The method of claim 60, further comprising the step of soliciting additional input in response to one or more deficiencies encountered during the step of constructing the navigation query.

67. The method of claim 66, wherein the deficiencies include unresolved words of the spoken NL request.

68. The method of claim 66, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken NL request.

69. The method of claim 66, wherein the step of soliciting additional input is performed in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query constructed in step (d).

70. The method of claim 69, wherein the deficiencies include existence of more than one data record within the data source responsive to the navigation query.

71. The method of claim 69, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.

72. The method of claim 60, wherein the method is performed with respect to a plurality of simultaneous users and corresponding client devices.

73. The method of claim 60, further including the step of selecting the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken NL request.

74. The method of claim 60, wherein at least a portion of a communications link between the client device and the data source is the Internet.

75. A system for utilizing spoken natural language for navigating an electronic data source, the electronic data source having a scripted online interface and being located at one or more network servers located remotely from a user, the system comprising:

(a) portable microphone operable to receive a spoken natural language (“

NL”

) request for desired information from the user;

(b) spoken language processing logic, operable to render an interpretation of the spoken natural language request;

(c) template extraction logic, operable to dynamically scrape the online scripted interface to the data source, to extract an input template;

(d) query construction logic, operable to construct a navigation query based upon the interpretation rendered by the spoken language processing logic and the template extracted by the template extraction logic;

(e) navigation logic, operable to select a portion of the electronic data source using the navigation query; and

(f) electronic communications infrastructure for transmitting the selected portion of the electronic data source from the network server to a client device of the user.

76. The system of claim 75, wherein the online scripted interface is a web site.

77. The system of claim 75, wherein the input template is extracted from an online form of the scripted interface.

78. The system of claim 75, wherein the spoken language processing logic includes speech recognition logic and an NL parsing logic for deriving linguistic information.

79. The system of claim 75, wherein at least a portion of the spoken language processing logic is hosted on a computing device located locally with the user.

80. The system of claim 75, further comprising user interaction logic for soliciting additional input in response to one or more deficiencies encountered during the construction of the navigation query.

81. The system of claim 80, wherein the deficiencies include unresolved words of the spoken NL request.

82. The system of claim 80, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken NL request.

83. The system of claim 80, wherein the user interaction logic solicits additional input in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query constructed by the query construction logic.

84. The system of claim 83, wherein the deficiencies include existence of more than one data record within the data source responsive to the navigation query.

85. The system of claim 83, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.

86. The system of claim 75, wherein the navigation logic selects the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken NL request.

87. The system of claim 75, wherein at least a portion of the electronic communications infrastructure is the Internet.

88. A computer program embodied on a computer readable medium for utilizing spoken natural language for navigating an electronic data source, the electronic data source having a scripted online interface and being located at one or more network servers located remotely from a user, comprising:

(a) a code segment that receives a spoken natural language (“

NL”

) request for desired information from the user;

(b) a code segment that renders an interpretation of the spoken natural language request;

(c) a code segment that dynamically scrapes the online scripted interface to the data source, to extract an input template;

(d) a code segment that constructs a navigation query based upon the interpretation of step (b) and the template of step (c);

(e) a code segment that uses the navigation query to select a portion of the electronic data source; and

(f) a code segment that transmits the selected portion of the electronic data source from the network server to a client device of the user.

89. The computer program of claim 88, wherein the online scripted interface is a web site.

90. The computer program of claim 88, wherein the input template is extracted from an online form of the scripted interface.

91. The computer program of claim 88, further comprising a code segment that derives linguistic information by using a speech recognition engine and an NL parser.

92. The computer program of claim 88, wherein the rendering of the interpretation and construction of the navigation query are performed, at least in part, on a computing device located locally with the user.

93. The computer program of claim 88, wherein the rendering of the interpretation and construction of the navigation query are performed, at least in part, on a network computing device located remotely from the user.

94. The computer program of claim 88, further comprising a code segment that solicits additional input in response to one or more deficiencies encountered during construction of the navigation query.

95. The computer program of claim 94, wherein the deficiencies include unresolved words of the spoken NL request.

96. The computer program of claim 94, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken NL request.

97. The computer program of claim 94, wherein the code segment that solicits additional input solicits the additional input in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query.

98. The computer program of claim 97, wherein the deficiencies include existence of more than one data record within the data source responsive to the navigation query.

99. The computer program of claim 97, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.

100. The computer program of claim 88, wherein the code segments of the computer program operate with respect to a plurality of simultaneous users and corresponding client devices.

101. The computer program of claim 88, further including a code segment that selects the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken NL request.

102. The computer program of claim 88, wherein at least a portion of a communications link between the client device and the data source is the Internet.