Semantic user interface

• US 6,438,545 B1
• Filed: 04/15/1999
• Issued: 08/20/2002
• Est. Priority Date: 07/03/1997
• Status: Expired due to Term

• Current Assignee: Activeword Systems, Inc.
• Original Assignee: Value Capital Management
• Inventors: Armijo-Tamez, Jesus Roberto, Beauregard, Serge Pierre
• Primary Examiner(s): Alam, Hosain T.
• Assistant Examiner(s): ALAM, SHAHID AL
• Application Number:
  US09/292,008
• Time in Patent Office: 1,223 Days
• Field of Search: 707/3, 707/5, 707/6, 707/10, 707/100, 707/103, 704/275, 704/270, 704/251, 704/252, 345/330, 345/347, 345/326, 345/327, 709/315, 709/316, 709/107, 717/11
• US Class Current: 1/1
• CPC Class Codes: G06F 9/45512 Command shells
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A system and method that allows a user to use their everyday language or user defined words to operate a computer in a highly efficient way. In short, every word, letter, control character and symbol is potentially actionable. A computer user'"'"'s productivity is dramatically increased by making available those functions that enable a user to produce most of his work through simple, language-based commands. The present invention provides an intuitive interface, referred to as a semantic user interface (SUI), that enhances the operation of the current standard window-based interface in a manner that is simple, richer and natural. By leveraging all of the richness and power inherent in a user'"'"'s language, the present invention provides an important tool that allows the personal computer to operate in a manner that is much closer to our natural way of interacting. A user is allowed to enter “commands” in his everyday natural language in order to control the operations of the computer. All commands are language-based and user-defined. These commands can be entered from any context of the user'"'"'s computer (e.g., any application or operating system workspace). The commands allows a user to launch applications and navigate within applications by using language rather than clicks from a pointing device such as a mouse. It also allows the replacement of keystrokes with stored words or keystrokes. The system also keeps a complete archive record of all the text content the user provides as input, regardless of which application program or operating system window the user is operating in at the time. The combined set of all user defined commands and the memory of all the input text that is stored in the archive constitutes the personality profile and is transportable from one computer to another.

Claims

1. A system for permitting a user to implement functionality on a computer, the functionality being provided across a plurality of application programs, the computer including a data entry device, comprising:

means for monitoring all data entered by a user within any one of the plurality of application programs, said data including one or more alphabetic letters, symbols and/or words;

a wordbase having stored therein a plurality of item records, each item record having an action word and one of a plurality of associated functions, wherein one of said item records includes a default action word;

means for searching said wordbase for a match with said data entered by said user; and

means for performing said function associated with said data.

2. The system of claim 1, wherein said data is entered via a microphone, selection device, or keyboard.

3. The system of claim 2, further comprising means for recognizing voice signals input via said microphone to produce recognizable data, wherein said recognizable data is used by said means for searching.

4. The system of claim 1, wherein said data entered by said user may be selected with a selection device by said user.

5. The system of claim 1, wherein a word entered by said user is a dual word, wherein said user disambiguates said dual word to indicate to said means for monitoring that said dual word is actionable.

6. The system of claim 1, wherein said function can be activated via entering a plurality of words.

7. The system of claim 1, wherein said means for monitoring monitors for a delineator, wherein data that is actionable is always followed by a delineator.

8. The system of claim 7, wherein said delineator is a punctuation mark, a special character, entry of a space bar, or a click of a selection device.

9. The system of claim 7, wherein said delineator is a predefined key on a keyboard.

10. The system of claim 7, wherein said delineator is a user-defined key.

11. They system of claim 7, wherein said delineator is a button on a point-and-click device.

12. The system of claim 1, further comprising means for displaying a charm box, said charm box having displayed therein information relating to said data entered by said user.

13. The system of claim 1, wherein said function includes launching an application program, a file or a folder.

14. The system of claim 1, wherein said function includes text substitution, wherein said text is substituted at the position of a displayed cursor.

15. The system of claim 1, wherein said means for monitoring can be toggled between on and off.

16. The system of claim 1, wherein said associated function includes calling an agent.

17. The system of claim 1, wherein said wordbase is located on a server connected to a network.

18. The system of claim 1, wherein a single action word can activate two or more functions, the system further comprising means for selecting between said two or more functions when said single action word is entered by said user.

19. The system of claim 1, wherein an action word can be formed by at least two natural language words.

20. The system of claim 1, wherein an action can be a code word or a dual word, the system further comprising means for allowing said user to turn said action word on and off within said wordbase.

21. The system of claim 1, populating said wordbase with help-desk action words relating to features and/or functions of the computer, whereby a customer service person can request said user to enter said help-desk action words in order to assist said user.

22. The system of claim 1, wherein said wordbase is populated by a third party.

23. The system of claim 1, wherein said means for monitoring monitors said data within all contexts of the computer, said data including one or more alphabetic letters, symbols and/or words.

124. The system of claim 22, wherein said wordbase is further populated by the user.

24. A system for permitting a user to access information across a plurality of application programs, comprising:

means for monitoring data selected by a user within any one of the plurality of application programs;

a wordbase; and

means for displaying information relating to said data selected by said user, wherein said wordbase is accessed prior to displaying said information.

25. The system of claim 24, wherein said wordbase having stored therein a plurality of records, each record having an action word and one of a plurality of associated functions, wherein said action word is formed from data selected by said user;

and

means for performing said function associated with said data.

26. The system of claim 25, wherein said means for monitoring also monitors for data entered by said user via a keyboard.

27. The system of claim 26, wherein said data is formed from one or more alphabetic letters, symbols and/or words.

28. The method of claim 26, further comprising the step of establishing a user profile, wherein said information is displayed after consulting said user profile.

29. The method claim 28, wherein said information is accessed from a web page.

30. The method of claim 28, wherein said information is one of a stock quote, statistics, or a translation of said data into another language.

31. The method of claim 28, wherein said information is from a dictionary or an encyclopedia.

32. The method of claim 26, further comprising storing said wordbase on a server connected to a network, wherein the application programs execute on a computer system, which is also connected to said network.

33. The method of claim 26, wherein said wordbase is populated by a third party.

34. The method of claim 33, wherein said wordbase is further populated by said user.

35. The method of claim 26, wherein one of said item records includes a default action word.

36. The system of claim 25, wherein one of said records includes a default action word.

37. The system of claim 25, wherein a single action word can activate two or more functions, the system further comprising means for selecting between said two or more functions when said single action word is selected by said user.

38. The system of claim 25, wherein said action word is formed by at least two natural language words.

39. The system of claim 25, wherein an action word can be a code word or a dual word, the system further comprising means for allowing said user to turn said code word and said dual word on and off within said wordbase.

40. The system of claim 25, wherein said wordbase is populated by a third party.

41. The system of claim 40, wherein said wordbase is further populated by the user.

42. The system of claim 40, further comprising storing said wordbase on a server connected to a network, wherein the application programs execute on a computer system, which is also connected to said network.

43. The system of claim 24, wherein said data is selected via a mouse.

44. The system of claim 24, wherein said means for displaying is configured to display said information based on a user profile.

45. The system of claim 44, further comprising means for maintaining an archive record of language preferences, word frequencies, and utterance behavior of said user.

46. The system of claim 44, further comprising means for maintaining multiple user profiles.

47. The system of claim 24, wherein said means for displaying includes displaying a number of resources relating to said selected data.

48. The system of claim 24, wherein said means for displaying displays a list of equivalent words in one or more languages related to said selected data.

49. The system of claim 24, wherein said means for displaying displays a list of Internet links (URL'"'"'s) related to said selected data.

50. The system of claim 24, wherein said means for displaying displays a list of synonyms for said selected data.

51. The system of claim 24, wherein the plurality of application programs include at least one of an e-mail program, a word processing program, or a browser.

52. The system of claim 24, wherein said data is selected by said user by activating a user-defined key.

53. The system of claim 24, wherein said data can be selected from a web page.

54. The system of claim 24, wherein said information is a stock quote.

55. The system of claim 24, wherein said information is from a dictionary or encyclopedia.

56. The system of claim 24, wherein said information is a statistic.

57. The system of claim 24, wherein said information is displayed within a pop-up window.

58. The system of claim 24, further comprises providing an indication that additional information is available relating to said selected data.

59. The system of claim 24, wherein said information is a translation of said data into another language.

60. The system of claim 24, wherein said information includes a weather report for a designated location.

61. The system of claim 24, wherein said data is selected by said user by pressing a button on a mouse.

62. The system of claim 24, wherein said data is selected by pressing a designated key on a keyboard.

63. The system of claim 62, wherein said information is displayed within a pop-up window.

64. The system of claim 62, wherein said information is a web page accessed via the Internet.

65. The system of claim 24, wherein said wordbase is populated by a third party.

66. The system of claim 65, wherein said wordbase is further populated by the user.

67. The system of claim 24, wherein said wordbase is located on a server connected to a network.

68. The system of claim 24, wherein said data can be selected from within an environment created by an operating system.

69. The system of claim 24, wherein said data is selected by entering a delineator.

70. The system of claim 69, wherein said delineator is user-definable.

71. The system of claim 69, wherein said delineator is a button on a point-and-click device.

72. The method of claim 24, wherein said data is in a first language, and further comprising:

means for searching said wordbase to determine a translation of said data into a second language;

means for displaying said translation of said word in said second language within said pop-up window.

73. The method of claim 72, further comprising the step of establishing a user profile, wherein said translation is displayed after accessing said user profile.

74. The method of claim 72, wherein said first language is English and said second language is Spanish or French.

75. The method of claim 72, wherein said data is selected using a mouse and a pre-defined key on a keyboard.

76. The method of claim 72, wherein said step of displaying includes providing a window that includes a translation of said data in a plurality of languages.

77. The method of claim 72, wherein said wordbase is located on a server connected to a network.

78. The system of claim 24, wherein said data including one or more alphabetic letters, symbols and/or words, and said means for displaying accesses said wordbase prior to displaying said information, wherein said wordbase includes other functions that can be performed upon selection of said selected data, wherein each of said functions is associated with one or more of said alphabetic letters, symbols and/or words.

79. The system of claim 24, further comprising:

means for converting voice signals received via microphone from a user to data, wherein said means for monitoring also monitors for said data entered via said microphone; and

wherein said wordbase having stored therein a plurality of item records, each item record having an action word and one of a plurality of associated functions.

80. The system of claim 79, further comprising:

means for establishing a user profile; and

means for displaying information relating to said selected data, wherein said information is based on said user profile.

81. The system of claim 24, wherein said data entered by said user is selected by using a mouse.

82. A method for permitting a user to access information across a plurality of application programs, comprising:

monitoring data entered by a user within any one of the plurality of application programs;

accessing a wordbase for a match with data entered by said user; and

displaying information within a display box relating to said data entered by said user.

83. The method of claim 82, wherein said step of displaying further includes consulting a user profile prior to displaying said information within said display box.

84. The method of claim 82, wherein said wordbase has stored therein a plurality of item records, each item record having an action word and one of a plurality of associated functions:

and

activating a function associated with said data if said data matches an action word stored within said wordbase.

85. The method of claim 82, wherein said data is selected via a mouse.

86. The method of claim 82, further comprising selecting said data from at least one of an e-mail program, a word processing program, or a browser.

87. The method of claim 82, wherein said data can be selected from a web page.

88. The method of claim 82, wherein said information is a stock quote or statistic.

89. The method of claim 82, wherein said information is from a dictionary.

90. The method of claim 82, wherein said information is from an encyclopedia.

91. The method of claim 82, further comprising providing an indication that additional information is available relating to said selected data.

92. The method of claim 82, wherein said data is selected by the user activating a user-defined key.

93. The method of claim 82, wherein said data is selected by the user pressing a button on a mouse.

94. The method of claim 82, wherein said data is selected by pressing a designated key on a keyboard.

95. The method of claim 82, wherein said wordbase is populated by a third party.

96. The method of claim 95, wherein said wordbase is further populated by the user.

97. The method of claim 96, wherein said information is one of:

a stock quote, a statistic, a translation, or from a dictionary, an encyclopedia, or a web page.

98. The method of claim 95, wherein said wordbase is located on a server connected to a network.

99. The method of claim 98, further comprising entering a user-defined delineator, whereby said data is selected by entering said delineator.

100. The method of claim 82, further comprising entering a user-defined delineator, whereby said data is selected by entering said delineator.

101. The method of claim 82, wherein said date entered by said user is selected by using a mouse.

102. The method of claim 82, wherein said data is monitored within any context of a computer system.

103. A method of accessing information, comprising the steps of:

selecting data via a point-and-click device associated with a computer, wherein said selection can occur within any one of a plurality of application programs executing on the computer;

entering a delineator;

accessing a wordbase using said selected data; and

displaying a window in response to said selection of said data, wherein said window contains information relating to said data.

104. The method of claim 103, further comprising populating said wordbase with a plurality of item records, each item record having an action word and one of a plurality of associated functions.

105. The method of claim 104, wherein one of said item records includes a default action word.

106. The method of claim 104, wherein an action word can be a dual word or a code word, wherein said user disambiguates a dual word to indicate that said dual word is an action word.

107. The method of claim 104, wherein a function can be activated via a plurality of words.

108. The method of claim 104, wherein said functions includes launching an application program, a file or a folder.

109. The method of claim 104, wherein said functions includes text substitution, wherein said text is substituted at the position of a displayed cursor.

110. The method of claim 103, wherein said delineator is a predefined key on a keyboard.

111. The method of claim 103, wherein said information is real-time information.

112. The method of claim 103, wherein said information includes the correct spelling of said data.

113. The method of claim 103, wherein said information includes a translation of said selected data in a different language.

114. The method of claim 103, wherein entering said delineator includes selecting, pointing, or pressing.

115. The method of claim 103, wherein said plurality of application programs include at least one of an e-mail program, a word processing program, or a browser.

116. The method of claim 103, herein said wordbase is populated by a third party.

117. The method of claim 103, further comprises providing an indication that additional information is available relating to said selected data.

118. The method of claim 103, wherein said delineator is a user-defined key.

119. The method of claim 103, wherein said wordbase is located on a server connected to a network.

120. The method of claim 103, further comprising the step of establishing a user profile, wherein said information is displayed after accessing said user profile.

121. A method for permitting a user to access information via a computer, comprising the steps of:

monitoring data selected by a user across a plurality of application programs, wherein said data is selected using a point-and-click device attached to the computer, wherein a selection of said data is followed by either pressing a pre-defined key on a keyboard attached to the computer or by pressing a button on said point-and-click device;

populating a wordbase;

accessing said wordbase in response to said user selecting said data; and

displaying information related to said data, based on said step of accessing said wordbase, within a pop-up window.

122. The method of claim 121, further comprising the step of establishing a user profile, wherein said information is displayed after accessing said user profile.

123. The method of claim 122, wherein said information is one of a stock quote, statistic, a translation, or from a dictionary, an encyclopedia, or a web page.

125. The method of claim 121, wherein said wordbase is located on a server connected to a network.

126. The method of claim 125, further comprising entering a user-defined delineator, whereby said data is selected by entering said delineator.

127. The method of claim 126, wherein said wordbase is populated by a third party.

128. The method of claim 121, wherein said information is one of:

a stock quote, statistic, a translation, or from a dictionary, an encyclopedia, or a web page.

129. The method of claim 121, wherein said wordbase is located on a server connected to a network.

130. The method of claim 121, further comprising entering a user-defined delineator, whereby said data is selected by entering said delineator.

131. The method of claim 121, further comprising monitoring data entered by said user via keyboard, accessing said wordbase in response to said data entered by said user, and displaying said information based on said data entered by said user.