Customizable answer engine implemented by user-defined plug-ins

• US 8,751,466 B1
• Filed: 01/12/2014
• Issued: 06/10/2014
• Est. Priority Date: 01/12/2014
• Status: Expired due to Fees

• Current Assignee: Machine Intelligence Services, Inc.
• Original Assignee: Machine Intelligence Services, Inc.
• Inventors: Tsay, Jeff
• Primary Examiner(s): DWIVEDI, MAHESH H
• Application Number:
  US14/153,034
• Time in Patent Office: 149 Days
• Field of Search: 707/706, 707/707
• US Class Current: 707/700
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Methods of computing answer results to user-submitted queries, using a plug-in executor, a user database, a plug-in database, content recognizers, answer generators and a controller. Controller submits content recognizer specifications and sets of answer generator specifications to the plug-in executor, combines content recognition results, combines answer generator answer results, determines the subset of answer generators to execute using an answer generator set determinator, looks up the specifications for each answer generator in the subset of answer generators, and responds to the client program with a list of answer results. Users customize how answer engines respond to queries, answers are generated by answer generator plug-ins created by other users, content recognizers are implemented by content recognizer plug-ins created by other users, and answer generators utilize content recognizers even if the content recognizers are not created by a single author and content recognition execution resources are bounded.

Claims

1. A method of computing answer results to queries submitted by a user comprising:

providing a plug-in executor comprising at least one plug-in created by a plug-in developer;

wherein said one or more plug-ins are configured to execute one or more of system-defined programs and plug-in developer-defined programs, and

wherein said plug-in executor is configured to one or more of execute arbitrary program text uploaded by said plug-in developer or execute routines parameterized by said plug-in developer;

providing a user database configured to store and retrieve user preferences;

providing a plug-in database configured to store and retrieve plug-in specifications;

providing a content recognizer set determinator configured to compute a subset of all available content recognizers,

wherein said computation of said subset depends on one or more of contents of a query submitted by said user,

providing an answer generator set determinator configured to compute a subset of all available answer generators,

wherein said compute of said subset depends on one or more of contents of said one or more queries submitted by said user and a dictionary of combined recognition results used as an input; and

,

providing a controller configured to

receive a query from a client program,

lookup said user'"'"'s preferences in said user database,

determine the subset of content recognizers to execute using said content recognizer set determinator, and lookup specifications for each content recognizer in said subset of content recognizers,

submit said specifications for each content recognizer to said plug-in executor, using said query and said user'"'"'s preferences as input, and combine all content recognition results into a dictionary of combined content recognition results,

determine the subset of answer generators to execute using said answer generator set determinator, and lookup the specifications for each answer generator in said subset of answer generators, to yield a set of answer generator specifications,

submit said set of answer generators specifications to said plug-in executor, using said query, said user'"'"'s preferences, and said dictionary of recognition results as input, and combine the answer results from each answer generator into a list of answer results, and

respond to said client program with said list of answer results, such that

said user is able to customize how an answer engine responds to queries,

answers are generated by at least one answer generator plug-in created by one or more plug-in developers,

content recognizers are implemented by at least one content recognizer plug-in created by one or more other plug-in developers, and

answer generators utilize content recognizers even if said content recognizers are not created by a single plug-in developer, and content recognition execution resources are bounded.

2. The method of claim 1, wherein said content recognizers are further configured to execute a plug-in developer-specified regular expression on said answer query using a regular expression engine, and if said regular expression matches a portion of said answer query, to output a recognition result with the matched groups of said regular expression as auxiliary data.

3. The method of claim 2, wherein said regular expression engine is modified to limit the number of backtracking attempts before reporting a negative match, such that the execution of said regular expression is bounded.

4. The method of claim 1, wherein said answer generators are further configured to generate answer locations using a URL template, which may have parameters that are substituted by data in said queries, said dictionary of combined content recognition results, or said user preferences.

5. The method of claim 1, wherein the content recognizer set determinator is further configured to

build the set of content recognizers by looking up in said user database a list of trigger-answer generator associations selected by said user initiating said query;

and, combining all referenced content recognizers referenced by a trigger in said list of trigger-answer generator associations.

6. The method of claim 1, further comprising providing an index of recognition keys mapped to content recognizers configured to lookup content recognizers that output a given recognition key, and wherein the content recognizer set determinator is further configured to

look up in said user database a list of answer generators selected by the user initiating the query;

build a set of recognition keys by combining all recognition keys referenced by said answer generator in said list of answer generators, into a set;

build the set of content recognizers by looking up each recognition key in said set of recognition keys in said index of recognition keys mapped to content recognizers; and

,

combine one or more found content recognizers into a set.

7. The method of claim 1, further comprising providing a term index configured to look up and store answer generator references associated with terms, and wherein the content recognizer set determinator is further configured to

perform transformations on said term dictionary by one or more of correcting spelling mistakes, adding synonyms, adding translations, stemming words, removing stop words, expanding acronyms, or any combination thereof;

lookup in said term index, using the transformed terms, to yield a set of answer generators; and

,

build the set of content recognizers by combining all content recognizers referenced by an answer generator in said set of answer generators.

8. The method of claim 1, wherein the answer generator set determinator is further configured to

look up in said user database a list of trigger-answer generator associations selected by said user;

look up trigger specification in said plug-in database, for each trigger referenced by said list of trigger-answer generator associations, to yield a set of trigger specifications;

submit said set of trigger specifications to said plug-in executor, using said dictionary of combined recognition results as input, to determine which triggers are activated; and

,

build the set of answer generators to activate by selecting one or more answer generators in said list of trigger-answer generator associations, that have their associated triggers activated.

9. The method of claim 1, wherein said plug-in executor is further configured to execute programs in a sandboxed environment in order to limit computation resources, memory, system resource access, or network access utilized by said programs.

10. The method of claim 9, wherein the programs comprises scripts and wherein said plug-in executor is further configured to execute the programs in an interpreter.

11. The method of claim 9, wherein said plug-in executor is further configured to

monitor said computation resources, said memory, and bandwidth used by one or more of said at least one plug-in created by said plug-in developer, wherein said bandwidth comprises upload bandwidth and download bandwidth, and

adjust a score for said at least one plug-in by penalizing for said computation resources, said memory, said upload bandwidth, and said download bandwidth used by said at least one plug-in.

12. The method of claim 1, wherein said answer engine is configured to use a report by adjusting the score of one or more of said at least one plug-in created by said plug-in developer, by a value dependent on both a type of user interface activity and a relevance score reported by one or more of said at least one plug-in, wherein said user interface activity is configured to better order and rank one or more plug-ins, wherein answer results and one or more plug-ins trade-off between accuracy and wherein said answer results are shown to said user or one or more other users more prominently.

13. The method of claim 12, wherein said at least one plug-in is an answer generator.

14. The method of claim 12, wherein said at least one plug-in is a content recognizer.

15. The method of claim 1, wherein said controller is further configured to sanitize said list of answer results such that the content in said list of answers, when inserted into a user interface, is not able to perform malicious operations, wherein said malicious operations include sending private information to one or more external sites.

16. The method of claim 1, further comprising providing a plug-in developer and system populated index from keywords to specifications for recognition results, wherein said controller is further configured to search said index from keywords to specifications for recognition results, using said query as input, to produce additional recognition results by processing said specifications for recognition results.

17. The method of claim 1, further comprising providing a knowledge base of facts configured to look up facts for a given query term, wherein said controller is further configured to search said knowledge base using said query term as input, to produce additional recognition results.

18. The method of claim 1, further comprising providing a term/recognition key index configured to look up and store answer generator references associated with terms and recognition keys, and wherein said answer generator set determinator is further configured to:

transform terms in an answer query by one or more of correcting spelling mistakes, adding synonyms, adding translations, stemming words, removing stop words, expanding acronyms, or any combination thereof; and

,

lookup in said term/recognition key index, using the transformed terms and the recognition keys output by said set of content recognizers as said input, to yield a partial set of answer generators included in the final set of answer generators output by said answer generator set determinator.

19. The method of claim 1, further comprising providing a plug-in developer populated generic database configured to store and retrieve data accepted from said plug-in developer, wherein said plug-in executor is further configured to provide a plug-in access to query said generic database during execution of said plug-in.

20. The method of claim 1, wherein said at least one plug-in created by said plug-in developer, provide documentation on how to use said one or more plug-ins given a partial query, and wherein said user is shown instructions on how to use said at least one plug-in created by said plug-in developer while typing an answer query.

21. A method of displaying a list of answer results in response to user-submitted queries comprising:

providing an answer engine configured to respond to user-submitted queries with a list of answer results;

providing an answer displayer comprising a browser plug-in, a customized browser, an application, or any combination thereof, wherein said answer display is configured to use browser components to

receive an answer query as input from a user,

send said answer query to said answer engine,

receive and parse answer results returned by said answer engine,

find an existing display area or open a new display area in a user interface to display said answer results,

remove one or more user interface element showing an answer result from a previous query, in said display area,

unless the removal of the one or more user interface elements would cause the display area to be closed by said answer displayer, or

unless said user has indicated to keep the one or more user interface elements,

compute a first group of answer results by selecting one link answer result, or one or more inline answer results from said answer results,

display said first group of answer results in said display area,

compute the remaining groups of answer results by selecting one link answer result, or one or more inline answer results from said answer results, until no more ungrouped answer results remain, and

display in said display area one or more remaining groups from said remaining groups by directly displaying answer results if said one or more remaining groups consists only of inline answer results, or by displaying a button if said one or more remaining groups consists of a link answer result, such that a linked answer location is loaded and displayed when said user selects said button;

wherein the answer displayer shows answer results to said user without requiring said user to manually open or close display areas or to remove user interface elements showing results of previous queries,

wherein the answer displayer shows a plurality of results to said user from different sources without requiring the user to submit said query to multiple user interfaces,

wherein links to answer locations are not fetched until the user selects the links, and

wherein the answer displayer shows inline answer results to said user without additional user interaction.

22. The method of claim 21, wherein said answer displayer is further configured to

detect when said user selects, closes, or rearranges each user interface element of said one or more user interface elements representing said one or more answer results, and

send a report of user interface activity to said answer engine.

23. A system configured to compute answer results to queries submitted by a user comprising:

a plug-in executor comprising at least one plug-in created by a plug-in developer,

wherein said one or more plug-ins are configured to execute one or more of system-defined programs and plug-in developer-defined programs, and

wherein said plug-in executor is configured to one or more of execute arbitrary program text uploaded by said plug-in developer or execute routines parameterized by said plug-in developer;

a user database configured to store and retrieve user preferences;

a plug-in database configured to store and retrieve plug-in specifications;

a content recognizer set determinator configured to compute a subset of all available content recognizers,

wherein said compute of said subset depends on one or more of contents of a query submitted by said user,

an answer generator set determinator configured to compute a subset of all available answer generators,

wherein said computation of said subset depends on one or more of contents of said one or more queries submitted by said user and a dictionary of combined recognition results used as an input; and

,

a controller configured to

receive a query from a client program,

lookup said user'"'"'s preferences in said user database,

determine the subset of content recognizers to execute using said content recognizer set determinator, and lookup specifications for each content recognizer in said subset of content recognizers,

submit said specifications for each content recognizer to said plug-in executor, using said query and said user'"'"'s preferences as input, and combine all content recognition results into a dictionary of combined content recognition results,

determine the subset of answer generators to execute using said answer generator set determinator, and lookup the specifications for each answer generator in said subset of answer generators, to yield a set of answer generator specifications,

submit said set of answer generators specifications to said plug-in executor, using said user'"'"'s preferences and said dictionary of recognition results as input, and combine the answer results from each answer generator into a list of answer results, and

respond to said client program with said list of answer results, such that

said user is able to customize how an answer engine responds to queries,

answers are generated by at least one answer generator plug-in created by one or more plug-in developers,

content recognizers are implemented by at least one content recognizer plug-in created by one or more other plug-in developer, and

answer generators utilize content recognizers even if said content recognizers are not created by a single plug-in developer, and content recognition execution resources are bounded.

24. A system configured to compute answer results to queries submitted by a user comprising:

an answer engine configured to respond to user-submitted queries with a list of answer results;

an answer displayer comprising a browser plug-in, a customized browser, an application, or any combination thereof, wherein said answer display is configured to use browser components to

receive an answer query as input from a user,

send said answer query to said answer engine,

receive and parse answer results returned by said answer engine,

find an existing display area or open a new display area in a user interface to display said answer results,

remove one or more user interface element showing an answer result from a previous query, in said display area,

unless the removal of the one or more user interface elements would cause the display area to be closed by said answer displayer, or

unless said user has indicated to keep the one or more user interface elements,

compute a first group of answer results by selecting one link answer result, or one or more inline answer results from said answer results,

display said first group of answer results in said display area,

compute the remaining groups of answer results by selecting one link answer result, or one or more inline answer results from said answer results, until no more ungrouped answer results remain, and

display in said display area one or more remaining groups from said remaining groups by directly displaying answer results if said one or more remaining groups consists only of inline answer results, or by displaying a button if said one or more remaining groups consists of a link answer result, such that a linked answer location is loaded and displayed when said user selects said button;

wherein the answer displayer shows answer results to said user without requiring said user to manually open or close display areas or to remove user interface elements showing results of previous queries,

wherein the answer displayer shows a plurality of results to said user from different sources without requiring the user to submit said query to multiple user interfaces,

wherein links to answer locations are not fetched until the user selects the links, and

wherein the answer displayer shows inline answer results to said user without additional user interaction.