Adaptive Experimentation Method and System

• US 20110137849A1
• Filed: 02/14/2011
• Published: 06/09/2011
• Est. Priority Date: 01/10/2006
• Status: Active Grant

• Current Assignee: Manyworlds Incorporated
• Original Assignee: Manyworlds Incorporated
• Inventors: Flinn, Steven Dennis, Moneypenny, Naomi Felina
• Granted Patent: US 9,159,027 B2
• Time in Patent Office: Days
• US Class Current: 706/52
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/045 Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A computer-implemented adaptive experimentation method and system is described that automatically selects and executes information gathering actions. The adaptive experimentation method and system integrates value of information considerations, experimental design, and inferences from experimental results. The experimental results may include behaviors of users of a computer-based system. The process enables an automatic, adaptive process for attaining additional information and applying the attained information in making subsequent experiment decisions.

Claims

1. A computer-implemented experimentation method comprising:

selecting a first experiment, wherein the selecting of the first experiment is based, at least in part, on the expected value of information of the experiment;

performing the first experiment, wherein the first experiment is executed on a processor-based computing device;

inferring automatically a user preference from a plurality of user behaviors that occur after execution of the first experiment; and

selecting a second experiment based, at least in part, on the inferred preference.

2. The method of claim 1, further comprising:

generating a recommendation.

3. The method of claim 1, further comprising:

generating a response to computer-implemented search request.

4. The method of claim 1, further comprising:

performing a computer-implemented information retrieval.

5. The method of claim 1, further comprising:

modifying a computer-based structural element.

6. The method of claim 1, further comprising:

applying information about the intrinsic characteristics of an item of content.

7. The method of claim 1, further comprising:

applying a statistical learning algorithm.

8. The method of claim 1, further comprising:

applying an experimental design algorithm.

9. The method of claim 1, further comprising:

accessing an expected cost of the first experiment.

10. The method of claim 1, further comprising:

determining an expected net value of information.

11. An adaptive experimentation system comprising:

an information gathering function executed on a processor-based computing device that conducts a first experiment;

a function that infers a user preference from a plurality of user behaviors occurring after the first experiment is conducted; and

an experimental design function, wherein the experimental design function automatically selects a second online experiment to perform based, at least in part, the inferred user preference and an expected value of information.

12. The system of claim 11, further comprising:

a recommendation generating function.

13. The system of claim 11, further comprising:

a function that modifies a computer-based structural element.

14. The system of claim 11, further comprising:

a statistical learning function.

15. The system of claim 11, further comprising:

an experimental design function.

16. The system of claim 11, further comprising:

a function that applies an intrinsic characteristic of an item of content.

17. The system of claim 11, further comprising:

an expected cost of the second experiment.

18. An adaptive decision method comprising:

simulating on a processor-based computing device an experimental infrastructure, wherein the simulation applies a plurality of probabilities;

determining a value of information that is expected to be generated by an implementation of the experimental infrastructure, based, at least in part, on the simulation of the experimental infrastructure; and

generating an expected value of the experimental infrastructure based on the expected value of information and an expected cost of the experimental infrastructure.

19. The method of claim 18, further comprising:

simulating experimental infrastructure that includes an instrument.

20. The method of claim 18, further comprising:

simulating experimental infrastructure that includes a digitized knowledge base.