System and method for performing compound computational experiments

• US 6,389,380 B1
• Filed: 09/16/1998
• Issued: 05/14/2002
• Est. Priority Date: 09/16/1997
• Status: Expired due to Term

• Current Assignee: Evolving Logic Associates
• Original Assignee: Evolving Logic Associates
• Inventors: Bankes, Steven C.
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): Broda, Samuel
• Application Number:
  US09/154,606
• Time in Patent Office: 1,336 Days
• Field of Search: 703/1, 703/2, 703/6, 703/13-22, 706/45
• US Class Current: 703/17
• CPC Class Codes: G06N 3/126 Evolutionary algorithms, e.g. genetic algorithms or genetic programming
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

A system and method for performing compound computational experiments include an experiment having at least one input. The experiment includes a process controllable by a computer and responsive to the input to produce at least one output, the output capable of being stored on a medium readable by the computer. The system includes a generator capable of running on the computer. The generator automates the performance of a plurality of experiments and automatically varies the inputs for the experiments. An outcome store of the system stores the outputs on the computer-readable medium. The system also includes a display capable of running on the computer. The display presents an interpretation of a set of the outputs extracted from the outcome store. The display is responsive to user input to alter the interpretation, to present an interpretation of a different set of outputs, or to cause the generator to perform a plurality of experiments.

Claims

1. A system for performing compound computational experiments, comprising:

a first plurality of experiments, each of said first plurality of experiments having at least one input, each of said first plurality of experiments including a process contollable by a computer and responsive to said at least one input to produce at least one output, said at least one output capable of being stored on a medium readable by said computer;

a generator capable of running on said computer, said generator automating the performance of said first plurality of experiments, said generator automatically varying said at least one input for each of said first plurality of experiments;

an outcomestore storing said at least one outputs on said medium; and

a display capable of running on said computer, said display presenting an interpretation of a set of said at least one outputs extracted from said outcomestore, said display responsive to user input to alter said interpretation, to present an interpretation of a different set of said at least one outputs, or to cause said generator to perform a second plurality of experiments.

2. The system as described in claim 1, further comprising:

a dispatch capable of running on said computer, said computer being one of a plurality of networked computers, said dispatch causing a first experiment to run on a first of said networked computers and causing a second experiment to run on a second of said networked computers.

3. The system as described in claim 1, further comprising:

a dispatch capable of running on said computer, said dispatch causing each of said experiments to be run on a computer upon determining, by examining data in the outcomeStore, that the experiment has not previously been run.

4. The system as described in claim 1, wherein said process includes first and second sub-models such that each of said first plurality of experiments uses either said first sub-model or said second sub-model, said first sub-model responsive to first inputs, said second sub-model responsive to second inputs, wherein said generator automatically produces sets of said first inputs and automatically varies the values of said first inputs in each of said sets of said first inputs, wherein said generator automatically produces sets of said second inputs and automatically varies the values of said second inputs in each of said sets of said second inputs, and wherein said generator automatically provides a set of said first inputs for each of said first plurality of experiments using said fist sub-model and automatically provides a set of said second inputs for each of said first plurality of experiments using said second sub-model.

5. The system as described in claim 1, further comprising:

a second generator capable of running on said computer, said second generator automating a second performance of said first plurality of experiments and automatically varying said inputs for said first plurality of experiments according to a sampling or search strategy different than that used by said generator; and

a second outcomestore storing on said medium outputs produced by said second performance of said first plurality of experiments.

6. The system as described in claim 1, wherein said display presents a plotted representation of at least one dimension of a first set of said fist plurality of experiments using a first set of said at least one outputs, said system further comprising:

a user interface capable of running on said computer, said user interface including a control manipulable by a user to associate a value with an unplotted dimension of said first set of said first plurality of experiments, said display responsive to a manipulation of said control to (1) identify a second set of said at least one outputs produced by a second set of said first plurality of experiments performed using a substantial equivalent of said value as an input associated with said unplotted dimension, and to (2) present a plotted representation of at least one dimension of said second set of said first plurality of experiments using said second set of said at least one outputs.

7. The system as described in claim 1, wherein said display presents a plotted representation of at least one dimension of a first set of said first plurality of experiments using a first set of said at least one outputs, said system her comprising:

a user interface capable of running on said computer, said user interface including a control manipulable by a user to associate a value with an unplotted dimension of said first set of said first plurality of experiments, said display responsive to a manipulation of said control to (1) cause said generator to perform a second plurality of experiments using a substantial equivalent of said value as an input associated with said unplotted dimension to produce a second set of said at least one outputs, and to (2) present a plotted representation of at least one dimension of said second plurality of experiments using said second set of said at least one outputs.

8. The system as described in claim 1, further comprising:

a second generator capable of running on said computer, said second generator automating a second performance of said first plurality of experiments and automatically varying said inputs for said first plurality of experiments according to a sampling or search strategy different than that used by said generator.

9. The system as described in claim 1, wherein each of said first plurality of experiments can accept at least two inputs, a first of said at least two inputs providing a model, said model affecting the manner in which said process of at least one of said first plurality of experiments responds to a second of said at least two inputs.

10. The system as described in claim 1, wherein each of said plurality of experiments can accept at least two inputs, a first of said at least two inputs representing a set of models, one of said set of models selectable to affect the manner in which said process of at least one of said first plurality of experiments responds to a second of said at least two inputs.

11. The system as described in claim 1, wherein said outcomestore is a storage readable by a computer, further comprising:

a transformer performing a transformation upon at least a set of said at least one outputs in said outcomestore and thereby providing a second outcomestore storing transformed outputs.

12. The system as described in claim 1, wherein said outcomestore is a storage readable by a computer, further comprising:

a user interface presenting data representing a selectable range of values and responsive to a selection of said range of values to cause said generator to vary a value of said at least one input only within said range of values selected for experiments within said first plurality of experiments performed after said selection of said range of values.

13. The system as described in claim 1, wherein said outcomestore is a storage readable by a computer, wherein said process includes a variable model, and wherein said generator automatically selects and varies one or more model parts to complete each variable model of each process and provides said one or more model parts to each experiment as an input, each varied model part affecting the manner in which said process responds to at least one input.

14. A system for performing compound computational experiments, comprising:

a first plurality of experiments, each of said first plurality of experiments including at least one of a set of sub-models, each sub-model responsive to a set of input objects to produce a set of output objects, each sub-model adapted to apply a unique set of functions to input objects;

a generator automating the performance of said first plurality of experiments to produce a first group of sets of output objects, said generator automatically generating a set of input objects for use by each of said first plurality of experiments, said generator providing one of said set of sub-models for use by each of said first plurality of experiments; and

a display presenting an interpretation of at least a portion of said first group of sets of output objects, said display responsive to user input to alter said interpretation, to present an interpretation of a different portion of said group of sets of output objects, or to cause said generator to perform a second plurality of experiments.

15. The system as described in claim 14, wherein said generator automatically varies the sets of input objects used by said first plurality of experiments.

16. The system as described in claim 14, further comprising:

a storage including sets of experiment data and result data, said experiment data representing information about one of said first plurality of experiments, said result data representing information about a set of outputs produced by one of said first plurality of experiments.

17. The system as described in claim 14, wherein said generator uses a first search strategy to generate said sets of input objects, the system further comprising:

a second generator automating the performance of a third plurality of experiments, said second generator using a second search strategy to generate sets of input objects for each of said third plurality of experiments.

18. The system as described in claim 17, wherein said first search strategy comprises full factorial sampling.

19. The system as described in claim 17, wherein said first search strategy comprises Monte Carlo sampling.

20. The system as described in claim 17, wherein said first search strategy comprises a generational search based on a population size and a number of generations.

21. The system as described in claim 14, further comprising:

a user interface including a control manipulable by a user to (1) associate an explore value with a dimension of said first plurality of experiments, and to (2) cause said display to present an interpretation of a second group of sets of output objects, said second group of sets of output objects produced by each experiment in said first plurality of experiments performed using a substantial equivalent to said explore value for input objects associated with said dimension.

22. The system as described in claim 14, further comprising:

a user interface including a control manipulable by a user to (1) associate an explore value with a dimension of said first plurality of experiments, to (2) cause said generator to perform a second plurality of experiments using a substantial equivalent to said explore value for input objects associated with said dimension, and to (3) cause said display to present an interpretation of a second group of sets of output objects produced by said second plurality of experiments.

23. The system as described in claim 22, further comprising:

a storage including sets of experiment data and result data, said experiment data representing information about one of said second plurality of experiments, said result data representing information about a set of outputs produced by one of said second plurality of experiments.

24. The system as described in claim 14, wherein a first of said first plurality of experiments is run on a first computer and a second of said first plurality of experiments is run on a second computer coupled to said first computer.

25. The system as described in claim 24, wherein said first of said first plurality of experiments runs on said first computer while said second of said first plurality of experiments runs on said second computer.

26. The system as described in claim 14, wherein each set of input objects generated for said first plurality of experiments is associated with one experiment in said first plurality of experiments and is also associated with a set of output objects produced by said one experiment, said system further comprising:

a filter comparing a set of output objects or an associated set of input objects to one or more predetermined values or ranges of values to select said second plurality of experiments.

27. A method of performing compound computational experiments, comprising:

automatically performing a first plurality of experiments, each of said first plurality of experiments including a process responsive to a set of input objects to produce a set of output objects;

automatically varying sets of input objects for a plurality of said processes;

storing each of said sets of output objects as a first group of sets of output objects;

presenting an interpretation of a portion of said first group of sets of output objects; and

responding to user input by altering said interpretation, by presenting an interpretation of a second portion of said first group of sets of output objects or by performing a second plurality of experiments.

28. The method of claim 27, wherein said interpretation is a plotted representation of at least one dimension of a first set of said first plurality of experiments, further comprising:

associating a value with an unplotted dimension of said first set of said first plurality of experiments;

identifying a second set of said at least one outputs produced by a second set of said first plurality of experiments performed using a substantial equivalent of said value as an input associated with said unplotted dimension; and

presenting a plotted representation of at least one dimension of said second set of said first plurality of experiments using said second set of said at least one outputs.

29. The method of claim 28, further comprising:

performing a first of said first plurality of experiments using a first computer; and

performing a second of said first plurality of experiments using a second computer connected to said first computer.

30. The method of claim 28, further comprising:

examining sets of output objects to determine that each of said second plurality of experiments has not been previous performed; and

performing each of said second plurality of experiments upon determining it has not been previously performed.

31. The method of claim 27, wherein said interpretation is a plotted representation of at least one dimension of a first set of said first plurality of experiments, further comprising:

associating a value with an unplotted dimension of said first set of said first plurality of experiments;

performing a second plurality of experiments using a substantial equivalent of said value as an input associated with said unplotted dimension to produce a second set of said at least one outputs; and

presenting a plotted representation of at least one dimension of said second plurality of experiments using said second set of said at least one outputs.

32. A process for performing a compound computational experiment, comprising:

automatically performing a first plurality of experiments to produce a first group of sets of output objects, each of said first plurality of experiments including at least one of a set of sub-models, each sub-model responsive to a set of input objects to produce a set of output objects, each sub-model adapted to apply a unique set of functions to input objects;

generating a set of input objects for use by each of said first plurality of experiments;

providing one of said set of sub-models for use by each of said first plurality of experiments;

presenting an interpretation of at least a portion of said first group of sets of output objects; and

responding to user input to alter said interpretation, to present an interpretation of a different portion of said group of sets of output objects, or to perform a second plurality of experiments.

33. The process for performing a compound computational experiment described in claim 32, further comprising:

varying said sets of input objects used by said first plurality of experiments.

34. The process for performing a compound computational experiment described in claim 32, further comprising:

storing associated sets of experiment data and result data, said experiment data representing information about one of said first plurality of experiments, said result data representing information about a set of outputs produced by one of said first plurality of experiments.

35. The process for performing a compound computational experiment described in claim 32, further comprising:

selecting a second plurality of experiments by comparing a set of output objects or a set of input objects to one or more predetermined values or ranges of values.

36. The process for performing a compound computational experiment described in claim 32, further comprising:

producing a second group of sets of output objects by applying a transformation to said first group of sets of output objects; and

presenting an interpretation of at least a portion of said second group of sets of output objects.

37. The process for performing a compound computational experiment described in claim 32, wherein said interpretation is a plotted representation of at least one dimension of a first set of said first plurality of experiments, further comprising:

associating a value with an unplotted dimension of said first set of said first plurality of experiments;

identifying a second group of sets of output objects produced by a second set of said first plurality of experiments performed using a substantial equivalent of said value in an input object associated with said unplotted dimension; and

presenting a plotted representation of at least one dimension of said second set of said first plurality of experiments using said second group of sets of output objects.

38. The process for performing a compound computational experiment described in claim 32, wherein said interpretation is a plotted representation of at least one dimension of a first set of said first plurality of experiments, further comprising:

associating a value with a unplotted dimension of said first set of said first plurality of experiments;

producing a second group of sets of output objects by performing a second plurality of experiments using a substantial equivalent of said value in an input object associated with said unplotted dimension; and

presenting a plotted representation of at least one dimension of said second plurality of experiments using said second group of sets of output objects.

39. The process for performing a compound computational experiment described in claim 32, wherein at least one of said input objects represents a result from a second compound computational experiment.

40. The process for performing a compound computational experiment described in claim 39, wherein said second compound computational experiment differs from said compound computation experiment.

41. The process for performing a compound computational experiment described in claim 32, wherein a plurality of said set of input objects are derived from results of a third plurality of experiments.

42. A method of performing compound computational experiments, comprising:

a step for performing a first plurality of experiments to produce a first group of sets of output objects;

a step for automatically generating input objects used by each of said first plurality of experiments;

a step for automatically varying some of said input objects;

a step for selecting and providing a model used by each of said first plurality of experiments, said model affecting the manner in which a respective one of said first plurality of experiments responds to input objects;

a step for presenting a plotted representation of at least one dimension of a first set of said first plurality of experiments;

a step for associating a value with an unplotted dimension of said first set of said first plurality of experiments;

a step for producing a second group of sets of output objects by performing a second plurality of experiments using a substantial equivalent of said value in input objects associated with said unplotted dimension; and

a step for presenting a plotted representation of at least one dimension of said second plurality of experiments using said second group of sets of output objects.

43. A method of performing compound computational experiments, comprising:

a step for performing a first plurality of experiments to produce a first group of sets of output objects;

a step for automatically generating input objects used by each of said first plurality of experiments;

a step for automatically varying some of said input objects;

a step for selecting and providing a model used by each of said first plurality of experiments, said model affecting the manner in which a respective one of said first plurality of experiments responds to input objects;

a step for presenting a plotted representation of at least one dimension of a first set of said first plurality of experiments;

a step for associating a value with an unplotted dimension of said first set of said first plurality of experiments;

a step for identifying a second group of sets of output objects produced by a second set of said first plurality of experiments performed using a substantial equivalent of said value in input objects associated with said unplotted dimension; and

a step for presenting a plotted representation of at least one dimension of said first plurality of experiments using said second group of sets of output objects.