Distributed testing for computing features
First Claim
1. A method comprising:
- distributing a feature testing tool to one or more client computing systems;
distributing one or more computing feature builds from a managing computing system to the client computing systems;
distributing one or more data sets from the managing computing system to the client computing systems;
running the feature testing tool on a given one of the client computing systems during times when the feature testing tool determines that the client computing system is not otherwise in active use, wherein the feature testing tool prompts the computing feature build to process one of the data sets, and collects results from the processing of the data set, including any erroneous results; and
providing to the managing computing system the results from the computing feature build processing the data sets on the client computing systems.
2 Assignments
0 Petitions
Accused Products
Abstract
A new method is provided for using distributed computing for computing process testing. The method includes distributing a feature testing tool, computing feature builds, and data sets to client computing systems. The feature testing tool is run on the client computing systems during times when the feature testing tool determines that client computing system is not otherwise in active use. The feature testing tool prompts the computing feature build to process the data sets, and collects results from the processing of the data set, including any erroneous results. The results from the computing feature build processing the data sets on the client computing systems are provided to a managing computing system. The managing computing system may try to reproduce the erroneous results on other client computing systems with alternate hardware-software configurations, to send characterized erroneous results to an investigation queue to isolate their cause, and to report useful information on the erroneous results to developers working on the computing feature.
-
Citations
20 Claims
-
1. A method comprising:
-
distributing a feature testing tool to one or more client computing systems; distributing one or more computing feature builds from a managing computing system to the client computing systems; distributing one or more data sets from the managing computing system to the client computing systems; running the feature testing tool on a given one of the client computing systems during times when the feature testing tool determines that the client computing system is not otherwise in active use, wherein the feature testing tool prompts the computing feature build to process one of the data sets, and collects results from the processing of the data set, including any erroneous results; and providing to the managing computing system the results from the computing feature build processing the data sets on the client computing systems. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A medium comprising executable instructions capable of being read and executed by one or more components of a computing system, thereby configuring the computing system with the capability for:
-
a server to provide one or more corpus segments over a network to one or more computing system clients; one or more clients to receive at least one of the corpus segments, to execute a feature driver that runs a natural language feature on the received corpus segment and tests for any indications of erroneous behavior by the natural language feature, and to provide results from the feature driver to the server; and the server to receive and compile the results from the feature driver from the one or more clients into an integrated test result for the natural language feature.
-
-
20. A computing system configured to:
-
distribute a natural language testing utility to one or more remote computing devices having a variety of different hardware-software configurations; iteratively distribute test versions of a natural language feature to the remote computing devices; iteratively distribute corpus segments from a cyclical queue of corpus segments to the remote computing devices to test the natural language feature on; iteratively receive test results from the remote computing devices; record whether the test results include no bugs in a fresh corpus segment, an original bug in a fresh corpus segment, no bugs in a corpus segment sent to a different-configuration remote computing device to try to reproduce a bug, a reproduction of a bug in a corpus segment sent to a different-configuration remote computing device, or a reproduction of a bug sent back to the original remote computing device on which it occurred after failure to reproduce it on a different-configuration remote computing device; send data segments with reproduced bugs to an investigation queue and run the natural language feature iteratively on increasingly smaller divisions or sequences of divisions of the data segments until the bug is isolated; and provide information on the isolated bugs to support debugging the natural language feature.
-
Specification