×

Methods and systems for predictive engine evaluation, tuning, and replay of engine performance

  • US 9,135,559 B1
  • Filed: 04/12/2015
  • Issued: 09/15/2015
  • Est. Priority Date: 03/20/2015
  • Status: Active Grant
First Claim
Patent Images

1. A system for evaluating and tuning a predictive engine, comprising:

  • a processor;

    a computer-readable working memory;

    an engine variant of the predictive engine stored in the working memory, wherein the engine variant is determined by an engine parameter set specifying a plurality of algorithms utilized by the engine variant and a plurality of algorithm parameters; and

    a non-transitory, computer-readable storage medium for storing program code, the program code when executed by the processor, causes the processor to perform a process to;

    deploy an initial engine variant of the predictive engine based on an initial engine parameter set;

    receive one or more queries to the initial predictive engine variant from an end-user device;

    in response to the queries, the initial engine variant generates one or more predicted results;

    receive one or more actual results corresponding to the predicted results;

    associate the queries, the predicted results, and the actual results with a replay tag, and record them with the corresponding initial engine variant;

    evaluate a performance of the initial engine variant by computing one or more evaluation results based on at least one evaluation metric with the queries, the predicted results, and the actual results;

    generate a new engine parameter set based on tuning of one or more parameters of the initial engine parameter set, according to the evaluation results of one or more engine variants selected from the group consisting of the initial engine variant and previous engine variants, wherein one or more parameters of the new engine parameter set is selected from the group consisting of a data source, an algorithm, an algorithm parameter, and a business rule;

    deploy a new engine variant of the predictive engine based on the new engine parameter set, and replace the initial engine variant with the new engine variant;

    receive a replay request specified by one or more identifiers of at least one currently or previously deployed engine variant from an operator; and

    in response to the replay request, replay at least one item selected from the group consisting of the queries, the corresponding predicted results, the actual results, and the evaluation results.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×