×

Method for determining a trend of a user engagement metric

  • US 10,277,694 B2
  • Filed: 08/01/2016
  • Issued: 04/30/2019
  • Est. Priority Date: 04/04/2016
  • Status: Active Grant
First Claim
Patent Images

1. A computer-implemented method for determining a trend of a user engagement metric with respect to a web service, the method being executable by a server, the method comprising:

  • receiving, at the server, a plurality of user device requests relating to a web service during an experimental period, the plurality of user device requests originating from a plurality of user devices;

    providing, by the server, a test version of the web service to a test group selected from the plurality of user devices, the test version of the web service being the web service having an experimental treatment applied thereto;

    providing, by the server, a control version of the web service to a control group selected from the plurality of user devices, the control version of the web service being a version of the web service without the experimental treatment applied thereto;

    acquiring an amplitude metric and a phase metric for each one of the user devices of the control and test groups, the acquiring including, for a given one of the user devices of the control and test groups;

    acquiring, by the server, a plurality of indications for the given one, the plurality of indications being based on interactions of the given one with its corresponding web service,accessing the plurality of indications for the given one,calculating a periodicity metric based at least in part on a discretization transform performed on the plurality of indications,calculating the amplitude metric based at least in part on a magnitude of the periodicity metric, the amplitude metric representing a magnitude of change of the user engagement metric with respect to the corresponding web service, andcalculating the phase metric based at least in part on an imaginary part of the periodicity metric, the phase metric representing a direction of change of the user engagement metric with respect to the corresponding web service;

    determining, by the server, average group metrics including;

    calculating a control average amplitude metric by averaging amplitude metrics calculated for each of the user devices of the control group,calculating a test average amplitude metric by averaging amplitude metrics calculated for each of the user devices of the test group, andcalculating a test average phase metric by averaging phase metrics calculated for each of the user devices of the test group;

    determining the trend of the user engagement metric with respect to the web service, the determining the trend being based on analyzing of the control average amplitude metric and the test average amplitude and phase metrics; and

    when a difference between the test average amplitude metric and the control average amplitude metric and the test average phase metric have oppositely signed values, determining that the experimental treatment applied to the web service has caused a decreasing trend in the user engagement metric over the web service without the experimental treatment,wherein;

    the difference between the test average amplitude metric and the control average amplitude metric is negative and the test average phase metric is positive, andthe decreasing trend in the user engagement metric indicates an increase in user engagement.

View all claims
  • 4 Assignments
Timeline View
Assignment View
    ×
    ×