Latency reduction in feedback-based system performance determination

• US 10,402,180 B2
• Filed: 06/29/2016
• Issued: 09/03/2019
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Ramachandran, Vinod Kumar, Wang, Jing, Swan, Steve, Kasatkin, Victor, Li, Fei, Liu, Zhe, Valle, Alex, Tang, Peng
• Primary Examiner(s): Chowdhury, Ziaul A
• Assistant Examiner(s): Wei, Zengpu
• Application Number:
  US15/196,891
• Publication Number:
  US 20180004497A1
• Time in Patent Office: 1,161 Days
• Field of Search: 717176
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 8/60 Software deployment
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 30/0202 Market predictions or forecasting for commercial activities
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0255 based on user history

Abstract

The present disclosure is directed to a technique to reduce latency in feedback-based system performance determination. A system receives, from an application developer device, indications of an in-application event and a first input value for an application content delivery profile. The system receives, via an interface from an application developed by an application developer and executed by a computing device remote from the data processing system and different from the application developer device, a ping indicative of an occurrence of the in-application event on the computing device. The system merges data from the ping with internal data determined by the data processing system to generate merged data. The system determines a predicted performance for the in-application event and provides an indication of the predicted performance. The system configures, responsive to the indication of the predicted performance, the application content delivery profile with a second input value.

Claims

1. A method of reducing latency in feedback-based system performance determination, comprising:

receiving, by a data processing system comprising one or more processor and memory, from an application developer device, an indication of an in-application event and a first input value for an application content delivery profile;

receiving, by the data processing system, via an interface from an application developed by an application developer and executed by a mobile computing device remote from the data processing system and different from the application developer device, an indication of an occurrence of the in-application event on the mobile computing device;

determining, by the data processing system, a number of installations of the application on other mobile computing devices;

generating, by the data processing system, a first merged data set that merges performance signals for the in-application event across the other mobile computing devices on which the application is installed;

identifying, by the data processing system, a plurality of applications that are similar to the application based on at least one characteristic of the application;

generating, by the data processing system, a second merged data set that merges performance signals for in-application events associated with installations of the plurality of applications;

generating, by the data processing system, a regression model that combines the first merged data set with the second merged data set using a machine learning technique;

determining, by the data processing system, using the regression model, a first predicted performance for the in-application event based on the first input value;

calibrating, by the data processing system, the first predicted performance based on a second predicted performance of the plurality of applications to determine a predicted performance;

providing, by the data processing system, via the interface, an indication of the predicted performance determined based on the regression model and the first input value; and

configuring, by the data processing system, responsive to the indication of the predicted performance, the application content delivery profile with a second input value, execution by the data processing system of the application content delivery profile configured with the second input value causing the data processing system to select content with an indication to install the application.

2. The method of claim 1, comprising:

receiving, by the data processing system, from a second plurality of applications developed by a plurality of application developers, a plurality of indication of occurrences of one or more in-application events; and

merging, by the data processing system, data from the plurality of indications of occurrences with the first merged data set.

3. The method of claim 1, comprising:

configuring, by the data processing system, the application content delivery profile with the second input value greater than the first input value responsive to the predicted performance being less than a threshold.

4. The method of claim 1, comprising:

receiving, by the data processing system, an instruction from the application developer device to configure the application content delivery profile with the second input value, the second input value comprising a same value as the first input value.

5. The method of claim 1, wherein the predicted performance comprises a predicted number of installations for the application.

6. The method of claim 1, comprising:

determining, by the data processing system, the predicted performance for the in-application event based on indications of occurrences of in-application events associated with the plurality of applications.

7. A system to reduce latency in feedback-based system performance determination, comprising:

a data processing system comprising one or more processors and memory to;

receive, from an application developer device, an indications of an in-application event and a first input value for an application content delivery profile;

receive, via an interface from an application developed by an application developer and executed by a mobile computing device remote from the data processing system and different from the application developer device, an indication of an occurrence of the in-application event on the mobile computing device;

determine a number of installations of the application on other mobile computing devices;

generate a first merged data set that merges performance signals for the in-application event across the other mobile computing devices on which the application is installed;

identify a plurality of applications that are similar to the application based on at least one characteristic of the application;

generate a second merged data set that merges performance signals for in-application events associated with installations of the plurality of applications;

generate a regression model that combines the first merged data set with the second merged data set using a machine learning technique;

determine, using the regression model, a first predicted performance for the in-application event based on the first input value;

calibrate the first predicted performance based on a second predicted performance of the plurality of applications to determine a predicted performance;

provide, via the interface, an indication of the predicted performance determined based on the regression model and the first input value; and

configure, responsive to the indication of the predicted performance, the application content delivery profile with a second input value, execution by the data processing system of the application content delivery profile configured with the second input value causing the data processing system to select content with an indication to install the application.

8. The system of claim 7, comprising the data processing system to:

receive, from a second plurality of applications developed by a plurality of application developers, a plurality of indications of occurrences of one or more in-application events; and

merge data from the plurality of indications of occurrences with the first merged data set.

9. The system of claim 7, comprising the data processing system to:

configure the application content delivery profile with the second input value greater than the first input value responsive to the predicted performance being less than a threshold.

10. The system of claim 7, comprising the data processing system to:

receive an instruction from the application developer device to configure the application content delivery profile with the second input value, the second input value comprising a same value as the first input value.

11. The system of claim 7, wherein the predicted performance comprises a predicted number of installations for the application.

12. The system of claim 7, comprising the data processing system to:

determine the predicted performance for the in-application event based on indications of occurrences of in-application events associated with the plurality of applications.

13. The system of claim 7, comprising the data processing system to generate the regression model with:

data received from a first plurality of computing devices that installed the application;

data received from a second plurality of computing devices that installed, responsive to a content item delivered by the data processing system, one or more applications different from the application, the one or more applications matching a characteristic of the application; and

data received from a third plurality of computing devices that installed, responsive to an organic search, the one or more applications different from the application, the one or more applications matching the characteristic of the application.

14. The system of claim 7, comprising the data processing system to:

provide a software development kit to the application developer device that configured the application to transmit an indication to the data processing system responsive to the occurrence of the in-application event.

15. The system of claim 7, comprising the data processing system to:

determine at least one of a type of application or a configuration of an installation of the application;

the data from the indication of the in-application event comprises at least one of a level of the application, a score of the application, or a query input into the application.