Deviation finder

• US 10,761,828 B2
• Filed: 01/06/2017
• Issued: 09/01/2020
• Est. Priority Date: 01/06/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kou, Lei, Li, Zhongyuan
• Primary Examiner(s): Zhen, Wei Y
• Assistant Examiner(s): Thatcher, Clint
• Application Number:
  US15/400,029
• Publication Number:
  US 20180196657A1
• Time in Patent Office: 1,334 Days
• Field of Search: 717168, 717128
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 11/362 Software debugging
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57

Abstract

Automatic detection of software program regression points enables developers to more quickly and accurately diagnose and solve issues in changes to software. Traces of instructions are collected from before and after a change to a program or settings thereto on a computing device and are compared to identify differences between the two traces. Areas of differences are ranked and returned to identify the point(s) of regression in the traces to alert a user of programs of behaviors to avoid and/or a developer a point to focus efforts on to patch the responsible program or update other programs around the responsible application.

Claims

1. A method for automatic program regression detection, comprising:

gathering, on a user device, system events into a first trace;

in response to observing an update to the user device, gathering system events occurring after the update into a second trace;

in response to observing a regression on the user device, comparing the first trace to the second trace to identify deviations in system events between the first trace and the second trace;

ranking the identified deviations based on their respective run lengths, wherein the identified deviations ranked based on their respective run lengths include at least a first predetermined number of different system events occurring over a run of a second predetermined number of system events;

identifying a start of a longest run of the deviations in system events in the second trace;

indicating the start of the longest run as a regression point;

identifying a program associated with the regression point; and

transmitting the regression point to a support services application associated with the identified program.

2. The method of claim 1, wherein the system events gathered into the first trace and the system events occurring after the update gathered into the second trace include instructions observed on a processor of the user device.

3. The method of claim 2, wherein the instructions observed on the processor are normalized into instructional blocks according to a function index.

4. The method of claim 3, wherein indicating the regression point includes indicating a function from the function index associated with the instructional block in which the regression point occurs.

5. The method of claim 1, wherein multiple starts of runs of deviations in system events in the second trace are provided in a ranked list based on lengths of the runs.

6. The method of claim 1, wherein the support services application is run on a remote device associated with a developer of the program associated with the regression point.

7. The method of claim 1, wherein the system events include local processes run on the user device and remote processes run on a remote device.

18. The method of claim 1, wherein the identified deviations ranked based on their respective run lengths include system events shared between the first trace and the second trace.

19. The method of claim 1, further comprising:

determining that the update is a first update type and gathering system events occurring after the update into the second trace; and

determining that the update is a second update type and not gathering system events occurring after the update.

8. A system for automatic program regression detection, comprising:

a processor; and

a memory storage device, including computer code that when executed by the processor is operable to provide;

an instruction collector, configured to observe instructions executed by the processor and organize the instructions observed into a plurality of traces;

an instruction normalizer, in communication with the instruction collector, configured to group pluralities of instructions into instructional blocks according to a function index; and

a trace comparator, in communication with the instruction normalizer, configured to identify deviations in the instructional blocks of a first trace of the plurality of traces from the instructional blocks of a second trace of the plurality of traces, wherein the trace comparator is further configured to;

rank the identified deviations based on their respective run lengths, wherein the identified deviations ranked based on their respective run lengths include at least a first predetermined number of different system events occurring over a run of a second predetermined number of system events;

indicate the start of the longest run of the identified deviations as a regression point;

identify a program associated with the regression point; and

transmit the regression point to a support services application associated with the identified program.

9. The system of claim 8, wherein the instructions are organized into the plurality of traces according to execution sequence for a given process on the processor.

10. The system of claim 9, wherein the processor is one of a plurality of processors, the instructions are organized into the plurality of traces based on a given processor of the plurality of processors on which the instructions are executed.

11. The system of claim 8, wherein a new trace of the plurality of traces is generated in response to a system update affecting at least one of:

a software update;

a hardware update;

a software settings update; and

a hardware settings update.

12. The system of claim 8, wherein the instruction normalizer includes a sequence of machine-level instructions in a given instructional block based on prior observations of the sequence of machine-level instructions.

13. The system of claim 8, wherein the deviations identified by the trace comparator are ranked according to a number of different instructional blocks between the first trace and the second trace occurring in a deviation run.

14. A computer readable storage device including instructions that when executed by a processing unit perform steps for automatic program regression detection, comprising:

gathering, on a user device, system events into a first trace;

in response to observing an update to the user device, gathering system events occurring after the update into a second trace;

in response to observing a regression on the user device, comparing the first trace to the second trace to identify deviations in system events between the first trace and the second trace;

ranking the identified deviations based on the respective run lengths of each identified deviation, wherein the identified deviations ranked based on their respective run lengths include at least a first predetermined number of different system events occurring over a run of a second predetermined number of system events;

identifying a start of a longest run of the deviations in system events in the second trace;

indicating the start of the longest run as a regression point;

identifying a software program associated with the regression point; and

transmitting the regression point to a support services application associated with the identified software program.

15. The computer readable storage device of claim 14, wherein the system events gathered into the first trace and the system events occurring after the update gathered into the second trace include instructions observed on a processor of the user device and are normalized into instructional blocks according to a function index.

16. The computer readable storage device of claim 14, wherein multiple starts of runs of deviations in system events in the second trace are provided in a ranked list based on lengths of the runs.

17. The computer readable storage device of claim 14, wherein the system events include local processes run on the user device and remote processes run on a remote device.