Automated analysis system for modeling online business behavior and detecting outliers

• US 8,832,265 B2
• Filed: 05/01/2012
• Issued: 09/09/2014
• Est. Priority Date: 05/01/2012
• Status: Active Grant

• Current Assignee: Acoustic, L.P.
• Original Assignee: International Business Machines Corporation
• Inventors: Kozine, Mikhail Borisovich, Wenig, Robert I., Powell, Travis Spence
• Primary Examiner(s): VU, VIET DUY
• Application Number:
  US13/461,168
• Publication Number:
  US 20130297767A1
• Time in Patent Office: 861 Days
• Field of Search: 709/217, 709/219, 709/223, 709/224, 709/227, 709/228, 719/318
• US Class Current: 709/224
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06Q 10/0639 Performance analysis of employees; Performance analysis of enterprise or organisation operations

Abstract

A two-module system is created for automated web activity monitoring. A model is generated and model outliers are identified by the first module of the system. Reports are generated that identify the events based on their significance to the outliers. The model may be automatically and periodically regenerated for different historical time periods of the web sessions. New groups of events may be periodically extracted from new web sessions and applied to the previously generated model by the second module of the system. Model outliers may be identified from the new groups of events. The new events may be analyzed and reported to a web session operator based on their contribution to any identified outliers. Even if no outliers are detected, the new events having a most significant impact on web session operating conditions may be identified and reported in real-time.

Claims

1. A method, comprising:

identify, by a processing device, events for web sessions, wherein the events comprise network data including webpages, requests, and/or responses sent over a network between a web server and user devices during the web sessions; and

user inputs entered at user devices for interacting with the webpages;

generating, by the processing device, a model from the events;

identifying, by the processing device, outliers against the model; and

generating, by the processing device, reports identifying the events based on a significance of the events to the outliers.

2. The method of claim 1, further comprising:

replaying the web sessions by synchronizing rendering of at least some of the network data associated with the outliers with replay of at least some of the user inputs associated with the outliers in substantially a same order as previously occurring during the web sessions.

3. The method of claim 1, further comprising:

automatically extracting, by the processing device, new events for new web sessions;

generating, by the processing device, transformed data by performing a modeling transformation with the new events;

calculating, by the processing device, a distance of the transformed data to the model;

determining, by the processing device, whether the transformed data represent an outlier; and

generating, by the processing device, at least one of the reports identifying outlier status of the new events and a list of top contributing new events to the distance of the transformed data to the model.

4. The method of claim 3,

wherein the top contributing new events comprise error messages generated by a web application during the web sessions.

5. The method of claim 3, further comprising;

identifying, by the processing device, a number of occurrences for each of the events for timestamp periods;

identifying, by the processing device, model distances of the transformed data for the timestamp periods; and

identifying, by the processing device, the outliers based on a comparison of the model distances to a threshold value.

6. The method of claim 1, further comprising:

identifying, by the processing device, some of the events as black-listed events; and

removing, by the processing device, the black listed events from the reports.

7. The method of claim 6 wherein the black-listed events comprise events known to deviate from a normal operation of the web sessions.

8. The method of claim 1, further comprising removing, by the processing device, a group of the events identified in a configuration file from the events used for generating the model, wherein the group of the events comprise activities known to deviate from a normal operation of the web sessions.

9. The method of claim 1, wherein

the reports identify the user inputs having a largest impact on the outliers.

10. The method of claim 1, wherein the model comprises a multivariate model for the web sessions.

11. The method of claim 1, further comprising:

identifying, by the processing device, Document Object Model (DOM) changes for the webpages displayed during the web sessions;

identifying, by the processing device, a number of occurrences of the DOM changes; and

using, by the processing device, the number of occurrences of the DOM changes to generate the model.

12. An apparatus, comprising:

a monitoring system configured to;

capture network data for the web sessions, wherein the network data comprises webpages, requests, and/or responses sent over a network between a web server and user devices during the web sessions; and

capture user interface events for the web sessions, wherein the user interface events comprise user inputs entered at the user devices for interacting with the webpages;

a memory configured to archive web session events for web sessions, wherein the web session events include the network data and the user interface events; and

logic circuitry configured to;

extract the web session events from the memory;

generate a model for the web sessions from the web session events; and

identify how the web session events impact the web sessions based on the model for the web sessions.

13. The apparatus of claim 12, further comprising:

describing with the model a web session customer experience data as a single point in multivariate space of principal components;

calculating a distance of transformed data points associated with the web session events to the model; and

identifying the transformed data points with distances outside of a statistically significant threshold as potential outliers for further analysis.

14. The apparatus of claim 12, wherein the logic circuitry is further configured to:

use the model to identify outliers for the web sessions; and

generate charts for the web session events having most significant contributions to the outliers.

15. The network monitoring system of claim 12 wherein the logic circuitry is further configured to:

use the model to identify outliers for the web sessions;

identify the network data and user interface events associated with the outliers; and

replay the web sessions by synchronizing rendering of at least some of the network data associated with the outliers with replay of at least some of the user interface events associated with the outliers in substantially a same order as previously occurring during the web sessions.

16. A method, comprising:

receiving network events for a network session from a network monitor, the network events including webpage data transmitted over a network between a web server and a user device during the network session;

sending the network events to a session archive;

receiving user interface events associated with the network session and entered from the user device, the user interface events including user inputs for interacting with the webpage data;

sending the user interface events to the session archive;

generating a model for the network session from the network events and the user interface events in the session archive; and

using the model to identify how the network events and user interface events impact operation of the network session.

17. The method of claim 16, further comprising:

identifying outliers for the model; and

identifying the network events and user interface events having a highest contribution to the outliers.

18. The method of claim 16, further comprising replaying the network session by synchronizing rendering of at least some of the webpage data with replay of at least some of the user interface events associated with the outliers.

19. The method of claim 16, further comprising:

describing the user interface events as a single point in multivariate space of principal components associated with corresponding timestamps;

aggregating occurrences of the network events and user interface events for timestamp periods; and

generating the model based on the occurrences of the network events and user interface events for the timestamp periods.

20. The method of claim 16, further comprising:

identifying outliers for the model;

identifying time stamps associated with the outliers;

identifying a portion of the network session associated with the time stamps; and

replaying the network events and user interface events for the portion of the network session.

21. The method of claim 16, further comprising:

identifying duplicate network events or user interface events while generating the model; and

disabling one of the duplicate network events from being captured during the network session.