Using natural language processing for detection of intended or unexpected application behavior

• US 10,015,181 B2
• Filed: 05/19/2016
• Issued: 07/03/2018
• Est. Priority Date: 05/19/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Horesh, Lior, Horesh, Raya, Pistoia, Marco, Tripp, Omer
• Primary Examiner(s): Do, Khang
• Application Number:
  US15/159,526
• Publication Number:
  US 20170339175A1
• Time in Patent Office: 775 Days
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

Detection of unintended application behaviors, where natural language processing (NLP) techniques are used to analyze the application, and specifically its graphical user interface (GUI), and construct an acceptable (or expected) list per-context actions. Actions executed by the application in a given context that do not fall within the list are flagged as unexpected (or anomalous).

Claims

1. A method of using natural language processing (NLP) for detecting unintended computer application behavior to construct a statistical model to determine for an application, given the application'"'"'s graphical user interface (GUI) and unlabeled text, whether the application is exhibiting unintended or abnormal behavior comprising steps of:

instrumenting the application such that user interface UI transitions are recorded as an acceptable list per-context action and security-critical operations are monitored for training the system;

capturing from the instrumentation as UI transitions occur for obtaining text associated with an enabled UI widget;

forming a sequence of NLP textural content from interactions with the application and interleaving into the sequence security-relevant operations that occur between UI transitions;

analyzing the suffix leading to the operation as a security-critical operation is about to execute to determine whether the operation is expected to occur at this point in the execution based upon the acceptable list per-context actions and its GUI, and

qualifying whether the application interaction is consistent with expected behavior based on identification of an intended semantic from the acceptable list per-context actions, and if not, raise a warning flag.

2. The method of claim 1, wherein the recorded transitions include changing the selection of the enabled UI widget and the monitored security-critical operation include sending text messages or HTTP messages, creating files, or reading sensitive identifiers.

3. The method of claim 1, wherein the steps are applied online.

4. The method of claim 1, wherein the steps are applied offline.

5. The method of claim 1 wherein the capturing comprises OCR (optical character recognition) or code analysis to obtain the text.

6. The method of claim 1, wherein the analyzing based on NLP is specifically based on the GUI.

7. The method of claim 1, wherein the application is performed on a mobile device.

8. A non-transitory computer readable medium having computer readable program for using natural language processing (NLP) for detecting unintended computer application behavior to construct a statistical model to determine for an application, given the application'"'"'s graphical user interface (GUI) and unlabeled text, whether the application is exhibiting unintended or abnormal behavior, program comprising:

instrumenting the application such that user interface (UI) transitions are recorded as an acceptable list per-context action and security-critical operations are monitored for training the system;

capturing from the instrumentation as UI transitions occur for obtaining text associated with an enabled UI widget;

forming a sequence of NLP textural content from interactions with the application and interleaving into the sequence security-relevant operations that occur between UI transitions;

analyzing the suffix leading to the operation as a security-critical operation is about to execute to determine whether the operation is expected to occur at this point in the execution based upon the acceptable list per-context actions and its GUI, and

qualifying whether the application interaction is consistent with expected behavior based on identification of an intended semantic from the acceptable list per-context actions, and if not, raise a warning flag.

9. The non-transitory computer readable medium of claim 8, wherein the recorded transitions include changing the selection of the enabled UI widget and the monitored security-critical operation include sending text messages or HTTP messages, creating files, or reading sensitive identifiers.

10. The non-transitory computer readable medium of claim 8, wherein the steps are applied online.

11. The non-transitory computer readable medium of claim 8, wherein the steps are applied offline.

12. The non-transitory computer readable medium of claim 8, wherein the capturing comprises OCR (optical character recognition) or code analysis to obtain the text.

13. The non-transitory computer readable medium of claim 8, wherein the analyzing based on NLP is specifically based on the GUI.

14. The non-transitory computer readable medium of claim 8, wherein the application is performed on a mobile device.