Artificial intelligence expert system for anomaly detection

• US 10,013,655 B1
• Filed: 03/28/2016
• Issued: 07/03/2018
• Est. Priority Date: 03/11/2014
• Status: Active Grant

• Current Assignee: Applied Underwriters, Inc. (Berkshire Hathaway Incorporated)
• Original Assignee: Applied Underwriters, Inc. (Berkshire Hathaway Incorporated)
• Inventors: Clark, David Alan
• Primary Examiner(s): Nguyen, Tien C
• Application Number:
  US15/081,991
• Time in Patent Office: 827 Days
• Field of Search: 705 4, 705 3, 705 2, 705 39, 705 40, 705 37, 235380, 235379, 714 26, 714 48, 714 3814, 714 33, 714 25, 714127, 717168, 717170, 717120, 717124, 706 45, 706 46, 706 50, 706 20, 706 12, 726 9, 726 10, 726 25, 707692, 707755, 707917, 709224, 709223, 709226, 703 27, 715500, 386248
• CPC Class Codes: G06F 16/382 using citations hypermedia G06F16/94
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

An artificial intelligence expert system detects an anomaly between an application and a publication provided by an applicant in a given class. The system calculates an application score then uses a decision tree to determine a publication score based on the presence or absence of tokens in the publication. An anomaly is detected when the ratio of the application score to the publication score is greater than a threshold value. The decision tree is trained using prior applications in the same class. The publication score of each leaf node of the decision tree is set equal to an average application score of the prior applications associated with the leaf node.

Claims

1. A computer implemented artificial intelligence expert system for anomaly detection comprising:

a) an application database comprising application data from an applicant, said application data comprising;

i) a citation to a publication by said applicant;

ii) a class of said applicant within a classification scheme; and

iii) an application representation by said applicant about itself;

b) a decision tree database comprising a decision tree associated with said class, said decision tree comprising;

i) a token from a set of confirmed tokens associated with said class, said token being associated with a branch node of said decision tree;

ii) a publication score associated with a first leaf node of said decision tree, said first leaf node being on a yes-branch of said branch node, said yes-branch being associated with said token being in said publication by said applicant; and

iii) a fail mode associated with a second leaf node of said decision tree, said second leaf node being on a no-branch of said branch node, said no-branch being associated with said token not being in said publication by said applicant;

c) a publication database comprising said publication, said publication comprising a public representation by said applicant about itself, said publication being retrievable by a member of the public using said citation;

d) a computer implemented application scoring system comprising computer readable instructions stored on a permanent memory operable to physically cause a microprocessor within said application scoring system to;

i) read in from said application database said application representation; and

ii) calculate an application score based on said application representation using an application scoring algorithm, said application score being a loss function for said applicant;

e) a computer implemented publication scoring system comprising computer readable instructions stored on a permanent memory operable to physically cause a microprocessor within said publication scoring system to;

i) read in from said application database;

1) said citation to a publication by said applicant; and

2) said class of said applicant;

ii) query said decision tree database using said class to retrieve from said decision tree database said decision tree associated with said class and said set of confirmed tokens associated with said class;

iii) query said publication database using said citation and said set of confirmed tokens associated with said class to retrieve from said publication database an indication of the presence or absence of each one of said confirmed tokens in said publication; and

iv) execute said decision tree based at least in part on the presence or absence of said token in said publication, said execution to either;

1) determine an applicant publication score;

or

2) detect said fail mode as a classification anomaly indicating that said applicant may not be in said class;

f) a computer implemented representation anomaly system comprising computer readable instructions stored on a permanent memory operable to physically cause a microprocessor within said representation anomaly system to;

i) read in said application score from said application scoring system;

ii) read in said applicant publication score from said publication scoring system when said executed decision tree produces an applicant publication score;

iii) calculate an anomaly ratio equal to the ratio of said applicant publication score to said application score;

iv) compare said anomaly ratio to a threshold associated with said class; and

v) detect a representation anomaly when said anomaly ratio is greater than said threshold, said representation anomaly indicating that said application representation by said applicant about itself may be inaccurate; and

g) a computer implemented modeling engine comprising;

i) a training database comprising;

1) a plurality of prior application records from a plurality of prior applicants, wherein each of said prior application records comprises;

a) a citation to a prior publication by a prior applicant;

b) a class of said prior applicant within said classification scheme; and

c) an application representation by said prior applicant about itself;

ii) a computer implemented prior application scoring system comprising computer readable instructions stored on a permanent memory operable to physically cause a microprocessor within said prior scoring system to;

1) read in said application representations of said prior applicants; and

2) calculate a prior application score for each of said prior applicants using said application scoring algorithm;

iii) a prior publication database comprising said prior publications wherein each of said prior publications comprises a public representation by a prior applicant about itself, said prior publications being retrievable by a member of the public using said prior citations; and

iv) a machine learning system comprising computer readable instructions stored on a permanent memory operable to physically cause a microprocessor within said machine learning system to;

1) query said training database using said class of said applicant to identify and count prior application records in said class of said applicant;

2) read in from said training database said prior citations in said prior applications in said class of said applicant;

3) query said prior application scoring system using said class of said applicant to identify prior application scores for prior application records in said class of said applicant;

4) read in from said prior scoring system said prior application scores for prior application records in said class of said applicant;

5) query said prior publication database using said class of said applicant to identify prior publications by said prior applicants in said class of said applicant;

6) identify the presence or absence of one or more prospective tokens in said prior publications by said prior applicants in said class of said applicant;

7) read in from said prior publications database said presence or absence of said one or more prospective tokens in said prior publications by said prior applicants in said class of said applicant;

8) construct said decision tree associated with said class of said applicant such that;

a) each branch node of said decision tree associated with said class of said applicant is associated with one of said prospective tokens; and

b) each leaf node of said decision tree associated with said class of said applicant;

i) is associated with a subset of said prior application records in said class of said applicant and is either;

1) assigned a publication score equal to an average of the prior application scores determined for each of said prior application records in said subset associated with said leaf node;

or

2) assigned said fail mode when there are none of said prospective tokens in said prior application records in said subset associated with said leaf node; and

9) construct said set of confirmed tokens consisting of the prospective tokens that are associated with the branch nodes of said decision tree associated with said class of said applicant.

2. The computer implemented artificial intelligence expert system of claim 1 wherein the number of branch nodes in said decision tree associated with said class of said applicant is not more than 1/50 of said counted number of prior application records in said class of said applicant.

3. The computer implemented artificial intelligence expert system of claim 1 wherein said machine learning system further comprises computer readable instructions stored on a permanent memory operable to physically cause a microprocessor within said machine learning system to:

a) calculate a prior anomaly ratio for each of said prior application records in said class of said applicant, said prior anomaly ratio being equal to the ratio of the publication score for the leaf node associated with said prior application record and the prior application score associated with said prior application record;

b) determine an anomaly ratio distribution for said prior anomaly ratios; and

c) set said threshold associated with said class equal to the anomaly ratio of a predetermined value of the upper tail of said anomaly ratio distribution.

4. The computer implemented artificial intelligence expert system of claim 3 where said predetermined value of the upper tail said of said distribution is 0.10 or less.