Systems, methods and computer program products for building a database associating N-grams with cognitive motivation orientations

• US 10,133,734 B2
• Filed: 02/03/2016
• Issued: 11/20/2018
• Est. Priority Date: 07/30/2012
• Status: Active Grant

• Current Assignee: Weongozi Inc.
• Original Assignee: Weongozi Inc.
• Inventors: Rose Charvet, Shelle, Tschichholz, Michael Horst, Busemann, Stephan, Steffen, Jorg, Rose, Jonathan Scott
• Primary Examiner(s): Yen, Eric
• Application Number:
  US15/014,747
• Publication Number:
  US 20160239479A1
• Time in Patent Office: 1,021 Days
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F

Abstract

Computer-implemented methods can transform a corpus of meaningful text sequences into a generalized computer-usable repository of neurolinguistic information that can be applied by one or more computer systems. The computer system(s) can use the neurolinguistic information to neurolinguistically analyze meaningful text sequences to derive statistical information and identify dominant cognitive motivation orientations expressed in those text sequences. The identified dominant cognitive motivation orientations can be used to improve the efficacy of both human-generated and machine-generated communications. The computer system(s) thereby transform a meaningful text sequence into actionable information about the dominant cognitive motivation orientation(s) of the author of that text sequence within the context in which the text sequence was composed. Computer systems and computer-program products for implementing the methods are also described.

Claims

1. A computer-implemented method for building an analysis database associating each of a plurality of n-grams with corresponding respective cognitive motivation orientations, comprising:

receiving a training corpus of training documents in electronic form;

each training document comprising a plurality of meaningfully arranged words;

each training document having at least one annotated word sequence therein;

wherein within each training document, each particular annotated word sequence is annotated with a corresponding word-sequence-level annotation identifying at least one cognitive motivation orientation that is associated with that particular annotated word sequence;

for each training document;

for each annotated word sequence in that particular training document;

extracting n-grams overlapping that particular annotated word sequence;

and

associating each extracted n-gram with the at least one cognitive motivation orientation associated with that particular annotated word sequence;

generating a set of indicator candidate n-grams wherein;

each indicator candidate n-gram represents all instances of a particular n-gram in the training corpus for which at least one instance of that particular n-gram was extracted from any annotated word sequence in any training document;

each indicator candidate n-gram being associated with every cognitive motivation orientation that is associated with at least one instance of the particular n-gram represented by that particular indicator candidate n-gram;

applying at least one relevance filter to each indicator candidate n-gram in the set of indicator candidate n-grams to obtain a set of indicator n-grams, wherein;

the set of indicator n-grams is a subset of the set of indicator candidate n-grams, so that each indicator n-gram corresponds to only one indicator candidate n-gram and thereby each indicator n-gram represents all instances of a corresponding particular n-gram in the training corpus for which at least one instance of that particular n-gram was extracted from any annotated word sequence in any training document;

each indicator n-gram is associated with only a single cognitive motivation orientation; and

each indicator n-gram has, as its associated single cognitive motivation orientation, that single cognitive motivation orientation with which the instances of the particular n-gram represented by that particular indicator n-gram are most frequently associated;

permitting the analysis database to support enhanced computer-implemented neurolinguistic analysis of a text sequence to identify cognitive motivation orientations expressed within the text sequence quickly, objectively and consistently.

2. The method of claim 1, wherein applying at least one relevance filter to each indicator candidate n-gram in the set of indicator candidate n-grams to obtain a set of indicator n-grams comprises:

excluding from the set of indicator n-grams those indicator candidate n-grams for which there is no single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus;

and further comprises at least one of;

(a) excluding from the set of indicator n-grams those indicator candidate n-grams for which a predetermined minimum multiple condition is not satisfied, wherein the predetermined minimum multiple is condition satisfied in respect of a particular indictor candidate n-gram only if;

(A) there is a single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus, wherein instances of the particular n-gram represented by that particular indicator candidate n-gram appear p times in the training corpus in association with the single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus;

(B) there is at least one cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are second-most frequently associated in the training corpus, wherein instances of the particular n-gram represented by that particular indicator candidate n-gram appear q times in the training corpus in association with the at least one cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are second-most frequently associated in the training corpus; and

(C) p exceeds the value resulting from multiplying q by a predetermined multiplier value r;

(b) excluding from the set of indicator n-grams those indicator candidate n-grams for which p is less than a predetermined minimum number; and

(c) excluding from the set of indicator n-grams those indicator candidate n-grams for which a calculated percentage, the calculated percentage being the value obtained by;

dividing p by a total number of appearances in the training corpus of instances of the particular n-gram represented by the particular indicator candidate n-gram; and

multiplying the resulting quotient by 100.

3. The method of claim 1, further comprising assigning a confidence weight to each indicator n-gram.

4. The method of claim 3, wherein:

assigning a confidence weight to each indicator n-gram comprises assigning to each indicator n-gram a confidence weight equal to p divided by a total number of appearances in the training corpus of instances of the particular n-gram represented by that particular indicator n-gram;

where instances of the particular n-gram represented by that particular indicator n-gram appear p times in the training corpus in association with the single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator n-gram are most frequently associated in the training corpus.

5. The method of claim 3, further comprising building a respective normalized dominance threshold associated with each of the cognitive motivation orientations.

6. The method of claim 5, wherein building a respective normalized dominance threshold associated with for each of the cognitive motivation orientations comprises:

receiving a tuning corpus of tuning documents;

each tuning document comprising a plurality of meaningfully arranged words;

each tuning document being annotated with a respective document-level annotation, wherein the respective document-level annotation identifies a dominant cognitive motivation orientation set assigned to the particular tuning document annotated by that particular document-level annotation;

obtaining, for each tuning document, respective document raw confidence weight scores associated with every cognitive motivation orientation expressed in that particular tuning document by, for each tuning document;

for each indicator n-gram;

identifying each instance of a matching n-gram appearing in that particular tuning document, wherein the matching n-gram matches the particular indicator n-gram; and

for each instance of a matching n-gram, incrementing a document raw confidence weight score associated with the cognitive motivation orientation that is associated with that particular indicator n-gram matched by the matching n-gram, wherein the incrementing is according to the confidence weight assigned to that particular n-gram;

for each tuning document, normalizing the document raw confidence weight scores for each cognitive motivation orientation expressed in that particular tuning document by;

dividing the raw confidence weight score associated with each cognitive motivation orientation expressed in that particular tuning document by a number of tokens contained in that particular tuning document to assign normalized document confidence weight scores to each cognitive motivation orientation expressed in that particular tuning document; and

selecting, for each cognitive motivation orientation, a normalized dominance threshold, wherein the normalized dominance threshold minimizes a number of incorrectly classified tuning documents, wherein;

incorrect classification of a given tuning document with respect to a particular cognitive motivation orientation is a condition selected from the group consisting of;

(a) the normalized document confidence weight score associated with that cognitive motivation orientation with respect to that particular tuning document exceeds the normalized dominance threshold but that cognitive motivation orientation is absent from the document-level annotation annotating that particular tuning document; and

(b) the normalized document confidence weight score associated with that cognitive motivation orientation with respect to that particular tuning document is less than or equal to the normalized dominance threshold but that cognitive motivation orientation is present in the document-level annotation annotating that particular tuning document.

7. A database-building data processing system configured for building an analysis database associating each of a plurality of n-grams with corresponding respective cognitive motivation orientations, the system comprising:

a host computer with memory and at least one processor coupled to the memory; and

a database-building module, the database-building module comprising program code that, when executed in the memory of the host computer;

receives a training corpus of training documents, wherein;

each training document comprises a plurality of meaningfully arranged words in electronic form;

each training document has at least one annotated word sequence therein;

wherein within each training document, each particular annotated word sequence is annotated with a corresponding word-sequence-level annotation identifying at least one cognitive motivation orientation that is associated with that particular annotated word sequence;

for each training document;

for each annotated word sequence in that particular training document;

extracts n-grams overlapping that particular annotated word sequence; and

associates each extracted n-gram with the at least one cognitive motivation orientation associated with that particular annotated word sequence;

generates a set of indicator candidate n-grams wherein;

each indicator candidate n-gram represents all instances of a particular n-gram in the training corpus for which at least one instance of that particular n-gram was extracted from any annotated word sequence in any training document;

each indicator candidate n-gram being associated with every cognitive motivation orientation that is associated with at least one instance of the particular n-gram represented by that particular indicator candidate n-gram;

applies at least one relevance filter to each indicator candidate n-gram in the set of indicator candidate n-grams to obtain a set of indicator n-grams, wherein;

the set of indicator n-grams is a subset of the set of indicator candidate n-grams, so that each indicator n-gram corresponds to only one indicator candidate n-gram and thereby each indicator n-gram represents all instances of a corresponding particular n-gram in the training corpus for which at least one instance of that particular n-gram was extracted from any annotated word sequence in any training document; and

each indicator n-gram is associated with only a single cognitive motivation orientation;

wherein each indicator n-gram has, as its associated single cognitive motivation orientation, that single cognitive motivation orientation with which the instances of the particular n-gram represented by that particular indicator n-gram are most frequently associated;

permitting the analysis database to support enhanced computer-implemented neurolinguistic analysis of a text sequence to identify cognitive motivation orientations expressed within the text sequence quickly, objectively and consistently.

8. The data processing system of claim 7, wherein the program code that, when executed in the memory of the host computer, applies at least one relevance filter to each indicator candidate n-gram in the set of indicator candidate n-grams to obtain a set of indicator n-grams comprises program code that, when executed in the memory of the host computer:

excludes from the set of indicator n-grams those indicator candidate n-grams for which there is no single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus;

and further does at least one of;

(a) excluding from the set of indicator n-grams those indicator candidate n-grams for which a predetermined minimum multiple condition is not satisfied, wherein the predetermined minimum multiple condition is satisfied in respect of a particular indictor candidate n-gram only if;

(A) there is a single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus, wherein instances of the particular n-gram represented by that particular indicator candidate n-gram appear p times in the training corpus in association with the single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus;

(B) there is at least one cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are second-most frequently associated in the training corpus, wherein instances of the particular n-gram represented by that particular indicator candidate n-gram appear q times in the training corpus in association with the at least one cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are second-most frequently associated in the training corpus; and

(C) p exceeds the value resulting from multiplying q by a predetermined multiplier value r;

(b) excluding from the set of indicator n-grams those indicator candidate n-grams for which p is less than a predetermined minimum number; and

(c) excluding from the set of indicator n-grams those indicator candidate n-grams for which a calculated percentage is less than a predetermined minimum percentage, the calculated percentage being the value obtained by;

dividing p by a total number of appearances in the training corpus of instances of the particular n-gram represented by that particular indicator candidate n-gram; and

multiplying the resulting quotient by 100.

9. The data processing system of claim 7, wherein the database-building module further comprises program code that, when executed in the memory of the host computer, assigns a confidence weight to each indicator n-gram.

10. The data processing system of claim 9, wherein the program code that, when executed in the memory of the host computer, assigns a confidence weight to each indicator n-gram comprises program code that, when executed in the memory of the host computer, assigns a confidence weight to each indicator n-gram wherein for each indicator n-gram, the respective confidence weight is equal to p divided by a total number of appearances in the training corpus of instances of the particular n-gram represented by that particular indicator n-gram;

where instances of the particular n-gram represented by that particular indicator n-gram appear p times in the training corpus in association with the single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator n-gram are most frequently associated in the training corpus.

11. The data processing system of claim 9, wherein the database-building module further comprises program code that, when executed in the memory of the host computer, builds a respective normalized dominance threshold associated with each of the cognitive motivation orientations.

12. The data processing system of claim 11, wherein the program code that, when executed in the memory of the host computer, builds a respective normalized dominance threshold associated with each of the cognitive motivation orientations comprises program code that, when executed in the memory of the host computer:

receives a tuning corpus of tuning documents, wherein;

each tuning document comprises a plurality of meaningfully arranged words;

each tuning document being annotated with a respective document-level annotation, wherein the respective document-level annotation identifies a dominant cognitive motivation orientation set assigned to the particular tuning document annotated by that particular document-level annotation;

obtains, for each tuning document, document raw confidence weight scores associated with every cognitive motivation orientation expressed in that particular tuning document by, for each tuning document;

for each indicator n-gram;

identifying each instance of a matching n-gram appearing in that particular tuning document, wherein the matching n-gram matches the particular indicator n-gram; and

for each instance of a matching n-gram, incrementing a document raw confidence weight score associated with the cognitive motivation orientation that is associated with that particular indicator n-gram matched by the matching n-gram, wherein the incrementing is according to the confidence weight assigned to that particular n-gram;

for each tuning document, normalizes the document raw confidence weight scores for each cognitive motivation orientation expressed in that particular tuning document by;

dividing the raw confidence weight scores associated with each cognitive motivation orientation expressed in that particular tuning document by a number of tokens contained in that particular tuning document to assign normalized document confidence weight scores to each cognitive motivation orientation expressed in that particular tuning document; and

selects, for each cognitive motivation orientation, a normalized dominance threshold, wherein the normalized dominance threshold minimizes a number of incorrectly classified tuning documents, wherein;

incorrect classification of a given tuning document with respect to a particular cognitive motivation orientation is a condition selected from the group consisting of;

(a) the normalized document confidence weight score associated with that cognitive motivation orientation, with respect to that particular tuning document, exceeds the normalized dominance threshold but that cognitive motivation orientation is absent from the document-level annotation annotating that particular tuning document; and

(b) the normalized document confidence weight score associated with that cognitive motivation orientation, with respect to that particular tuning document, is less than or equal to the normalized dominance threshold but that cognitive motivation orientation is present in the document-level annotation annotating that particular tuning document.

13. A computer program product for building an analysis database associating each of a plurality of n-grams with corresponding respective cognitive motivation orientations, the computer program product comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code adapted to, when executed by a computer, cause the computer to receive a training corpus of training documents, wherein;

each training document comprises a plurality of meaningfully arranged words in electronic form;

each training document has at least one annotated word sequence therein;

wherein, within each training document, each particular annotated word sequence is annotated with a corresponding word-sequence-level annotation identifying at least one cognitive motivation orientation that is associated with that particular annotated word sequence;

computer readable program code adapted to, when executed by a computer, cause the computer to, for each training document;

for each annotated word sequence in that particular training document;

extract n-grams overlapping that particular annotated word sequence; and

associate each extracted n-gram with the at least one cognitive motivation orientation associated with that particular annotated word sequence;

computer readable program code adapted to, when executed by a computer, cause the computer to generate indicator candidate n-grams wherein;

each indicator candidate n-gram represents all instances of a particular n-gram in the training corpus for which at least one instance of that particular n-gram was extracted from any annotated word sequence in any training document;

each indicator candidate n-gram being associated with every cognitive motivation orientation that is associated with at least one instance of the particular n-gram represented by that particular indicator candidate n-gram;

computer readable program code adapted to, when executed by a computer, cause the computer to apply at least one relevance filter to each indicator candidate n-grams in the set of indicator candidate n-grams to obtain a set of indicator n-grams, wherein;

the set of indicator n-grams is a subset of the set of indicator candidate n-grams, so that each indicator n-gram corresponds to only one indicator candidate n-gram and thereby each indicator n-gram represents all instances of a corresponding particular n-gram in the training corpus for which at least one instance of that particular n-gram was extracted from any annotated word sequence in any training document; and

each indicator n-gram is associated with only a single cognitive motivation orientation;

wherein each indicator n-gram has, as its associated single cognitive motivation orientation, that single cognitive motivation orientation with which the instances of the particular n-gram represented by that particular indicator n-gram are most frequently associated;

permitting the analysis database to support enhanced computer-implemented neurolinguistic analysis of a text sequence to identify cognitive motivation orientations expressed within the text sequence quickly, objectively and consistently.

14. The computer program product of claim 13, wherein the computer readable program code adapted to, when executed by a computer, cause the computer to apply at least one relevance filter to each indicator candidate n-gram in the set of indicator candidate n-grams to obtain a set of indicator n-grams comprises computer readable program code adapted to, when executed by a computer, cause the computer to:

exclude from the set of indicator n-grams those indicator candidate n-grams for which there is no single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus;

and further comprises computer readable program code adapted to, when executed by a computer, cause the computer to do at least one of;

(a) excluding from the set of indicator n-grams those indicator candidate n-grams for which a predetermined minimum multiple condition is not satisfied, wherein the predetermined minimum multiple condition is satisfied in respect of a particular indictor candidate n-gram only if;

(A) there is a single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus, wherein instances of the particular n-gram represented by that particular indicator candidate n-gram appear p times in the training corpus in association with the single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are most frequently associated in the training corpus;

(B) there is at least one cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are second-most frequently associated in the training corpus, wherein instances of the particular n-gram represented by that particular indicator candidate n-gram appear q times in the training corpus in association with the at least one cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator candidate n-gram are second-most frequently associated in the training corpus; and

(C) p exceeds the value resulting from multiplying q by a predetermined multiplier value r;

(b) excluding from the set of indicator n-grams those indicator candidate n-grams for which p is less than a predetermined minimum number; and

(c) excluding from the set of indicator n-grams those indicator candidate n-grams for which a calculated percentage is less than a predetermined minimum percentage, the calculated percentage being the value obtained by;

dividing p by a total number of appearances in the training corpus of instances of the particular n-gram represented by that indicator candidate n-gram; and

multiplying the resulting quotient by 100.

15. The computer program product of claim 13, further comprising computer readable program code adapted to, when executed by a computer, cause the computer to assign a confidence weight to each indicator n-gram.

16. The computer program product of claim 15, wherein the computer readable program code adapted to, when executed by a computer, cause the computer to assign a confidence weight to each indicator n-gram comprises computer readable program code adapted to, when executed by a computer, cause the computer to assign to each indicator n-gram a confidence weight equal to p divided by a total number of appearances in the training corpus of instances of the particular n-gram represented by that particular indicator n-gram;

where instances of the particular n-gram represented by that particular indicator n-gram appear p times in the training corpus in association with the single cognitive motivation orientation with which instances of the particular n-gram represented by that particular indicator n-gram are most frequently associated in the training corpus.

17. The computer program product of claim 15, further comprising computer readable program code adapted to, when executed by a computer, cause the computer to build a respective normalized dominance threshold associated with each of the cognitive motivation orientations.

18. The computer program product of claim 17, wherein the computer readable program code adapted to, when executed by a computer, cause the computer to build a respective normalized dominance threshold associated with each of the cognitive motivation orientations comprises:

computer readable program code adapted to, when executed by a computer, cause the computer to receive a tuning corpus of tuning documents, wherein;

each tuning document comprises a plurality of meaningfully arranged words;

each tuning document being annotated with a respective document-level annotation, wherein the respective document-level annotation identifies a dominant cognitive motivation orientation set assigned to the particular tuning document annotated by that particular document-level annotation;

computer readable program code adapted to, when executed by a computer, cause the computer to obtain, for each tuning document, document raw confidence weight scores associated with every cognitive motivation orientation expressed in that particular tuning document by, for each tuning document;

for each indicator n-gram;

identifying each instance of a matching n-gram appearing in that particular tuning document, wherein the matching n-gram matches the particular indicator n-gram; and

for each instance of a matching n-gram, incrementing a document raw confidence weight score associated with the cognitive motivation orientation that is associated with that particular indicator n-gram matched by the matching n-gram, wherein the incrementing is according to the confidence weight assigned to that particular n-gram;

computer readable program code adapted to, when executed by a computer, cause the computer to, for each tuning document, normalize the document raw confidence weight scores associated with each cognitive motivation orientation expressed in that particular tuning document by;

dividing the raw confidence weight scores associated with each cognitive motivation orientation by a number of tokens in that particular tuning document to assign normalized document confidence weight scores to each cognitive motivation orientation expressed in that particular tuning document; and

computer readable program code adapted to, when executed by a computer, cause the computer to select, for each cognitive motivation orientation, a normalized dominance threshold, wherein the normalized dominance threshold minimizes a number of incorrectly classified tuning documents, wherein;

incorrect classification of a given tuning document with respect to a particular cognitive motivation orientation is a condition selected from the group consisting of;

(a) the normalized document confidence weight score for that cognitive motivation orientation with respect to that particular tuning document, exceeds the normalized dominance threshold but that cognitive motivation orientation is absent from the document-level annotation annotating that particular tuning document; and

(b) the normalized document confidence weight score associated with that cognitive motivation orientation with respect to that particular tuning document is less than or equal to the normalized dominance threshold but that cognitive motivation orientation is present in the document-level annotation annotating that particular tuning document.