Generative text using a personality model

• US 10,437,936 B2
• Filed: 02/01/2018
• Issued: 10/08/2019
• Est. Priority Date: 02/01/2018
• Status: Active Grant

• Current Assignee: Jungle Disk LLC (Apollo Global Management, Inc.)
• Original Assignee: Jungle Disk LLC (Apollo Global Management, Inc.)
• Inventors: DeFelice, Michael
• Primary Examiner(s): Nguyen, Khai N.
• Application Number:
  US15/885,813
• Publication Number:
  US 20190236148A1
• Time in Patent Office: 614 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10L 19/0212 using orthogonal transformation

Abstract

A personality model is created for a population and used as an input to a text generation system. Alternative texts are created based upon the emotional effect of the generated text. Certain words or phrases are “pinned” in the output, reducing the variability of the generated text so as to preserve required information content, and a number of tests provide input to a discriminator network so that proposed outputs both match an outside objective regarding the information content, emotional affect, and grammatical acceptability. A feedback loop provides new “ground truth” data points for refining the personality model and associated generated text.

Claims

1. A system for generating natural language, the system comprising:

a prospect modeling component, operable to correlate known information about a target prospect personality with a quantitative personality model, the quantitative personality model being expressed as a vector indicating a relative expression of a plurality of relatively mutually orthogonal personality traits; and

a neural sequence-to-sequence encoder-decoder, wherein the encoder is operable to deconstruct a source text and represent it as a sequence of weights on a pre-built conditional text model, and wherein the decoder is operable to create a generated text with approximately equal semantic content but differing syntax and word choice,

wherein the quantitative personality model provides an input to the decoder, and

wherein the syntax and word choice of the generated text varies as a function of the relative expression of the quantitative personality model.

2. The system of claim 1, wherein the encoder-decoder further includes an attention mechanism, wherein the attention mechanism provides varying weights to different types of syntax and word choice depending upon the input quantitative personality model.

3. The system of claim 1, further comprising an evaluator coupled to the encoder and decoder,

wherein the evaluator provides a personality score reflecting an association of a language use in the generated text with the input quantitative personality model provided to the decoder,

wherein the personality score is provided to the decoder, and

wherein internal weights associated with the decoder are updated to reinforce high personality association and to discourage lower personality association.

4. The system of claim 1, wherein the prospect modeling component further includes one of a correlator, a named entity recognition component, a disambiguator, a segmentation component, an imputation component, and a categorization component.

5. The system of claim 1, wherein the generated text is provided to a representative of a modeled prospect class, and wherein a response of the representative is used to update the input quantitative personality model.

6. The system of claim 5, wherein the updating of the input quantitative personality model is relative to a measured receptiveness of the representative to the generated text.

7. The system of claim 5, wherein the updating of the input quantitative personality model is relative to an imputed measurement of personality traits.

8. A method of generating natural language, the method comprising:

collecting representative text information attributable to one or more natural persons;

preprocessing and tokenizing the representative text information;

converting the tokenized text information into at least one corresponding input personality embedding, wherein the at least one corresponding input personality embedding indicates a relative expression of a plurality of mutually orthogonal personality traits of a quantitative personality model expressed in the representative text information;

providing a text prompt and the at least one corresponding input personality embedding as inputs to a text prediction layer;

sampling a maximum likelihood next token from the text prediction layer;

outputting a natural language element corresponding to the maximum likelihood next token; and

collecting the output natural language element into a generated text.

9. The method of claim 8, wherein the text prediction layer is a variational autoencoder.

10. The method of claim 8, wherein a syntax and word choice of the generated text varies as a function of the relative expression of the quantitative personality model.

11. The method of claim 8, further including the steps of:

preprocessing and tokenizing the generated text;

converting the tokenized generated text into at least one corresponding output personality embedding, where the at least one corresponding output personality embedding indicates the relative expression of a plurality of relatively mutually orthogonal personality traits;

comparing values of the at least one corresponding input personality embedding with the at least one corresponding output personality embedding to generate a similarity score;

if the similarity score is above a confidence threshold, providing a positive feedback result; and

if the similarity score is below the confidence threshold, providing a negative feedback result.

12. The method of claim 11, further comprising the steps of updating the at least one corresponding input personality embedding based upon one of the positive and the negative feedback result.

13. The method of claim 11, wherein the calculation of the maximum likelihood next token is modified to reinforce high congruence between a relative strength of the personality traits expressed in the generated text and a relative strength of the personality traits expressed in the representative text information.

14. The method of claim 8, wherein the relative expression of the plurality of relatively mutually orthogonal personality traits uses the “

OCEAN”

personality model.

15. The method of claim 8, wherein the one or more natural persons are associated with a modeled prospect class, the modeled prospect class being defined by one or more commonly shared values associated with the relatively mutually orthogonal personality traits.

16. The method of claim 15, wherein the generated text is provided to a representative of the modeled prospect class, and wherein a response of the representative is used to update the input quantitative personality model.

17. A system, including a processor and instructions included in a non-volatile computer-readable medium, wherein the instructions when interpreted by the processor cause the processor to:

collect representative text information attributable to one or more natural persons;

impute a strength of one or more mutually orthogonal personality traits as expressed in the representative text;

divide the one or more natural persons into one or more modeled classes based upon shared relative values of the imputed personality trait;

receive an input text including a first semantic content and a first expressive content, wherein the first expressive content is measured by the imputed personality trait; and

generate an output text that includes the first semantic content and a second expressive content.

18. The system of claim 17, wherein values of the one or more mutually orthogonal personality traits in the second expressive content match values for the imputed strength of the one or more mutually orthogonal personality traits in the one or more modeled class.

19. The system of claim 17, wherein values of the first expressive content are different than values of the second expressive content.

20. The system of claim 17, wherein the generated output text is modified based upon a similarity of a modeled class and the second expressive content.