Using classified text and deep learning algorithms to identify drafting risks in a document and provide early warning

• US 9,754,206 B1
• Filed: 01/23/2017
• Issued: 09/05/2017
• Est. Priority Date: 07/01/2016
• Status: Expired due to Fees

• Current Assignee: IP3 2017, Series 200 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Intraspexion, Inc.
• Inventors: Brestoff, Nelson E.
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US15/413,309
• Time in Patent Office: 225 Days
• Field of Search: 706 25
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Deep learning is used to identify a potential risk that a contract will be unenforceable due to a drafting error whereby one or more terms or phrases are ambiguous. The system involves mining and using existing classifications of data (e.g., from a litigation database) to train one or more deep learning algorithms, and then examining the internal electronic drafts of contracts with the trained algorithm, to generate a scored output that will enable enterprise personnel to be alerted to the ambiguity risks and take action in time to prevent the risks from resulting in harm to the enterprise.

Claims

1. A method of using classified text and deep learning algorithms to identify ambiguity risks in a draft document intended to create contractual obligations and provide early warning comprising:

obtaining one or more training datasets for textual data corresponding to court opinions in which a contract is the subject of a dispute and one of the issues is whether the language of a provision is ambiguous;

training one or more deep learning algorithms using said one or more training datasets;

obtaining an internal electronic draft of a contract;

applying said one or more deep learning algorithms to said internal electronic draft contract to identify and report any one of said one or more ambiguous terms or phrases and related data;

determining if said identified one of said one or more ambiguous terms or phrases is a false positive or a true positive;

re-training said one or more deep learning algorithms if said identified one of said one or more ambiguous terms or phrases is a false positive; and

saving said internal electronic draft contract in a true positive database if said identified one of said one or more ambiguous terms or phrases is a true positive.

2. The method of claim 1, wherein said one or more deep learning algorithms is a framework for natural language processing of text.

3. The method of claim 1, wherein said one or more deep learning algorithms is a recurrent neural network with a multiplicity of layers and various features, including but not limited to long short-term memory or gated recurrent units.

4. The method of claim 1, wherein the one or more deep learning algorithms have been trained with different classifiers using previously classified data sourced and provided by a subject matter expert to become models for ambiguous terms and phrases.

5. The method of claim 1, wherein each one of said one or more deep learning algorithms has also been trained with one or more datasets unrelated to the ambiguous terms or phrases.

6. The method of claim 5, wherein each one of said one or more deep learning algorithms scores the data for accuracy with the deep learning model classification of the data.

7. The method of claim 6, wherein said report of said one of said one or more ambiguous terms or phrases comprises providing the scores and related data to one or more designated users through a graphical user interface.

8. The method of claim 7, wherein the report may be limited to scores which surpass a specified threshold associated with each of said one or more deep learning algorithms.

9. The method of claim 8, further comprising retraining said one or more deep learning algorithms if said identified one of said one or more ambiguous terms or phrases is a true positive or a false positive.

10. The method of claim 1, wherein said one or more training datasets is obtained by mining one or more litigation databases for court opinions in which a contract is the subject of a dispute and the issue is whether the contract is unenforceable because the language of a provision is ambiguous.

11. A method of using classified text and deep learning algorithms to identify ambiguity risks in contractual documents and provide early warning comprising:

obtaining one or more training datasets for textual data corresponding to court opinions for which ambiguity is a risk classification, wherein said risk classification comprises one or more ambiguous terms or phrases;

training one or more deep learning algorithms using said one or more training datasets;

obtaining an internal electronic contractual document;

applying said one or more deep learning algorithms to said internal electronic contractual document to identify and report any one of said one or more ambiguous terms or phrases;

determining if said identified one of said one or more ambiguous terms or phrases is a false positive or a true positive; and

re-training said one or more deep learning algorithms if said identified one of said one or more ambiguous terms or phrases is a false positive.

12. The method of claim 11, wherein the one or more deep learning algorithms have been trained with different classifiers using previously classified data sourced and provided by a subject matter expert to become models for ambiguous terms and phrases.

13. The method of claim 11, wherein each one of said one or more deep learning algorithms has also been trained with one or more datasets unrelated to the ambiguous terms or phrases.

14. The method of claim 13, wherein each one of said one or more deep learning algorithms scores the data for accuracy with the deep learning model classification of the data.

15. The method of claim 13, wherein the report may be limited to scores which surpass a specified threshold associated with each of said one or more deep learning algorithms.

16. The method of claim 11, wherein said one or more training datasets is obtained by mining one or more litigation databases for court opinions in which a contract is the subject of a dispute and the issue is whether the language of a provision is ambiguous.

17. The method of claim 11, wherein said one or more training datasets is obtained by mining one or more litigation databases for court opinions in which a patent is the subject of a dispute and the issue is whether a disputed term or phrase is ambiguous.