Standard exact clause detection

• US 9,805,025 B2
• Filed: 07/13/2015
• Issued: 10/31/2017
• Est. Priority Date: 07/13/2015
• Status: Active Grant

• Current Assignee: DocuSign International (EMEA) Ltd. (DocuSign Incorporated)
• Original Assignee: Seal Software Ltd.
• Inventors: Gidney, Kevin
• Primary Examiner(s): Serrou, Abdelali
• Application Number:
  US14/797,959
• Publication Number:
  US 20170017641A1
• Time in Patent Office: 841 Days
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Embodiments relate to a system and a method for identifying, from contractual documents, (i) standard exact clauses matching clause examples and (ii) non-standard clauses semantically related to but not matching the clause examples. A standard feature data set comprising standard exact clauses matching clause examples is obtained. In addition, a mirror feature data set comprising semantically related clauses of the clause examples is obtained using semantic language analysis, where the mirror feature data set encompasses the standard feature data set. Non-standard clauses are obtained by extracting a difference between the mirror feature data set and the standard exact feature data set.

Claims

1. A non-transitory computer readable medium storing program code for determination of standard exact clauses and non-standard clauses from a plurality of documents, the program code comprising instructions that when executed by a processor cause the processor to:

obtain a primary policy comprising one or more features, a clause example, and a first threshold for use in a semantic language evaluator to generate a plurality of feature replaced clauses by automatically replacing one or more of a plurality of original clauses in a plurality of documents with the one or more features and to compare each of the plurality of feature replaced clauses according to the primary policy to provide a first feature data set comprising standard clauses;

replace, automatically, an available variation of one of the standard clauses in the plurality of documents with a variable;

compare a clause and at least one of the clause example and the variable, the clause obtained from the plurality of documents with the available variation replaced with the variable;

obtain, in response to the comparison, a standard exact clause comprising the clause matching at least one of the clause example and the variable;

obtain a second feature data set encompassing the first feature data set, the second feature data set corresponding to a secondary policy, the secondary policy comprising the one or more features, the clause example, and a second threshold for use in the semantic language evaluator;

obtain a difference data set comprised of a difference between the first feature data set and the second feature data set, the difference data set comprising a non-standard clause, the non-standard clause being semantically related to but not matching the clause example; and

update, automatically in response to obtaining the difference data set, a database to identify the standard exact clause and the non-standard clause from the plurality of documents.

2. The non-transitory computer readable medium of claim 1, wherein the standard exact clause and the at least one of the clause example and the variable comprise exact one or more words in a same order.

3. The non-transitory computer readable medium of claim 1, further comprising instructions when executed by the processor cause the processor to:

determine whether optical character recognition performed on a word is recognizable.

4. The non-transitory computer readable medium of claim 3, further comprising instructions when executed by the processor cause the processor to:

obtain, responsive to determining the optical character recognition performed on the word is not recognizable, a candidate clause in one of the plurality of documents with the available variation replaced with the variable, the candidate clause comprising a first word before the word and a second word after the word; and

determine, responsive to at least one of the clause example and the variable comprising the first word, a third word and the second word in that sequence, the clause to be a candidate standard exact clause.

5. The non-transitory computer readable medium of claim 1, further comprising instructions when executed by the processor cause the processor to:

replace, automatically, available variations of one or more of the standard clauses in the plurality of documents with additional variables; and

compare additional clauses and additional clause examples to obtain additional standard exact clauses, the additional clauses obtained from the plurality of documents with the available variations replaced with the additional variables, the additional standard exact clauses comprising the additional clauses matching respective ones of the additional clause examples and the additional variables.

6. A non-transitory computer readable medium storing program code for determination of standard exact clauses and non-standard clauses from a plurality of documents, the program code comprising instructions that when executed by a processor cause the processor to:

obtain a primary policy comprising one or more features, a clause example, and a first threshold for use in a semantic language evaluator to generate a plurality of feature replaced clauses by automatically replacing one or more of a plurality of original clauses in a document with the one or more features and to compare each of the plurality of feature replaced clauses according to the primary policy to provide a first feature data set comprising standard clauses;

replace, automatically, an available variation of one of the standard clauses in the document with variable;

compare a clause and at least one of the clause example and the variable, the clause obtained from the document with the available variation replaced with the variable;

obtain, in response to the comparison, a standard exact clause comprising the clause matching at least one of the clause example and the variable;

replace, automatically, available variations of one or more of the standard clauses in the document with additional variables;

compare additional clauses and additional clause examples to obtain additional standard exact clauses, the additional clauses obtained from the document with the available variations replaced with the additional variables, the additional standard exact clauses comprising the additional clauses matching respective ones of the additional clause examples and the additional variables;

obtain a secondary policy comprising the one or more features, the clause example, and a second threshold for use in the semantic language evaluator to compare each of the plurality of feature replaced clauses according to the secondary policy to provide a second feature data set encompassing the first feature data set;

obtain a third feature data set comprising the standard exact clause and the additional standard exact clauses;

obtain a difference data set comprised of a difference between the second feature data set and the third feature data set, the difference data set comprising non-standard clauses; and

update, automatically, a database to identify the standard exact clause and the non standard clauses from the document.

7. A computer implemented method for determination of standard exact clauses and non-standard clauses from a plurality of documents, the method comprising:

obtaining a primary policy comprising one or more features, a clause example, and a first threshold for generating a plurality of feature replaced clauses by automatically replacing one or more of the original clauses in the plurality of documents with a feature of the one or more features;

comparing each of the plurality of feature replaced clauses and a clause example using a semantic language evaluator to obtain a first feature data set comprising standard clauses;

automatically replacing an available variation of one of the standard clauses in the plurality of documents with a variable;

comparing a clause and at least one of the clause example and the variable, the clause obtained from the plurality of documents with the available variation replaced with the variable;

obtaining, in response to the comparison, the standard exact clause comprising the clause matching at least one of the clause example and the variable;

obtaining a second feature data set encompassing the first feature data set, the second feature data set corresponding to a secondary policy, the secondary policy comprising the one or more features, the clause example, and a second threshold for use in the semantic language evaluator;

obtaining a difference data set comprised of a difference between the first feature data set and the second feature data set, the difference data set comprising a non-standard clause, the non-standard clause being semantically related to but not matching the clause example; and

updating, automatically in response to obtaining the difference data set, a database to identify the standard exact clause and the non-standard clause from the plurality of documents.

8. The method of claim 7, wherein the standard exact clause and the at least one of the clause example and the variable comprise exact one or more words in a same order.

9. The method of claim 7, further comprising:

determining whether optical character recognition performed on a word is recognizable.

10. The method of claim 9, further comprising:

obtaining, responsive to determining the optical character recognition performed on the word is not recognizable, a candidate clause in one of the plurality of documents with the available variation replaced with the variable, the candidate clause comprising a first word before the word and a second word after the word; and

determining, responsive to at least one of the clause examples and the variable comprising the first word, a third word and the second word in that sequence, the clause to be a candidate standard exact clause.

11. The method of claim 7, further comprising:

automatically replacing available variations of one or more of the standard clauses in the plurality of documents with additional variables; and

comparing additional clauses and additional clause examples to obtain additional standard exact clauses, the additional clauses obtained from the plurality of documents with the available variation replaced with the additional variables, the additional standard exact clauses comprising the additional clauses matching respective ones of the additional clause examples and the additional variables.

12. The method of claim 11, further comprising:

obtaining the secondary policy comprising the one or more features, the clause example, and the second threshold for use in the semantic language evaluator to compare each of the plurality of feature replaced clauses according to the secondary policy to provide the second feature data set encompassing the first feature data set;

obtaining a third feature data set comprising the standard exact clause and the additional standard exact clauses; and

updating the difference data set comprised of a difference between the second feature data set and the third feature data set, difference data set comprising a second non-standard clause.

13. A non-transitory computer readable medium storing program code for determination of standard exact clauses and non-standard clauses from a plurality of documents, the program code comprising instructions that when executed by a processor cause the processor to:

obtain a primary policy comprising one or more features, a clause example, and a first threshold to generate a plurality of feature replaced clauses by automatically replacing one or more original clauses in a plurality of documents with a feature of the one or more features;

compare each of the plurality of feature replaced clauses and a clause example using a semantic language evaluator to obtain a first feature data set comprising standard clauses;

automatically replace an available variation of one of the standard clauses in the plurality of documents with a variable;

compare a clause and at least one of the clause example and the variable, the clause obtained from the plurality of documents with the available variation replaced with the variable;

obtain, in response to the comparison, a standard exact clause comprising the clause matching at least one of the clause example and the variable;

obtain a second feature data set encompassing the first feature data set, the second feature data set corresponding to a secondary policy, the secondary policy comprising the one or more features, the clause example, and a second threshold for use in the semantic language evaluator;

obtain a difference data set comprised of a difference between the first feature data set and the second feature data set, the difference data set comprising a non-standard clause, the non-standard clause being semantically related to but not matching the clause example; and

update, automatically in response to obtaining the difference data set, a database to identify the standard exact clause and the non-standard clause from the plurality of documents.

14. The non-transitory computer readable medium of claim 13, wherein the standard exact clause and the at least one of the clause example and the variable comprise exact one or more words in a same order.

15. The non-transitory computer readable medium of claim 13, further comprising instructions when executed by the processor cause the processor to:

determine whether optical character recognition performed on a word is recognizable.

16. The non-transitory computer readable medium of claim 15, further comprising instructions when executed by the processor cause the processor to:

obtain, responsive to determining the optical character recognition performed on the word is not recognizable, a candidate clause in one of the plurality of documents with the available variation replaced with the variable, the candidate clause comprising a first word before the word and a second word after the word; and

determine, responsive to at least one of the clause example and the variable comprising the first word, a third word and the second word in that sequence, the clause to be a candidate standard exact clause.

17. The non-transitory computer readable medium of claim 13, further comprising instructions when executed by the processor cause the processor to:

automatically replace available variations of one or more of the standard clauses in the plurality of documents with additional variables; and

compare additional clauses and additional clause examples to obtain additional standard exact clauses, the additional clauses obtained from the plurality of documents with the available variations replaced with the additional variables, the additional standard exact clauses comprising the additional clauses matching respective ones of the additional clause examples and the additional variables.

18. The non-transitory computer readable medium of claim 17, further comprising instructions that when executed by the processor causes the processor to:

obtain the secondary policy comprising the one or more features, the clause example, and the second threshold for use in the semantic language evaluator to compare each of the plurality of feature replaced clauses according to the secondary policy to provide the second feature data set encompassing the first feature data set;

obtain a third feature data set comprising the standard exact clause and the additional standard exact clauses; and

update the difference data set comprised of a difference between the second feature data set and the third feature data set, difference data set comprising a second non-standard clause.

19. A system for determination of standard exact clauses and non-standard clauses from the plurality of documents, the system comprising:

a document parsing module configured to obtain a primary policy comprising one or more features, a clause example, and a first threshold to generate a plurality of feature replaced clauses by replacing one or more original clauses in the plurality of documents with the one or more features and to replace an available variation of one of standard clauses in the plurality of documents with a variable;

a standard clause detection module configured to compare each of the plurality of feature replaced clauses and a clause example using a semantic language evaluator to obtain a first feature data set comprising the standard clauses;

a standard exact clause matching module configured to (i) compare a clause and at least one of the clause example and the variable, the clause obtained from the plurality of documents with the available variation replaced with the variable, and (ii) obtain, in response to the comparison, the standard exact clause comprising the clause matching at least one of the clause example and the variable;

a non-standard exact clause matching module configured to (i) obtain a second feature data set encompassing the first feature data set, the second feature data set corresponding to a secondary policy, the secondary policy comprising the one or more features, the clause example, and a second threshold for use in the semantic language evaluator, and (ii) obtain a difference data set comprised of a difference between the first feature data set and the second feature data set, the difference data set comprising a non-standard clause, the non-standard clause being semantically related to but not matching the clause example; and

a database update module configured to update, automatically in response to obtaining the difference data set, a database to identify the standard exact clause and the non-standard clause from the plurality of documents.

20. The system of claim 19, further comprising:

an optical character recognition engine to obtain the plurality of original clauses from the plurality of documents.

21. The system of claim 20, further comprising:

an input quality analysis module configured to determine whether optical character recognition performed on a word is recognizable.