Non-standard and standard clause detection

• US 9,268,768 B2
• Filed: 06/16/2014
• Issued: 02/23/2016
• Est. Priority Date: 12/05/2013
• Status: Active Grant

• Current Assignee: DocuSign International (EMEA) Ltd. (DocuSign Incorporated)
• Original Assignee: Seal Software Ltd.
• Inventors: Gidney, Kevin
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US14/306,069
• Publication Number:
  US 20150161102A1
• Time in Patent Office: 617 Days
• Field of Search: 704/9, 704/8, 704/4, 704/2, 704/10, 704/1, 715/257, 715/230, 709/206, 709/204, 707/802, 707/776, 707/770, 707/766, 707/708, 706/45, 705/7.13
• US Class Current: 1/1
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A non-standard and standard clause detection system imports raw input data or contractual documents, and extracts non-standard and standard clauses that are semantically linked. One embodiment of a disclosed configuration is a system and a method for identifying non-standard and standard clauses in contractual documents. The system and the method comprise of generating a primary policy and a secondary policy, obtaining a first feature data set by applying the primary policy to a semantic language evaluator, and obtaining a second feature data set by applying the secondary policy to the semantic language evaluator. The first feature data set obtained is the aggregation of the standard clauses used in the document. Furthermore, the second feature data set encompasses the first feature data set, thus the difference between the first feature data set and the second feature data set is the aggregation of the non-standard clauses.

Claims

1. A system for identifying standard clauses and non-standard clauses in a document comprised of a plurality of original clauses, the system comprising:

a processor; and

a memory storing modules comprised of instructions executable by the processor, the modules comprising;

a policy definition module configured to generate a primary policy;

a standard clause detection module configured to apply the primary policy for performing a first semantic language evaluation to generate a first feature data set; and

a non-standard clause detection module configured to;

generate a secondary policy,

apply the secondary policy for performing a second semantic language evaluation to generate a second feature data set, the second feature data set encompassing the first feature data set, and

obtain a difference data set comprising a difference between the first feature data set and the second feature data set, the difference data set comprising the non-standard clauses.

2. The system for claim 1, wherein the memory further stores a semantic language evaluator comprised of instructions executable by the processor, the semantic language evaluator for performing the first semantic language evaluation using the primary policy and the second semantic language evaluation using the secondary policy.

3. The system for claim 1, wherein the modules further comprise a custom feature generation module configured to allow an end user to define primary features.

4. The system for claim 3, wherein the modules further comprise a document segmentation module configured to generate a plurality of feature replaced clauses by replacing each of the plurality of original clauses with the primary features.

5. The system for claim 1, wherein the modules further comprise a high level processing module configured to generate a predefined policy, wherein the policy definition module generates the primary policy based on the predefined policy.

6. The system for claim 5, wherein the memory further stores engines comprised of instructions executable by the processor, the engines comprising an internal search engine to obtain search results by applying the predefined policy.

7. The system for claim 5, wherein the memory further stores engines comprised of instructions executable by the processor, the engines comprising an external search engine to obtain search results by applying the predefined policy.

8. The system for claim 5, wherein the modules further comprise:

a pre-normalization module configured to convert textual information in the document into XML or Hyper Text Markup Language (HTML),

a language detection module configured to identify language used in the document,

a structuration function module configured to group the document into a sentence level, a paragraph level, a section level and a document level,

a rules processing module configured to generate predefined features from predefined rules, and

a post processing and reduction module configured to reduce the predefined features by removing duplicate features in the predefined features, the duplicate features comprising a plurality of the predefined features with equal representation,

wherein the high level processing module is configured to generate the predefined policy comprising predefined rules, predefined features and predefined clause examples.

9. The system for claim 1, wherein the modules further comprise:

a file import system module configured to receive raw input data comprising a document, the document comprising textual information,

a format standardization system module configured to obtain the textual information in a standardized format, and

a correction module configured to perform typographical corrections.

10. A system for identifying standard clauses and non-standard clauses in a document comprised of a plurality of original clauses, the system comprising:

an input processor configured to retrieve the document;

a memory storing modules comprised of instructions executable by a processor, the modules comprising;

a policy definition module configured to generate a primary policy,

a standard clause detection module configured to apply the primary policy for performing a first semantic language evaluation with a semantic language evaluator to generate a first feature data set, and

a non-standard clause detection module configured to;

generate a secondary policy,

apply the secondary policy for performing a second semantic language evaluation with the semantic language evaluator to generate a second feature data set, the second feature data set encompassing the first feature data set, and

obtain a difference data set comprising a difference between the first feature data set and the second feature data set, the difference data set comprising the non-standard clauses; and

a database.

11. The system for claim 10, wherein the memory stores engines comprised of instructions executable by the processor, the engines comprising an internal search engine communicatively coupled to the database to obtain search results by applying a predefined policy, wherein the policy definition module is configured to generate the primary policy based on the predefined policy.

12. The system for claim 10, wherein the memory stores engines comprised of instructions executable by the processor, the engines comprising an external search engine coupled to the database.

13. The system for claim 10, wherein the modules further comprise:

a file import system module configured to receive raw input data comprising a document, the document comprising textual information,

a format standardization system module configured to obtain the textual information in a standardized format, and

a correction module configured to perform typographical corrections.

14. The system for claim 13, wherein the modules further comprise:

a pre-normalization module configured to convert the textual information into XML or Hyper Text Markup Language (HTML),

a language detection module configured to identify language used in the document,

a processing queue module,

a structuration function module configured to group the document into a sentence level, a paragraph level, a section level and a document level,

a rules processing module configured to generate predefined features from predefined rules,

a post processing and reduction module configured to reduce the predefined features by removing duplicate features in the predefined features, the duplicate features comprising a plurality of the predefined features with equal representation, and

a high level processing module configured to generate a predefined policy comprising predefined rules, predefined features and predefined clause examples.