Analyzing externally generated documents in document management system

• US 20060271519A1
• Filed: 05/25/2005
• Published: 11/30/2006
• Est. Priority Date: 05/25/2005
• Status: Active Grant

• Current Assignee: Datix (USA) Inc.
• Original Assignee: ECTEON, Inc. (DATIX (Holdings) Ltd.)
• Inventors: Green, Gordon, Eckerstrom, Richard, Blackwood, Lonnie, Schiffrin, Marc
• Granted Patent: US 8,112,401 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

A computer implemented method for analyzing an externally generated document for use in a document management system having a Native Template database including a list of templates for one or more types of documents having common characteristics and a Conversion Database including a list of one or more data points associated with each listed document type, one or more descriptive text entries associated with each listed data point, and proximity range information relating to the location of the data point within the descriptive text. The externally generated document is introduced into the system. The locations of words, sentences, paragraphs, and sections within the document are recorded. A document type is selected from the Native Template database that has characteristics in common with the externally generated document. A data point is selected from the template. The introduced document is searched for Possible Data Points based on the Data Type of the selected data point in the Conversion Database. Proximity range information is obtained from the Conversion Database for the Descriptive Text entries associated with the selected data point. A determination is made as to whether Possible Data Point values for the selected data point are located within the Proximity range for each Descriptive Text entry. A cumulative Evaluation Score is calculated for each Possible Data Point value based on its proximity to each Descriptive Text entry. The Possible Data Point with the highest score that has been accepted by the user is recorded. Upon user acceptance of a Possible Data Point, additional Descriptive Text entries are stored to apply to other externally generated documents. These steps are repeated until each data point has been selected. The user reviews the recorded data which is approved, modified or rejected.

Claims

1. A computer implemented method for analyzing an externally generated document for use in a document management system of the type having a Native Template database including a list of templates for one or more types of documents having common characteristics and a Conversion Database including a list of one or more data points associated with each listed document type, one or more Descriptive Text entries associated with each listed data point, and Proximity information which describes the location of the data point in relation to the Descriptive Text, the method comprising the steps of:

(a) introducing the externally generated document into the system;

(b) indexing the externally generated document by recording the locations of words, sentences, paragraphs, and sections within the document;

(c) selecting a document type from the Native Template database that has characteristics in common with the externally generated document;

(d) selecting a data point from the template;

(e) searching the introduced document for Possible Data Points based on the Data Type of the selected data point in the Conversion Database;

(f) obtaining Proximity range information from the Conversion Database for the Descriptive Text entries associated with the selected data point;

(g) determining whether Possible Data Point values for the selected data point are located within the Proximity range for each Descriptive Text entry, using the index information created in (b);

(h) calculating a cumulative Evaluation Score for each Possible Data Point value based on its proximity to each Descriptive Text entry;

(i) recording the Possible Data Point with the highest score that has been accepted by the user;

(j) upon user acceptance of a Possible Data Point, storing additional Descriptive Text entries to apply to other externally generated documents; and

(k) repeating steps (d)-(j) until each data point has been selected.

2. The method of claim 1 further comprising the steps of reviewing the introduced document, identifying and storing section headers and associating the selected data point with the text under the appropriate section header in the Conversion Database.

3. The method of claim 1 further comprising the steps of reviewing the introduced document and identifying and classifying the numbering/lettering system used therein.

4. The method of claim 2 further comprising the steps of adjusting the cumulative Evaluation Score of each Possible Data Point based on its presence within the applicable Section Topic Headers, and whether the hierarchy is regular, incomplete or unknown

5. The method of claim 1 further comprising the step of assigning a preference to the descriptive text with the highest Evaluation Score, if the text associated with the selected data point is located in more than one location in the introduced document.

6. The method of claim 1 further comprising the step of presenting the Possible Data Points to the user for review.

7. The method of claim 6 further comprising the steps of approving, modifying or rejecting each of the Possible Data Points.

8. The method of claim 1 further comprising the step of setting up the system to determine how a specific variation of a document relates to a template.

9. The method of claim 8 wherein the step of setting up the system comprises the steps of:

(l) introducing a set-up document of the type associated with the template;

(m) presenting the user with the set-up document and template simultaneously;

(n) selecting a data point from the template;

(o) assigning the selected data point to a field in the Conversion Database;

(p) associating the text accompanying the selected data point with the selected data point in the assigned Conversion Database field;

(q) assigning a data type to the selected data point in the Conversion Database;

(r) storing a default set of Descriptive Text values and Proximities based on the context of the selected data point value;

(s) storing an additional user-specified set of Descriptive text values;

(t) assigning proximity information to each user-specified Descriptive Text value; and

(u) repeating steps (n) through (t) for each data point selected in the set-up document.

10. The method of claim 9 further comprising the step of introducing multiple set-up documents into the system and repeating steps (l) through (u) for each.

11. The method of claim 9 further comprising the step of assigning an Expected Value range to the selected data point in the Conversion Database.

12. The method of claim 9 further comprising the step of assigning Section Topic Headers to the selected data point in the Conversion Database.

13. The method of claim 2 wherein the steps of identifying and storing the section headers comprise the steps of:

(v) identifying paragraphs shorter than a given length; and

(w) marking each such identified paragraph as a section header.

14. The method of claim 2 wherein the steps of identifying and storing the section headers comprise the steps of:

(x) identifying paragraphs shorter than a given length that end with a hyphen or colon; and

(y) marking each such identified paragraph as a section header.

15. The method of claim 14 further comprising the steps of:

(z) identifying paragraphs shorter than a given length with no period;

(aa) identifying paragraphs shorter than a given length with first character capitalized words;

(bb) identifying paragraphs shorter than a given length with all bolded or underlined words; and

(cc) determining the frequency of occurrence of (z), (aa) and (bb) in the introduced document, and marking each paragraph if the frequency is over a given value.

16. The method of claim 3 wherein the steps of identifying and classifying the numbering/lettering system used therein comprises the steps of:

(dd) recording the number or letter at the beginning of each paragraph in the introduced document;

(ee) determining the hierarchy type used in the introduced document; and

(ff) recording the hierarchy type determination in the Conversion Database.

17. A computer implemented method for analyzing an externally generated document for use in a document management system of the type having a Native Template database including a list of templates for one or more types of documents having common characteristics and Conversion Database including a list of one or more data points associated with each listed document type, one or more descriptive text entries associated with each listed data point, and Proximity information which describes the location of the data point in relation to the Descriptive Text, the method comprising the steps of:

(gg) setting up the system to determine how a specific variation of a document relates to a template.

(hh) introducing the externally generated document into the system;

(ii) analyzing the introduced document by selecting a template from the Native Template database relating to a document type that has characteristics in common with the introduced document, selecting a data point from the template, searching the introduced document for text associated with the selected data point and recording the data point and the location of the associated text in the Conversion Database;

(jj) presenting the recorded data to the user for review; and

(kk) approving, modifying or rejecting the presented data.

18. The method of claim 17 further comprising the steps of reviewing the introduced document, identifying and storing section headers and associating the selected data point with the text under the appropriate section header in the Conversion Database.

19. The method of claim 17 further comprising the steps of reviewing the introduced document and identifying and classifying the numbering/lettering system used therein.

20. The method of claim 19 further comprising the steps of adjusting the cumulative Evaluation Score of each Possible Data Point based on its presence within the applicable Section Topic Headers, and whether the hierarchy is regular, incomplete or unknown

21. The method of claim 19 further comprising the steps of assigning a numerical score to the data point based upon its location in the introduced document and assigning a preference to the descriptive text with the highest Evaluation Score if the text associated with the selected data point is located in more than one location in the introduced document.

22. The method of claim 17 wherein the step of setting up the system comprises the steps of:

(ll) introducing a set-up document of the type associated with the template;

(mm) presenting the user with the set-up document and template simultaneously;

(nn) selecting a data point from the template;

(oo) assigning the selected data point to a field in the Conversion Database;

(pp) associating the text accompanying the selected data point with the selected data point in the assigned Conversion Database field;

(qq) assigning a Data Type to the selected data point in the Conversion Database;

(rr) storing a default set of Descriptive Text values and Proximities based on the context of the selected data point value;

(ss) storing an additional user-specified set of Descriptive text values;

(tt) assigning proximity information to each user-specified Descriptive Text value; and

(uu) repeating steps (nn) through (tt) for each data point selected in the set-up document.

23. The method of claim 22 further comprising the step of introducing multiple set-up documents into the system and repeating steps (ll) through (rr) for each.

24. The method of claim 22 further comprising the step of assigning an Expected Value range to the selected data point in the Conversion Database.

25. The method of claim 18 further comprising the step of assigning a section topic header to the selected data point in the Conversion Database.

26. The method of claim 18 wherein the steps of identifying and storing the section headers comprise the steps of:

(uu) identifying paragraphs shorter than a given length; and

(vv) marking each such identified stand-alone paragraph as a section header.

27. The method of claim 18 wherein the steps of identifying and storing the section headers comprise the steps of:

(ww) identifying paragraphs shorter than a given length that end with a hyphen or colon; and

(xx) marking each such identified paragraph as a section header.

28. The method of claim 27 further comprising the steps of:

(yy) identifying paragraphs shorter than a given length with no period;

(zz) identifying paragraphs shorter than a given length with first character capitalized words;

(aaa) identifying paragraphs shorter than a given length with all bolded or underlined words; and

(bbb) determining the frequency of occurrence of (zz), (aaa) and (bbb) in the introduced document, and marking each paragraph if the frequency is over a given value.

29. The method of claim 19 wherein the step of identifying and classifying the numbering/lettering system used therein comprises the steps of:

(ccc) recording the number or letter at the beginning of each paragraph in the introduced document;

(ddd) determining the hierarchy type used in the introduced document; and

(eee) recording the hierarchy type determination in the Conversion Database.