Apparatus and method for text extraction

• US 8,924,846 B2
• Filed: 07/03/2009
• Issued: 12/30/2014
• Est. Priority Date: 07/03/2009
• Status: Active Grant

• Current Assignee: The Conundrum IP LLC (UiPath Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zhou, Bao-Yao, Xiong, Yuhong, Liu, Wei
• Primary Examiner(s): Nguyen, Maikhanh
• Application Number:
  US13/258,464
• Publication Number:
  US 20120066587A1
• Time in Patent Office: 2,006 Days
• Field of Search: 715/200, 715/201, 715/202, 715/205, 715/206, 715/234, 715/700, 715/760, 715/255, 715/256, 715/267, 715/272, 715/273
• US Class Current: 715/234
• CPC Class Codes: G06F 40/137 Hierarchical processing, e.g. outlines

Abstract

A method of determining main text in a mark-up document is provided, which comprises determining a length of each paragraph in the mark-up document; and determining one or more main paragraphs of the mark-up document based upon the length of the paragraphs in the mark-up document.

Claims

1. A method of determining main text in a mark-up document, comprising:

removing, by a system having a processor, first predetermined mark-up tags from the mark-up document, and replacing second predetermined mark-up tags in the mark-up document with separation elements, wherein the removing and the replacing cause the mark-up document to contain text paragraphs and the separation elements without the first and second predetermined mark-up tags;

determining, by the system, a length of each of the text paragraphs in the mark-up document; and

determining, by the system, one or more main paragraphs of the mark-up document based upon the lengths of the text paragraphs in the mark-up document.

2. The method of claim 1, further comprising:

normalizing the lengths of the text paragraphs in the mark-up document with respect to a length of a longest text paragraph in the mark-up document, wherein determining the one or more main paragraphs is based on the normalized lengths of the text paragraphs.

3. The method of claim 2, wherein the lengths of the text paragraphs are normalized according to the equation:

4. The method of claim 1, comprising:

smoothing the lengths of the text paragraphs in the mark-up document based upon one or more adjacent text paragraph lengths, wherein determining the one or more main paragraphs is based on the smoothed lengths.

5. The method of claim 4, wherein the lengths of the text paragraphs are smoothed according to the equation:

6. The method of claim 1, wherein the replacing comprises replacing the second predetermined mark-up tags in the mark-up document with line breaks.

7. The method of claim 1, wherein the replacing comprises replacing the second predetermined mark-up tags with carriage returns.

8. The method of claim 1, comprising:

determining first main paragraphs of the mark-up document based upon the lengths of the text paragraphs in the mark-up document;

discarding from consideration the first main paragraphs of the mark-up document and determining second main paragraphs of the mark-up document based upon the lengths of the text paragraphs remaining in the mark-up document.

9. The method of claim 8, further comprising combining the first and second main paragraphs as the main paragraphs of the mark-up document.

10. An apparatus for determining main text in a mark-up document, comprising:

a memory having a mark-up document stored therein; and

a processor configured to;

remove first predetermined mark-up tags from the mark-up document, and replace second predetermined mark-up tags in the mark-up document with separation elements, wherein the removing and the replacing cause the mark-up document to contain text paragraphs and the separation elements without the first and second predetermined mark-up tags;

determine lengths of the text paragraphs in the mark-up document; and

determine one or more main paragraphs of the mark-up document based upon the lengths of the text paragraphs in the mark-up document.

11. The apparatus of claim 10, wherein the processor is configured to further compute a paragraph length threshold based upon the lengths of the text paragraphs in the mark-up document, and wherein the determining of the one or more main paragraphs is further according to the paragraph length threshold.

12. The apparatus of claim 10, wherein the processor is configured to further normalize the lengths of the text paragraphs in the mark-up document with respect to a length of a longest text paragraph in the mark-up document, wherein the determining of the one or more main paragraphs is based on the normalized lengths of the text paragraphs.

13. A non-transitory computer-readable storage medium storing instructions that upon execution cause a system to:

remove first predetermined mark-up tags from the mark-up document, and replacing second predetermined mark-up tags in the mark-up document with separation elements, wherein the removing and the replacing cause the mark-up document to contain text paragraphs and the separation elements without the first and second predetermined mark-up tags;

determine a length of each of the text paragraphs in the mark-up document; and

determine one or more main paragraphs of the mark-up document based upon the lengths of the text paragraphs in the mark-up document.

14. The non-transitory computer-readable storage medium of claim 13, wherein the instructions upon execution cause the system to further:

normalize the lengths of the text paragraphs in the mark-up document with respect to a length of a longest text paragraph in the mark-up document, wherein determining the one or more main paragraphs is based on the normalized lengths of the text paragraphs.

15. The non-transitory computer-readable storage medium of claim 14, wherein the lengths of the text paragraphs are normalized according to the equation:

16. The non-transitory computer-readable storage medium of claim 13, wherein the instructions upon execution cause the system to further:

smooth the lengths of the text paragraphs in the mark-up document based upon one or more adjacent text paragraph lengths, wherein determining the one or more main paragraphs is based on the smoothed lengths.

17. The non-transitory computer-readable storage medium of claim 13, wherein the replacing comprises replacing the second predetermined mark-up tags in the mark-up document with line breaks.

18. The non-transitory computer-readable storage medium of claim 13, wherein the replacing comprises replacing the second predetermined mark-up tags with carriage returns.

19. The non-transitory computer-readable storage medium of claim 13, wherein the instructions upon execution cause the system to further:

determine first main paragraphs of the mark-up document based upon the lengths of the text paragraphs in the mark-up document;

discard from consideration the first main paragraphs of the mark-up document and determining second main paragraphs of the mark-up document based upon the lengths of the text paragraphs remaining in the mark-up document.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions upon execution cause the system to further combine the first and second main paragraphs as the main paragraphs of the mark-up document.

21. The non-transitory computer-readable storage medium of claim 13, wherein the instructions upon execution cause the system to further compute a paragraph length threshold based upon the lengths of the text paragraphs in the mark-up document, and wherein the determining of the one or more main paragraphs is further according to the paragraph length threshold.