Automatic method of identifying drop words in a document image without performing character recognition
First Claim
1. A method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, a processor implementing the method by executing instructions stored in electronic form in a memory coupled to the processor, the method comprising the steps of:
- a) analyzing the document image to identify word equivalence classes, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences;
b) for each word equivalence class determining the likelihood that word equivalence class is a drop word;
c) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are drop words.
4 Assignments
0 Petitions
Accused Products
Abstract
A method of automatically identifying drop words in a document image without performing character recognition to generate an ASCII representation of the document text. First, the document image is analyzed to identify word equivalence classes, each of which represents at least one word of the multiplicity of words included in the document. Second, for each word equivalence class, the likelihood that it is not a drop word is determined. Third, document length is analyzed to determine whether the document is short. For a short document, the number of word equivalence classes identified as drop words based upon their likelihood is proportional to document length. For long documents, a fixed number of word equivalence classes are identified as drop words based upon the likelihood that they are not drop words.
-
Citations
11 Claims
-
1. A method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, a processor implementing the method by executing instructions stored in electronic form in a memory coupled to the processor, the method comprising the steps of:
-
a) analyzing the document image to identify word equivalence classes, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences; b) for each word equivalence class determining the likelihood that word equivalence class is a drop word; c) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are drop words. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, each word occurrence of the second multiplicity of word occurrences having a font size, a processor implementing the method by executing instructions stored in electronic form in a memory coupled to the processor, the method comprising the steps of:
-
a) identifying a dominant font size within the document image; b) identifying sentence boundaries for sentences of the dominant font size within the document image; c) analyzing the document image to identify word equivalence classes of the dominant font size, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences; d) for each word equivalence class determining the likelihood that word equivalence class is not a drop word based upon a width of each word equivalence class, a number of occurrences of the word equivalence class, and a number of occurrences of the word equivalence class as a first non-short word of a sentence; e) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are drop words. - View Dependent Claims (9)
-
-
10. An article of manufacture comprising:
-
a) a memory; and b) instructions stored in the memory for a method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, method comprising the steps of; 1) analyzing the document image to identify word equivalence classes, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences; 2) for each word equivalence class determining the likelihood that word equivalence class is not a drop word; 3) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are drop words.
-
-
11. An article of manufacture comprising:
-
a) a memory; and b) instructions stored in the memory for a method of identifying drop words in a document image without performing character recognition, the document image including a first multiplicity of sentences and a second multiplicity of word occurrences, each word occurrence of the second multiplicity of word occurrences having a font size, the method comprising the steps of; 1) identifying a dominant font size within the document image; 2) identifying sentence boundaries for sentences of the dominant font size within the document image; 3) analyzing the document image to identify word equivalence classes of the dominant font size, each word equivalence class including at least one word occurrence of the second multiplicity of word occurrences; 4) for each word equivalence class determining the likelihood that word equivalence class is not a drop word based upon a width of each word equivalence class, a number of occurrences of the word equivalence, and a number of occurrences of the word equivalence class as a first non-short word of a sentence; 5) designating a number of the word equivalence classes as drop words based upon the likelihood that the word equivalence classes are
-
Specification