Fast identification of text intensive pages from photographs

• US 10,372,981 B1
• Filed: 09/22/2016
• Issued: 08/06/2019
• Est. Priority Date: 09/23/2015
• Status: Active Grant

• Current Assignee: Bending Spoons SpA
• Original Assignee: Evernote Corp. (Bending Spoons SpA)
• Inventors: Pashintsev, Alexander, Gorbatov, Boris, Livshitz, Eugene, Glazkov, Vitaly
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US15/272,744
• Time in Patent Office: 1,048 Days
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

Determining if a document is a text page includes partitioning the document into a plurality of cells, scaling each of the cells to a standardized number of pixels to provide a corresponding snippet for each of the cells, using a classifier to examine the snippets to determine which of the cells are classified as text and which of the cells are not classified as text, determining a volume of text for the document based on a total amount of text in the document corresponding to a sum of an amount of text in each of the cells classified as text, and determining that the document is a text page in response to the total amount exceeding a pre-determined threshold. In response to the total amount being less than the pre-determined threshold, cells not classified as text may be examined further. The classifier may be provided by training a neural net.

Claims

1. A method of determining if a document is a text page, comprising:

partitioning the document into a first plurality of cells;

scaling each respective cell of the first plurality of cells to a standardized number of pixels to provide a first set of snippets, wherein the first set of snippets correspond to the first plurality of cells;

using a classifier to examine the first set of snippets to determine which of the first plurality of cells are classified as text and which of the cells are not classified as text;

determining a volume of text for the document based on a determined total amount of text in the document corresponding to a sum of an amount of text in each of the first plurality of cells classified as text;

determining whether the determined total amount of text in the document meets a pre-determined threshold;

in response to determining that the determined total amount of text in the document does not meet the pre-determined threshold and that criteria for a next partition level are met, for respective cells that are not classified as text;

a) applying further partitioning to the respective cells that are not classified as text to generate a respective further plurality of cells;

b) scaling each of the further partitioned cells to provide a respective further set of snippets that correspond to the further partitioned cells;

c) using the classifier to examine the respective further set of snippets to determine which of the respective further partitioned cells are classified as text and which of the respective further partitioned cells are not classified as text;

d) augmenting the volume of text for the document based on an updated determined total amount of text in the document corresponding to a sum of the amount of text in each of the cells of the plurality of cells and the respective further partitioned cells classified as text;

in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for a next partition level are met, repeating a)-d);

and

determining that the document is a text page in response to the determined total amount of text in the document exceeding the pre-determined threshold.

2. A method, according to claim 1, wherein the classifier examines the respective further set of snippets in a random order.

3. A method, according to claim 1, wherein the classifier examines the respective further set of snippets in an order that prioritizes snippets that are adjacent to snippets previously classified as text.

4. A method, according to claim 1, further comprising:

in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for the next partition level are not met, determining that the document is a text page in response to cells that are classified as text having a satisfactory geometry.

5. A method, according to claim 4, wherein at least some of the cells corresponding to snippets that are classified as text are aligned to form at least one text line and wherein the at least one text line is one of:

horizontal or vertical.

6. A method, according to claim 1, wherein the snippets that are not classified as text are classified as images.

7. A method, according to claim 1, wherein the snippets that are not classified as text are classified as one of:

images and unknown.

8. A method, according to claim 1, wherein partitioning the document into the first plurality of cells includes partitioning the document into six cells.

9. A method, according to claim 1, wherein the document is captured using a smartphone.

10. A method, according to claim 1, wherein the classifier is provided by training a neural net using a plurality of image documents and a plurality of text pages having various formats, layouts, text sizes, ranges of word, line and paragraph spacing.

11. A method, according to claim 1, including determining that the document is not a text page in response to determining that the augmented volume of text for the document does not meet the pre-determined threshold and that criteria for a next partition level are not met.

12. A non-transitory computer readable medium containing software that determines if a document is a text page, the software comprising:

executable code that partitions the document into a first plurality of cells;

executable code that scales each respective cell of the first plurality of cells to a standardized number of pixels to provide a first set of snippets, wherein the first set of snippets correspond to the first plurality of cells;

executable code that uses a classifier to examine the first set of snippets to determine which of the first plurality of cells are classified as text and which of the cells are not classified as text;

executable code that determines a volume of text for the document based on a determined total amount of text in the document corresponding to a sum of an amount of text in each of the first plurality of cells classified as text;

executable code that determines whether the determined total amount of text in the document meets a pre-determined threshold;

executable code that, in response to determining that the determined total amount of text in the document does not meet the pre-determined threshold and that criteria for a next partition level are met, for respective cells that are not classified as text;

a) applies further partitioning to the respective cells that are not classified as text to generate a respective further plurality of cells;

b) scales each of the further partitioned cells to provide a respective further set of snippets that correspond to the further partitioned cells;

c) uses the classifier to examine the respective further set of snippets to determine which of the respective further partitioned cells are classified as text and which of the respective further partitioned cells are not classified as text;

d) augments the volume of text for the document based on an updated determined total amount of text in the document corresponding to a sum of the amount of text in each of the cells of the plurality of cells and the respective further partitioned cells classified as text

executable code that, in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for a next partition level are met, repeats a)-d);

and

executable code that determines that the document is a text page in response to the total amount exceeding a pre-determined threshold.

13. A non-transitory computer readable medium, according to claim 12, wherein the classifier examines the respective further set of snippets in a random order.

14. A non-transitory computer readable medium, according to claim 12, wherein the classifier examines the respective further set of snippets in an order that prioritizes snippets that are adjacent to snippets previously classified as text.

15. A non-transitory computer readable medium, according to claim 12, further comprising:

executable code that, in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for the next partition level are not met, determines that the document is a text page in response to cells that are classified as text having a satisfactory geometry.

16. A non-transitory computer readable medium, according to claim 15, wherein at least some of the cells corresponding to snippets that are classified as text are aligned to form at least one text line and wherein the at least one text line is one of:

horizontal or vertical.

17. A non-transitory computer readable medium, according to claim 12, wherein the snippets that are not classified as text are classified as images.

18. A non-transitory computer readable medium, according to claim 12, wherein the snippets that are not classified as text are classified as one of:

images and unknown.

19. A non-transitory computer readable medium, according to claim 12, wherein partitioning the document into the first plurality of cells includes partitioning the document into six cells.

20. A non-transitory computer readable medium, according to claim 12, wherein the document is captured using a smartphone.

21. A non-transitory computer readable medium, according to claim 12, wherein the classifier is provided by training a neural net using a plurality of image documents and a plurality of text pages having various formats, layouts, text sizes, ranges of word, line and paragraph spacing.