Photo-document segmentation method and system

• US 8,265,393 B2
• Filed: 05/01/2008
• Issued: 09/11/2012
• Est. Priority Date: 05/01/2007
• Status: Active Grant

• Current Assignee: Compulink Management Center Incorporated
• Original Assignee: Compulink Management Center Incorporated
• Inventors: Tribelhorn, Benjamin R., Wolin, Aaron D., Smith, Stephen T., Field, Adam R., Dodds, Zachary B.
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US12/113,940
• Publication Number:
  US 20090175537A1
• Time in Patent Office: 1,594 Days
• Field of Search: 382/112, 382/173, 382/174, 382/176, 382/199, 358/449, 358/453, 358/462, 358/464
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

The present application provides an improved segmentation method and system for processing digital images that include an imaged document and surrounding image. A plurality of edge detection techniques are used to determine the edges of the imaged document and then segment the imaged document from the surrounding image.

Claims

1. A method for processing a digital image, said digital image comprising an imaged document and surrounding image, said method comprising:

finding potential edges of said imaged document by at least two different computer implemented edge detection techniques;

grouping the found potential edges into top, bottom, left and right side potential edge groups;

for each edge group, selecting a subset of potential edges that are determined to likely represent an edge of the imaged document;

determining the edges of the imaged document, wherein said determining step includes further analysis of the subsets of potential edges; and

segmenting the imaged document from the digital image using the determined edges of the imaged document.

2. The method of claim 1, wherein the selecting step comprises culling a plurality of edges, snapping the culled edges, and unifying the snapped edges.

3. The method of claim 1, wherein the determining step comprises a process of generating a plurality of edge sets from the subsets of potential edges.

4. The method of claim 3, wherein the process of generating the plurality of edge sets further comprises determining each possible edge set from the subsets of the potential edges.

5. The method of claim 4, wherein the determining step further comprises forming a second plurality of edge sets by adding an additional edge set generated by an edge detection technique to the plurality of edge sets determined from the subsets of the potential edges.

6. The method of claim 5, wherein the determining step further comprises selecting from the second plurality of edge sets an edge set that most accurately represent the edges of the imaged document based on predefined comparison criteria.

7. The method of claim 6, wherein the predefined comparison criteria includes at least one attribute of the edge sets selected from the group consisting of trapezoidality, contrast, contrast consistency, area, texture, and edge length variation.

8. The method of claim 7, wherein the predefined comparison criteria is a linear combination of the trapezoidality of an edge set, the median contrast of edges of an edge set, the area formed by the edges of an edge set, and the contrast consistency of edges of an edge set.

9. The method of claim 3, wherein the determining step further comprises forming a second plurality of edge sets by adding an additional edge set generated by an edge detection technique to the plurality of edge sets determined from the subsets of the potential edges.

10. The method of claim 9, wherein the added edge set is generated by a lines of text technique.

11. The method of claim 9, wherein the added edge set is generated by a RANSAC technique.

12. The method of claim 3, wherein the determining step further comprises selecting from the plurality of edge sets an edge set that most accurately represent the edges of the imaged document based on predefined comparison criteria.

13. A computer system for processing a digital image comprising an imaged document and surrounding image, said computer system comprising:

one or more processors; and

a memory including one or more sequences of one or more instructions which, when executed by one or more processors, cause the one or more processors to perform the computer-implemented steps of;

finding potential edges of said imaged document by at least two different computer implemented edge detection techniques;

grouping the found potential edges into top, bottom, left and right side potential edge groups;

for each edge group, selecting a subset of potential edges that are determined to likely represent an edge of the imaged document;

determining the edges of the imaged document, wherein said determining step includes further analysis of the subsets of potential edges; and

segmenting the imaged document from the digital image using the determined edges of the imaged document.

14. A non-transitory computer readable medium for processing a captured image including a digital image comprising an imaged document and surrounding image, the non-transitory computer readable medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause the one or more processors to perform the computer-implemented steps of:

finding potential edges of said imaged document by at least two different computer implemented edge detection techniques;

grouping the found potential edges into top, bottom, left and right side potential edge groups;

for each edge group, selecting a subset of potential edges that are determined to likely represent an edge of the imaged document;

determining the edges of the imaged document, wherein said determining step includes further analysis of the subsets of potential edges; and

segmenting the imaged document from the digital image using the determined edges of the imaged document.

15. A method for processing a digital image, said digital image comprising an imaged document and surrounding image, said method comprising:

generating a plurality of potential edge sets for the imaged document using one or more computer implemented edge detection techniques, wherein at least one of the edge sets in the plurality of potential edge sets includes edges formed from multiple edge detection techniques;

selecting from the plurality of edge sets an edge set that most accurately represent the edges of the imaged document based on predefined comparison criteria; and

segmenting the imaged document from the digital image using the selected edge set.

16. The method of claim 15, wherein the predefined comparison criteria includes at least one attribute of the edge sets selected from the group consisting of trapezoidality, contrast, contrast consistency, area, texture, and edge length variation.

17. The method of claim 15, wherein the plurality of potential edge sets also includes an edge set the edges of which are all determined by a single edge detection technique.

18. The method of claim 17, wherein the edge set the edges of which are all determined by a single edge detection technique is generated by a lines of text technique.

19. The method of claim 18, wherein the edge set the edges of which are all determined by a single edge detection technique is generated by RANSAC technique.

20. The method of claim 18, wherein the plurality of potential edge sets also includes a second edge set the edges of which are all determined by a RANSAC technique.

21. A method for processing a digital image, said digital image comprising an imaged document and surrounding image, said method comprising:

generating a plurality of potential edge sets for the imaged document using one or more computer implemented edge detection techniques;

selecting from the plurality of edge sets an edge set that most accurately represent the edges of the imaged document based on predefined comparison criteria;

wherein the predefined comparison criteria includes at least one attribute of the edge sets selected from the group consisting of trapezoidality, contrast, contrast consistency, area, texture, and edge length variation, wherein the predefined comparison criteria is a linear combination of the trapezoidality of an edge set, the median contrast of edges of an edge set, the area formed by the edges of an edge set, and the contrast consistency of edges of an edge set; and

segmenting the imaged document from the digital image using the selected edge set.

22. A method for processing a digital image, said digital image comprising an imaged document and surrounding image, said method comprising:

finding potential edges of said imaged document by at least two different computer implemented edge detection techniques;

grouping the found potential edges into top, bottom, left and right side potential edge groups;

for each edge group, selecting a subset of potential edges that are determined to likely represent an edge of the imaged document;

generating a plurality of edge sets from the subsets of potential edges;

selecting from the plurality of edge sets a small subset of edge sets that are determined to most accurately represent the edges of the imaged document by using predefined comparison criteria;

displaying a small subset of the edge sets on a computer graphic user interface;

receiving an input from a user corresponding to the best displayed edge set; and

segmenting the imaged document from the digital image using the best displayed edge set.

23. A method for processing a digital image, said digital image comprising an imaged document and surrounding image, said method comprising:

generating a plurality of potential edge sets for the imaged document using one or more computer implemented edge detection techniques, wherein at least one of the edge sets in the plurality of potential edge sets includes edges formed from multiple edge detection techniques;

selecting from the plurality of edge sets a small subset of edge sets that are determined to most accurately represent the edges of the imaged document by using predefined comparison criteria;

displaying the small subset of edge sets on a computer graphic user interface;

receiving an input from a user corresponding to the best displayed edge set; and

segmenting the imaged document from the digital image using the best displayed edge set.