Method and system for processing an image extracted from a document

• US 9,786,039 B2
• Filed: 03/10/2016
• Issued: 10/10/2017
• Est. Priority Date: 01/26/2016
• Status: Active Grant

• Current Assignee: Wipro Limited
• Original Assignee: Wipro Limited
• Inventors: Kumar, Sendil Kumar Jaya, Jagannath, Sujatha
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US15/066,206
• Publication Number:
  US 20170213326A1
• Time in Patent Office: 579 Days
• Field of Search: 382156, 382157, 382164, 382165, 382170, 382266, 382269, 382275, 382282, 358537, 358538, 358452, 358453
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

The present disclosure relates to a method and system for processing an image extracted from a document. The image processing system detects one or more edges of the image and determines a missing edge in the image by comparing color gradients of the edges. The missing edges are recreated by cloning image pixels based on data pixels of an edge, opposite to the missing edge, amongst the one or more edges to obtain a reconstructed image. Outer corner points in the reconstructed image are identified based on quadratic corner points, wherein the quadratic corner points are determined based on the one or more edges and the reconstructed edge. Further, the image processing system performs an image perspective correction on the reconstructed image, based on the outer corner points, to obtain a processed image.

Claims

1. A method for processing an image extracted from a document, the method comprising:

detecting, by an image processing system, one or more edges of the image;

determining, by the image processing system, a missing edge in the image by comparing color gradients of the one or more edges;

computing a color gradient along each side of the image; and

comparing the color gradient of each side with remaining sides to detect the missing edge;

reconstructing, by the image processing system, the missing edge by cloning image pixels based on data pixels of an edge, opposite to the missing edge, amongst the one or more edges to obtain a reconstructed image, wherein the reconstructed image includes a reconstructed edge;

identifying, by the image processing system, outer corner points in the reconstructed image based on quadratic corner points, wherein the quadratic corner points are determined based on the one or more edges and the reconstructed edge; and

performing, by the image processing system, an image perspective correction on the reconstructed image, based on the outer corner points, to obtain a processed image.

2. The method as claimed in claim 1 further comprises:

extracting, by the image processing system, at least one unique feature from the processed image;

comparing, by the image processing system, the at least one unique feature with a trained data set to validate the processed image, wherein the trained data set comprises template images and predefined features set; and

storing, by the image processing system, the processed image for future usage upon successful validation.

3. The method as claimed in claim 1, wherein reconstructing the missing edge further comprises:

extending size of the image along the missing edge until a predetermined offset is reached to obtain an extended image;

determining an empty region in the extended image by moving data pixels based on the predetermined offset; and

cloning the empty region, based on the data pixel of the edge opposite to the missing edge, to reconstruct the missing edge.

4. The method as claimed in claim 3, wherein cloning the empty region further comprises:

computing a color variation between background color and a color of the document; and

applying an edge gradient direction on cloned empty region, wherein the edge image gradient direction is determined based on the color variation.

5. The method as claimed in claim 1, wherein identifying the outer corner points further comprises:

deriving the quadratic corner points by extending lines detected along the one or more missing edges and the reconstructed edge;

filtering redundant quadratic corner points amongst the quadratic corner points to obtain a filtered set of quadratic corner points;

detecting outer corner points in the filtered set of quadratic corner points based on coordinates, along a vertical direction and a horizontal direction, corresponding to each quadratic corner point present in the filtered set of quadratic corner points;

increasing an edge detection filter threshold value of the reconstructed image upon unsuccessful detection of all the outer corner points; and

iterating detection of the outer corner points with the increased edge detection filter threshold value till all the outer corner points are detected.

6. The method as claimed in claim 1, wherein performing the image perspective correction on the reconstructed image further comprises:

performing polygon approximation on the outer corner points;

determining a mass center of the reconstructed image based on the polygon approximation;

computing a perspective transformation for the reconstructed image based on the mass center and a predefined image size; and

applying the perspective transformation to the reconstructed image to obtain the processed image.

7. An image processing system for processing an image extracted from a document, the image processing system comprising:

a processor; and

a memory communicatively coupled to the processor, wherein the memory stores processor-executable instructions, which, on execution, causes the processor to perform operations comprising;

detecting one or more edges of the image;

determining a missing edge in the image by comparing color gradients of the one or more edges;

computing a color gradient along each side of the image; and

comparing the color gradient of each side with remaining sides to detect the missing edge;

reconstructing the missing edge by cloning image pixels based on data pixels of an edge, opposite to the missing edge, amongst the one or more edges to obtain a reconstructed image, wherein the reconstructed image includes a reconstructed edge;

identify outer corner points in the reconstructed image based on quadratic corner points, wherein the quadratic corner points are determined based on the one or more edges and the reconstructed edge; and

performing an image perspective correction on the reconstructed image, based on the outer corner points, to obtain a processed image.

8. The system as claimed in claim 7 wherein the instructions further causes the processor to perform operations comprising:

extracting at least one unique feature from the processed image;

comparing the at least one unique feature with a trained data set to validate the processed image, wherein the trained data set comprises template images and predefined features set; and

storing the processed image for future usage upon successful validation.

9. The system as claimed in claim 7, wherein operations of reconstructing the missing edge further comprise:

extending size of the image along the missing edge until a predetermined offset is reached to obtain an extended image;

determining an empty region in the extended image by moving data pixels based on the predetermined offset; and

cloning the empty region, based on the data pixel of the edge opposite to the missing edge, to reconstruct the missing edge.

10. The system as claimed in claim 9, wherein operations of cloning the empty region further comprise:

computing a color variation between background color and a color of the document; and

applying an edge gradient direction on cloned empty region, wherein the edge image gradient direction is determined based on the color variation.

11. The system as claimed in claim 7, wherein operations of identifying the outer corner points further comprise:

deriving the quadratic corner points by extending lines detected along the one or more missing edge and the reconstructed edge;

filtering redundant quadratic corner points amongst the quadratic corner points to obtain a filtered set of quadratic corner points;

detecting outer corner points in the filtered set of quadratic corner points based on coordinates, along a vertical direction and a horizontal direction, corresponding to each quadratic corner point present in the filtered set of quadratic corner points;

increasing an edge detection filter threshold value of the reconstructed image upon unsuccessful detection of all the outer corner points; and

iterating detection of the outer corner points with the increased edge detection filter threshold value till all the outer corner points are detected.

12. The system as claimed in claim 7, wherein operations of performing the image perspective correction on the reconstructed image further comprise:

performing polygon approximation on the outer corner points;

determining a mass center of the reconstructed image based on the polygon approximation;

computing a perspective transformation for the reconstructed image based on the mass center and a predefined image size; and

applying the perspective transformation to the reconstructed image to obtain the processed image.

13. A non-transitory computer readable medium including instructions stored thereon that when processed by at least one processor cause an image processing system to perform operations comprising:

detecting one or more edge of an image extracted from a document;

determining a missing edge in the image by comparing color gradients of the one or more edges;

computing a color gradient along each side of the image; and

comparing the color gradient of each side with remaining sides to detect the missing edge;

reconstructing the missing edge by cloning image pixels based on data pixels of an edge, opposite to the missing edge, amongst the one or more edges to obtain a reconstructed image, wherein the reconstructed image includes a reconstructed edge;

identify outer corner points in the reconstructed image based on quadratic corner points, wherein the quadratic corner points are determined based on the one or more edges and the reconstructed edge; and

performing an image perspective correction on the reconstructed image, based on the outer corner points, to obtain a processed image.