Method of Detecting and Correcting Digital Images of Books in the Book Spine Area

• US 20120294528A1
• Filed: 05/19/2011
• Published: 11/22/2012
• Est. Priority Date: 05/19/2011
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Li, Jia, Brusnitsyn, Mikhail, Sukumaran, Sujay
• Granted Patent: US 8,457,403 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3873 defined only by a limited number of coordinate points or parameters, e.g. corners, centre; for trimming

Abstract

An image of a scanned book is segmented using a feature image to map pixels corresponding to a page area and to create page objects and detect borders of the page. A book spine region is detected by locating a plain background area between two of the page objects, analyzing the page borders to detect their shape, and analyzing their shape to detect the book spine end points. Using the page borders, the feature image is examined to detect top-to-bottom and bottom-to-top declines in pixel values to determine the corners of a shadow distortion in the original scanned image. Squeeze and curvature distortion are also detected. A Bezier curve is used to model each of the three distortions detected on the page. The detected distortion is corrected by first defining a trapezoidal correction area. The intensity, squeeze, and curvature corrections are then applied along lines within the trapezoidal correction area.

Claims

1. A method for detecting distortion in an input image of a scanned book, the input image being one of a digitized image stored in a memory or a scanned image from a scanner, comprising:

using a processor to

divide the input image into a plurality of blocks of pixels;

associate each block of pixels in the input image with a pixel in a feature image;

analyze values of each pixel in the feature image to create a binary map of the feature image, wherein each pixel in the binary map corresponds to a page area or not a page area;

subject the binary map to object segmentation to group binary map pixels corresponding to a page area;

create a set of page objects;

detect borders of the page area; and

detect distortion within the page area.

2. A method as in claim 1 wherein two page objects are created and detecting distortion comprises:

(a) in the feature image, selecting a first column next to a left border of the page area and track the values of background components of pixels in the first column from a center of a top page object to a middle of the two page objects;

(b) checking whether the tracked values have a top-to-bottom decline that is greater than a minimum value, and when the top-to-bottom decline is greater than the minimum value identify the pixel location in the feature image where the top-to-bottom decline starts in the first column, and when the top-to-bottom decline is not greater than the minimum value then return to (a) and select another column that is next to the first column; and

mapping the pixel location in the feature image where the top-to-bottom decline starts in the first column to the input image to identify the coordinate of the top-left corner of a shadow distortion in the input image.

3. A method as in claim 2 wherein detecting distortion further comprises:

(a) in the feature image, selecting a second column next to a right border of the page area and track the values of background components of pixels in the second column from a center of a top page object to a middle of the two page objects;

(b) checking whether the tracked values have a top-to-bottom decline that is greater than a minimum value, and when the top-to-bottom decline is greater than the minimum value identify the pixel location in the feature image where the top-to-bottom decline starts in the second column, and when the top-to-bottom decline is not greater than the minimum value then return to (a) and select another column that is next to the second column; and

mapping the pixel location in the feature image where the top-to-bottom decline starts in the second column to the input image to identify the coordinate of the top-right corner of the shadow distortion in the input image.

4. A method as in claim 3 wherein detecting distortion further comprises:

(a) in the feature image, selecting a third column next to a left border of the page area and track the values of background components of pixels in the third column from a center of a bottom page object to a middle of the two page objects;

(b) checking whether the tracked values have a bottom-to-top decline that is greater than the minimum value, and when the bottom-to-top decline is greater than the minimum value identify the pixel location in the feature image where the bottom-to-top decline starts in the third column, and when the bottom-to-top decline is not greater than the minimum value then return to (a) and select another column that is next to the third column; and

mapping the pixel location in the feature image where the bottom-to-top decline starts in the third column to the input image to identify the coordinate of the bottom-left corner of the shadow distortion in the input image.

5. A method as in claim 4 wherein detecting distortion further comprises:

(a) in the feature image, selecting a fourth column next to a right border of the page area and track the values of background components of pixels in the fourth column from a center of a bottom page object to a middle of the two page objects;

(b) checking whether the tracked values have a bottom-to-top decline that is greater than the minimum value, and when the bottom-to-top decline is greater than the minimum value identify the pixel location in the feature image where the bottom-to-top decline starts in the fourth column, and when the bottom-to-top decline is not greater than the minimum value then return to (a) and select another column that is next to the fourth column; and

mapping the pixel location in the feature image where the bottom-to-top decline starts in the fourth column to the input image to identify the coordinate of the bottom-right corner of the shadow distortion in the input image.

6. A method as in claim 2 wherein detecting distortion further comprises:

smoothing the values of the background components using a Gaussian filter.

7. A method as in claim 1 further comprising using the processor to detect a book spine region by

locating a plain background area between two of the page objects;

and

analyzing the page borders to detect a shape of the borders; and

analyzing the shape of the borders to detect the book spine end points.

8. A method as in claim 7 wherein detecting distortion further comprises approximating an intensity change in pixel values in a direction perpendicular to the spine.

9. A method as in claim 8 further comprising fitting intensity values to a Bezier curve to create an intensity curve.

10. A method as in claim 9 further comprising transforming the intensity curve to a squeeze curve.

11. A method as in claim 1 wherein detecting distortion comprises detecting curvature by examining text in the page area and fitting points on a string of text segments to a Bezier curve to create a curvature curve.

12. A method as in claim 1 further comprising correcting the detected distortion by defining a trapezoidal correction area.

13. A method as in claim 12 further applying intensity corrections defined with Bezier curves along lines within the trapezoidal correction area.

14. A method as in claim 12 further applying squeeze and curvature corrections defined with Bezier curves along lines within the trapezoidal correction area.

15. A device for detecting distortion in an input image of a scanned book, the input image being one of a digitized image stored in a memory or a scanned image from a scanner, comprising:

an image capture unit that captures the input image;

a memory that stores the input image; and

a processor that

divides the input image into a plurality of blocks of pixels;

associates each block of pixels in the input image with a pixel in a feature image;

analyzes values of each pixel in the feature image to create a binary map of the feature image, wherein each pixel in the binary map corresponds to a page area or not a page area;

subjects the binary map to object segmentation to group binary map pixels corresponding to a page area;

creates a set of page objects;

detects borders of the page area; and

detects distortion within the page area.

16. A device as in claim 15 wherein the processor creates two page objects and detecting distortion by the processor comprises:

(a) in the feature image, selecting a first column next to a left border of the page area and track the values of background components of pixels in the first column from a center of a top page object to a middle of the two page objects;

(b) checking whether the tracked values have a top-to-bottom decline that is greater than a minimum value, and when the top-to-bottom decline is greater than the minimum value identify the pixel location in the feature image where the top-to-bottom decline starts in the first column, and when the top-to-bottom decline is not greater than the minimum value then return to (a) and select another column that is next to the first column; and

mapping the pixel location in the feature image where the top-to-bottom decline starts in the first column to the input image to identify the coordinate of the top-left corner of a shadow distortion in the input image.

17. A device as in claim 16 wherein detecting distortion by the processor further comprises:

(a) in the feature image, selecting a second column next to a right border of the page area and track the values of background components of pixels in the second column from a center of a top page object to a middle of the two page objects;

(b) checking whether the tracked values have a top-to-bottom decline that is greater than a minimum value, and when the top-to-bottom decline is greater than the minimum value identify the pixel location in the feature image where the top-to-bottom decline starts in the second column, and when the top-to-bottom decline is not greater than the minimum value then return to (a) and select another column that is next to the second column; and

mapping the pixel location in the feature image where the top-to-bottom decline starts in the second column to the input image to identify the coordinate of the top-right corner of the shadow distortion in the input image.

18. A device as in claim 17 wherein detecting distortion by the processor further comprises:

(a) in the feature image, selecting a third column next to a left border of the page area and track the values of background components of pixels in the third column from a center of a bottom page object to a middle of the two page objects;

(b) checking whether the tracked values have a bottom-to-top decline that is greater than the minimum value, and when the bottom-to-top decline is greater than the minimum value identify the pixel location in the feature image where the bottom-to-top decline starts in the third column, and when the bottom-to-top decline is not greater than the minimum value then return to (a) and select another column that is next to the third column; and

mapping the pixel location in the feature image where the bottom-to-top decline starts in the third column to the input image to identify the coordinate of the bottom-left corner of the shadow distortion in the input image.

19. A device as in claim 15, wherein the image capture unit is a scanning unit.

20. One or more tangible computer-readable media having computer-readable instructions thereon, which, when executed by a processor detects distortion in an input image of a scanned book, the input image being one of a digitized image stored in a memory or a scanned image from a scanner, wherein the processor:

divides the input image into a plurality of blocks of pixels;

associates each block of pixels in the input image with a pixel in a feature image;

analyzes values of each pixel in the feature image to create a binary map of the feature image, wherein each pixel in the binary map corresponds to a page area or not a page area;

subjects the binary map to object segmentation to group binary map pixels corresponding to a page area;

creates a set of page objects;

detects borders of the page area; and

detects distortion within the page area.