Tilt adjustment for optical character recognition in portable reading machine

• US 8,320,708 B2
• Filed: 04/01/2005
• Issued: 11/27/2012
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: National Federation of the Blind
• Original Assignee: K-NFB Reading Technology Inc.
• Inventors: Kurzweil, Raymond C., Albrecht, Paul, Gibson, Lucy, Lvovsky, Lev
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US11/097,978
• Publication Number:
  US 20060071950A1
• Time in Patent Office: 2,797 Days
• Field of Search: 382/289, 382/290, 382/295, 382/296, 382/112, 382/151, 382/206, 382/281, 382/317, 356/138, 702/150
• US Class Current: 382/289
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1475 Inclination or skew detection or correction of characters or of image to be recognised
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals

Abstract

A reading machine that operates in various modes includes image correction processing is described. The reading device pre-processes an image prior to optical character recognition processing by detect distortion in an image of a page by measuring an extent to which page boundaries in the image deviate from a simple rectangular shape and correcting for the optical distortion by transforming the image to restore the page to a rectangular shape.

Claims

1. A computer program product tangibly embodied on a computer readable storage device for causing a portable reading device to pre-process a captured image prior to optical character recognition processing, comprises instructions for causing a computer to:

detect distortion in an image of a page lying in a plane by measuring an extent to which page boundaries of the page in the image deviates from a simple rectangular shape;

receive from an image input device that incorporates tilt sensors, tilt data, which data corresponding to an angle of tilt resulting from a forward/backward positioning of the image input device with respect to a plane on which the page resided;

receive data corresponding to a side to side angle of tilt from the tilt sensors to use as an approximation for an angle of rotation to transform the image;

correct the detected distortion based at least in part on the data corresponding to the angle of tilt from the tilt sensors by transforming the image to restore the page to a rectangular shape; and

perform optical character recognition on the transformed image restored to the rectangular shape.

2. The computing program product of claim 1 further comprising instructions to:

determine an angle of rotation of the page needed to align text in the page for optical character recognition on text in the page by performing edge detection on page boundaries to determine a degree of rotation of the image with respect to image input device.

3. The computing program product of claim 2 further comprising instructions to:

estimate rotation of the page by instructions to perform edge detection on the image of the page to find edge transitions at different angles, with the largest of the detected edges transitions being related to the boundaries of the page, and the angle of the edge transitions in the image providing an indication of a rotation of the page.

4. The computing program product of claim 1 wherein the data corresponding to the angle of tilt are used as an approximation for an angle of rotation to transform the image.

5. The computing program product of claim 3 further comprising instructions to:

determine a quality factor related to the number of words recognized by optical character recognition.

6. The computing program product of claim 5 further comprising instructions to:

determine if a value of the quality factor is sufficient, and when not sufficient,

select different rotation angles and determine which angle yields the highest quality factor.

7. The computing program product of claim 6 wherein instructions to determine a rotation angle include the Hough transform.

8. The computing program product of claim 7 wherein the Hough transform examines a projection of the image onto an axis at a given angle.

9. A computer-implemented method of operating a portable device, the method comprising:

pre-processing by a portable device including a computing device, a captured image of a page prior to optical character recognition processing of text on the page by;

detecting, by the computing device, distortion in an image, with the distortion caused by a tilting of an image input device at a tilt angle relative to a plane on which the page resides by measuring by the computer, an extent to which page boundaries in the image deviate from a simple rectangular shape;

receiving by the computing device from the image input device that incorporates tilt sensors, tilt data which data corresponding to the angle of tilt resulting from a forward/backward positioning of the image input device with respect to the plane on which the page resides;

receiving data corresponding to a side to side angle of tilt from the tilt sensors to use as an approximation for an angle of rotation to transform the image;

correcting by the computing device based at least in part on the data corresponding to the angle of tilt from the tilt sensors the distortion by transforming the image with the computer to restore the page to a rectangular shape; and

performing optical character recognition on the transformed image restored to the rectangular shape.

10. The method of claim 9 further comprising:

determining an angle of rotation of the page needed to align text in the page for optical character recognition on text in the page by performing edge detection on page boundaries to determine a degree of rotation of the image with respect to image input device.

11. The method of claim 10 further comprising:

estimating by the computing device rotation of the page by instructions to perform edge detection on the image of the page to find edge transitions at different angles, with the largest of the detected edges transitions being related to the boundaries of the page, and the angle of the edge transitions in the image providing an indication of a rotation of the page.

12. The method of claim 9 wherein the data corresponding to the angle of tilt are used as an approximation for an angle of rotation to transform the image.

13. The method of claim 11 further comprising:

determining a quality factor related to the number of words recognized by optical character recognition.

14. The method of claim 13 further comprising:

determining if a value of the quality factor is sufficient, and when the value is not sufficient,

selecting different rotation angles; and

determining which rotation angle yields the highest quality factor.

15. The method of claim 10 wherein determining a rotation angle is determined using the Hough transform.

16. A portable reading apparatus comprises:

a computing device;

an image input device;

tilt sensors associated with the image input device;

a computer readable medium storing a computer program product to pre-process captured images from the image input device prior to optical character recognition processing by the apparatus, the computer program product comprises instructions for causing the computing device to;

detect distortion in a captured image of a page lying in a plane by measuring an extent to which page boundaries of the page in the image deviate from a simple rectangular shape;

receive from an image input device that incorporates tilt sensors, tilt data, which data corresponding to an angle of tilt resulting from a forward/backward positioning of the image input device with respect to a plane on which the page resided;

receive data corresponding to a side to side angle of tilt from the tilt sensors to use as an approximation for an angle of rotation to transform the image;

correct the detected distortion based at least in part on the data corresponding to the angle of tilt conveyed by the data from the tilt sensors by transforming the image to restore the page boundaries to a rectangular shape; and

perform optical character recognition on the transformed image restored to the rectangular shape.

17. The apparatus of claim 16 wherein the computing program product further comprises instructions to:

determine an angle of rotation of the page needed to align text in the page for optical character recognition on text in the page by performing edge detection on page boundaries to determine a degree of rotation of the image with respect to image input device.

18. The apparatus of claim 17 wherein the computing program product further comprises instructions to:

estimate rotation of the page by instructions to perform edge detection on the image of the page to find edge transitions at different angles, with the largest of the detected edges transitions being related to the boundaries of the page, and the angle of the edge transitions in the image providing an indication of a rotation of the page.

19. The apparatus of claim 18 wherein the computing program product further comprises instructions to:

determine if a quality factor based on the number of recognized words provided from the optical character recognition is sufficient, and if not sufficient,

select different rotation angles; and

determine which rotation angle yields the highest quality factor.

20. The apparatus of claim 17 wherein the computing program product further comprises instructions to:

determine an angle of rotation of the page needed to align text in the page for optical character recognition on text in the page using the angle of tilt as an approximation for an angle of rotation to transform the image.