De-warping of scanned images

• US 7,463,772 B1
• Filed: 09/13/2004
• Issued: 12/09/2008
• Est. Priority Date: 09/13/2004
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lefevere, Francois-Marie, Saric, Marin
• Primary Examiner(s): Chawan; Sheela C
• Application Number:
  US10/938,502
• Time in Patent Office: 1,548 Days
• Field of Search: 382/100, 382/131, 382/294, 382/215, 382/285, 382/154, 382/293, 382/276, 382/128, 358/474, 358/497, 358/488, 358/471, 358/493, 358/530, 358/400, 715/209, 715/274, 600/407, 600/426, 345/427, 345/419, 345/421, 345/422, 345/418, 250/208.1, 250/226, 356/402, 355/25, 355/23, 399/361, 399/362, 348/E13.025, 348/E13.003, 348/E13.002, 348/E13.001, 707/E17.11, 378/23
• US Class Current: 382/215
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1478 of characters or characters lines
• CPC Class Codes: G06V 30/1607 Correcting image deformation, e.g. trapezoidal deformation caused by perspective

Abstract

A system includes a stereoscopic camera to generate images of a document, such as a book, that can be converted to a three-dimensional contour of the document. A second camera take a two-dimensional image of the document. A model is generated that defines a mapping between the three-dimensional contour of the document and a two-dimensional flattened (de-warped) version of the document. The model may be applied to the two-dimensional image taken by the second camera to compensate for warping in the image.

Claims

1. A system comprising:

an infrared projector to project infrared radiation in an output path toward a page of a document;

a pattern mask located in the output path of the infrared projector;

an infrared stereoscopic camera to generate images of the page of the document from which a three-dimensional image of a surface of the page of the document is generated a second camera to generate a two-dimensional image of the page of the document; and

control logic to process the three-dimensional image and the two-dimensional image and to generate a model that relates the three-dimensional image of the surface of the page of the document to a flat, un-warped two-dimensional image of the page.

2. The system of claim 1, wherein the control logic processes the three-dimensional image and the two-dimensional image by using the three-dimensional image to determine warping that occurs on the page of the document.

3. The system of claim 2, wherein the control logic further corrects for the determined warping in the two-dimensional image of the page of the document.

4. The system of claim 1, wherein the second camera includes:

first and second cameras.

5. The system of claim 1, wherein the second camera is a high definition camera.

6. The system of claim 1, wherein the infrared stereoscopic camera includes two lenses configured to take stereoscopic images.

7. The system of claim 1, wherein the infrared projector projects infrared radiation visible to the infrared stereoscopic camera and not visible to the second camera.

8. The system of claim 1, wherein the control logic generates a model that relates the three-dimensional image of the surface of the page of the document to a flat, un-warped two-dimensional image of the page.

9. The system of claim 8, wherein the model is based on a page model in which the page is constrained to not shear or stretch.

10. The system of claim 8, wherein the model is generated by sequentially fitting a series of three-dimensional lines along a surface of the page and mapping the series of three-dimensional lines to a corresponding series of two-dimensional lines.

11. The system of claim 1, further comprising:

a platform for supporting the document.

12. A method comprising:

generating a model, that describes a three-dimensional profile of a surface of a document relative to a two-dimensional un-warped profile of the surface by;

locating with an infrared stereoscopic camera a first set of lines in a three-dimensional image of the surface of the document; and

determining two-dimensional lines corresponding to the first set of lines in the two-dimensional un-warped profile of the surface; and

storing the model, in a computer-readable medium, as the first set of three-dimensional lines in the three-dimensional image of the surface and the corresponding determined two-dimensional lines.

13. The method of claim 12, wherein locating the first set of lines includes:

fitting a next line in the three-dimensional image of the surface at a predetermined distance from a previous line.

14. The method of claim 13, wherein fitting the next line and determining the two-dimensional line are repeated until an end of the surface of the document is reached.

15. The method of claim 13, wherein fitting the next line further includes:

estimating a three-dimensional curve that describes a surface of the document in a local vicinity of the next line.

16. The method of claim 13, wherein determining the two-dimensional line further includes:

locating a plane that best passes through the next line and the previous line; and

determining the two-dimensional line based on the plane.

17. The method of claim 12, wherein a first line of the three-dimensional lines is a groove defined by a spine of the document.

18. A device comprising:

means for sensing infrared radiation to locate a first line in a three-dimensional image of a surface of a document;

means for fitting a next line in the three-dimensional image of the surface at a predetermined distance from the first line;

means for determining a two-dimensional line equation corresponding to the next line in a two-dimensional profile of the surface; and

means for storing a model that includes a plurality of fitted next lines and the corresponding two-dimensional line equations, the model describing a three-dimensional profile of a surface of the document relative to a two-dimensional un-warped surface of the document.

19. The device of claim 18, wherein the two-dimensional profile is determined from a pattern projected onto the surface of the document.

20. A method comprising:

generating a model, that describes a three-dimensional profile of a surface of a document relative to a two-dimensional un-warped profile of the surface by;

generating a first series of equations for a first set of points in a three-dimensional image of the surface of the document and the two-dimensional un-warped profile of the surface of the document that define stretching of the document between the first set of points;

generating a second series of equations for a second set of points in the three-dimensional image of the surface of the document and the two-dimensional un-warped profile of the surface of the document that define a shearing of the document between the second set of points; and

minimizing the stretching and shearing of the document defined by the first and second series of equations; and

storing the model, in a computer-readable medium, based on the second set of points minimized for stretching and shearing.

21. The method of claim 20, wherein the two-dimensional profile is determined from a known image pattern projected onto the surface of the document.

22. A method of compensating for distortion in a two-dimensional image of a document, the method comprising:

gathering a three-dimensional image of a contour of a surface of the document by detecting non-visible infrared radiation;

processing the three-dimensional image of the contour of the surface of the document to obtain a model relating the three-dimensional contour of the surface to a flat two-dimensional contour of the surface of the document;

projecting the two-dimensional image to a three-dimensional image based on the three-dimensional contour of the surface; and

re-projecting the projected three-dimensional image to a second two-dimensional image based on the model, where the second two-dimensional image is compensated for distortion.

23. The method of claim 22, further comprising:

projecting an infrared pattern onto the surface of the document.

24. The method of claim 22, further comprising:

performing optical character recognition (OCR) on the second two-dimensional image.

25. The method of claim 22, wherein processing the three-dimensional image of a contour of a surface of the document to obtain the model comprises:

sequentially fitting a series of three-dimensional lines along the surface of the document, and

mapping the series of three-dimensional lines to a corresponding series of two-dimensional lines.

26. The method of claim 22, wherein processing the three-dimensional image of a contour of a surface of the document to obtain the model comprises:

constraining the three-dimensional contour of the surface to the flat two-dimensional contour of the surface such that stretch and shear between the contours is minimized.

27. A method of compensating for distortion in a two-dimensional image of a document, the method comprising:

gathering three-dimensional image data of a contour of a surface of the document from an infrared stereoscopic camera;

processing the three-dimensional image data to obtain a model relating the three-dimensional contour of the surface to a flat two-dimensional contour of the surface of the document, the model being generated by sequentially fitting a series of three-dimensional lines along a surface of the document and mapping the series of three-dimensional lines to a corresponding series of two-dimensional lines;

projecting the two-dimensional image to a three-dimensional image based on the three-dimensional contour of the surface; and

re-projecting the projected three-dimensional image to a second two-dimensional image based on the model, whereby the second two-dimensional image is compensated for distortion.