Camera-based scanning

• US 8,704,896 B2
• Filed: 11/14/2012
• Issued: 04/22/2014
• Est. Priority Date: 09/23/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Nijemcevic, Djordje, Vukosavljevic, Magdalena, Vugdelija, Milan, Mitrovic, Ana, Petschnigg, Georg F., Dresevic, Bodin
• Primary Examiner(s): Misleh, Justin P
• Application Number:
  US13/677,117
• Publication Number:
  US 20130070122A1
• Time in Patent Office: 524 Days
• Field of Search: None
• US Class Current: 348/207.1
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: H04N 1/00251 with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Embodiments of camera-based scanning are described. In various embodiments, one or more quadrangular objects are automatically selected from a captured image for scanning. The automatic selection is determined to be successful based on the selected quadrangular objects having an associated score that exceeds a predefined threshold. One or more scanned documents are created from portions of the captured image corresponding to the selected quadrangular objects, and the created scanned documents include corrections for perspective distortions of the selected quadrangular objects in the captured image.

Claims

1. A computing device comprising:

a memory and a processor configured to execute instructions in the memory to implement a capture application, the capture application configured to;

automatically select one or more quadrangular objects in a captured image for scanning by detecting edges in the captured image using a configurable threshold, the configurable threshold defining an amount of contrast difference sufficient to detect an edge and configurable for different portions of the image to increase in some areas of the captured image to reduce detection of false edges in the some areas and decrease in other areas of the captured image to increase edge detection within the other areas;

determine that the automatic selection of the one or more quadrangular objects is successful based on the one or more quadrangular objects having an associated score that exceeds a predefined threshold; and

create one or more scanned documents from portions of the captured image corresponding to the selected one or more quadrangular objects, the one or more scanned documents including corrections to the portions for perspective distortion of the one or more quadrangular objects in the captured image.

2. A computing device as recited in claim 1, wherein the capture application is further configured to automatically select the one or more quadrangular objects by at least:

determining the one or more quadrangular objects as combinations of the detected edges.

3. A computing device as recited in claim 2, wherein the capture application is further configured to:

employ Canny'"'"'s algorithm to detect the edges and a Hough transform to form potential quadrangles from the detected edges; and

apply a scoring scheme to the potential quadrangles to determine the one or more quadrangular objects and identify the associated score of the one or more quadrangular objects.

4. A computing device as recited in claim 1, wherein the capture application is further configured to expose a user interface having a portion to display the captured image and another portion to concurrently display the one or more scanned documents created from the captured image.

5. A computing device as recited in claim 4, wherein the user interface is configured to:

present indicators within the captured image to identify the selected one or more quadrangular objects;

enable user interaction with the indicators to make a user modification to the selected one or more quadrangular objects; and

responsive to the user interaction with the indicators, update and display the one or more scanned documents created in accordance with the user modification.

6. A computing device as recited in claim 1, wherein the capture application is further configured to:

create the one or more scanned documents using the automatic selection when the automatic selection is successful; and

in an event that the automatic selection is unsuccessful, create the one or more scanned documents using quadrangular objects in the captured image selected through a semi-automatic correction.

7. A computing device as recited in claim 1, wherein the capture application is further configured to, in an event that the automatic selection is unsuccessful, employ semi-automatic correction by at least:

receiving a user input to select a location in the captured image; and

automatically deriving a corresponding quadrangle based upon the user input, the selected location being one corner of the corresponding quadrangle.

8. A computing device as recited in claim 1, wherein the capture application is further configured to apply one or more visual enhancements to improve an appearance of the one or more scanned documents.

9. A computing device as recited in claim 1, wherein the corrections to the portions for perspective distortions comprise rectification of each of the portions of the captured image to correspond to a rectangle.

10. A method comprising:

automatically selecting a portion of a captured image that corresponds to a rectangular object in the captured image based on the portion having an associated score that exceeds a predefined scoring threshold, the selecting including detecting edges within the captured image by applying a configurable threshold defining an amount of contrast difference usable to detect an edge and re-configurable based on a number of potential edges in the portion of the captured image such that the configurable threshold is configured to increase to reduce false edge detection in the portion and decrease to increase edge detection in the portion according to the number of potential edges in the portion;

creating a scanned document from the portion of the captured image;

initiating a display of a user interface configured to present a split screen having the captured image on one side of the split screen and the scanned document created from the captured image on another side of the split screen;

enabling user interaction with the captured image via the one side of the split screen to modify the portion of the captured image; and

updating the scanned document displayed on the other side of the split screen to reflect a modification caused by the user interaction.

11. A method as recited in claim 10, wherein the image is captured via a camera of a computing device.

12. A method as recited in claim 10, further comprising:

determining lines corresponding to the detected edges;

deriving potential quadrangles by combining the detected edges and lines; and

using the derived potential quadrangles for the automatic selection.

13. A method as recited in claim 12, wherein detecting the edges comprises applying an algorithm to find sharp changes of brightness corresponding to the edges.

14. Computer-readable media having stored thereon computer-executable instructions that, if executed by a computing device, initiate the computing device to:

identify at least one quadrangle in a captured image based upon a detection of one or more potential quadrangles in the captured image, the at least one quadrangle corresponding to a rectangular object in the captured image, edges for the one or more potential quadrangles being detected in the captured image using a configurable threshold, the configurable threshold defining an amount of contrast difference sufficient to detect an edge and configurable for different portions of the image to increase in some areas of the captured image to reduce detection of false edges in the some areas and decrease in other areas of the captured image to increase edge detection within the other areas;

conduct an automatic selection of the at least one quadrangle for scanning from the captured image;

determine that the automatic selection is successful based on the selected at least one quadrangle having an associated score that exceeds a predetermined threshold; and

based on the automatic selection being successful, process the captured image to produce a scanned document corresponding to the selected at least one quadrangle.

15. Computer-readable media as recited in claim 14, wherein the computer-executable instructions, if executed by the computing device, cause the computing device to perform the detection of the one or more potential quadrangles by at least:

determining lines corresponding to the detected edges; and

deriving the one or more potential quadrangles by combining the detected edges and the determined lines.

16. Computer-readable media as recited in claim 15, wherein detecting the edges comprises applying an algorithm to find sharp changes of brightness corresponding to the edges.

17. Computer-readable media as recited in claim 15, wherein determining lines corresponding to the detected edges comprises applying a linear transform to correct imperfections in the detected edges.

18. Computer-readable media as recited in claim 14, wherein the automatic selection is conducted by at least applying a scoring scheme to the one or more potential quadrangles detected in the captured image that accounts for at least a location of the one or more potential quadrangles in the captured image and relative size of the one or more potential quadrangles in the captured image.

19. Computer-readable media as recited in claim 14, wherein the captured image is processed to produce the scanned document by at least:

cropping the captured image to provide a cropped image that corresponds to the at least one quadrangle;

un-warping the cropped image to correct for perspective distortions; and

applying one or more visual enhancements to the cropped image.

20. Computer-readable media as recited in claim 14, wherein the computer-executable instructions, if executed by the computing device, cause the computing device to output a user interface having a portion to display the captured image and another portion to concurrently display the scanned document produced by processing the captured image.