Document portion identification by fast image mapping

• US 7,885,465 B2
• Filed: 09/16/2008
• Issued: 02/08/2011
• Est. Priority Date: 01/07/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Chen, Liyong, Ma, Xiaoxu, Dang, Yingnong, Wang, Jian
• Primary Examiner(s): ZARKA, DAVID PETER
• Application Number:
  US12/211,473
• Publication Number:
  US 20090016614A1
• Time in Patent Office: 875 Days
• Field of Search: 382/218
• US Class Current: 382/218
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus

Abstract

An efficient technique is provided for determining a portion of a document corresponding to a captured image. Areas of a document in which the pattern is at least partially obscured are identified. A reference pixel in the image is selected, and an offset between the pixel and the pattern is determined. A pixel-by-pixel comparison is then made of the image with the document such that the reference pixel is only compared with locations in the document that are both within the identified areas and have the determined offset from the pattern. The comparison with the highest correspondence between the image pixels and the electronic document then identifies the position of the reference pixel relative to the electronic document.

Claims

1. A method of determining a portion of a document corresponding to a captured image, the method comprising:

employing a processor that executes instructions retained in a computer-readable medium, the instructions when executed by the processor implement at least the following operations;

receiving a captured image with an image capturing device of a portion of a document stored in electronic form, wherein the document has a pattern that uniquely identifies different positions in the document and wherein the captured image contains a portion of the pattern, and the portion of the pattern is at least partially obscured by overlaid printed content of the electronically stored document;

determining obscured areas of the document in which the pattern is at least partially obscured;

comparing the captured image with determined obscured areas of the electronically stored document;

identifying the position of the portion of the document corresponding to the captured image;

determining a spatial relationship between a reference pixel in the captured image and the pattern within the captured image, the determined spatial relationship is a shortest distance measured by a pixel offset between the reference pixel and an unobscured portion of the pattern; and

identifying locations in the obscured areas of the document exhibiting a similar spatial relationship to the pattern in the obscured areas.

2. The method according to claim 1 further comprising:

comparing the captured image with portions of the obscured areas of the document associated with the identified locations.

3. The method according to claim 2, further comprising:

warping the captured image so that the scale and orientation of the captured image matches the scale and orientation of the document.

4. The method according to claim 3, further comprising:

repeating a comparison of the captured image with a particular portion of the document for each 90″

rotation of the captured image.

5. The method according to claim 2, further comprising:

identifying the position of the portion of the document corresponding to the captured image based on the comparisons which produce a correlation of 50% or greater.

6. The method according to claim 2, further comprising:

identifying the position of the captured image to be the portion of the document corresponding to the comparisons which produce a correlation at least 10% higher than all other comparison correlation values.

7. The method according to claim 1, wherein determining obscured areas of the document comprises determining, for a given pixel of the document, whether the amount of the pattern that is obscured in a portion of the document around the given pixel is more than a predetermined amount.

8. The method according to claim 7, wherein the portion of the document around a given pixel is the same size as the captured image.

9. The method according to claim 7, wherein the predetermined amount represents an amount of obscuring of the pattern such that the pattern in that portion of the document does not uniquely identify the position of the portion within the document.

10. The method according to claim 1, wherein the pattern is randomly or pseudorandomly generated.

11. One or more computer-readable storage media storing computer-executable instructions that, when executed, causes one or more processors to perform operations comprising:

receiving a captured image of a portion of a document, the document having a pattern that uniquely identifies different positions in the document and the captured image containing a portion of the pattern that uniquely identifies different positions in the document, and the portion of the pattern being at least partially obscured by overlaid printed content of the electronically stored document;

determining obscured areas of the document in which the pattern is at least partially obscured;

comparing the captured image with determined obscured areas of the document;

identifying the position of the portion of the document corresponding to the captured image;

determining a spatial relationship between a reference pixel in the captured image and the pattern within the captured image, the determined spatial relationship is a shortest distance measured by a pixel offset between the reference pixel and an unobscured portion of the pattern; and

identifying locations in the obscured areas of the document exhibiting a similar spatial relationship to the pattern in the obscured areas.

12. The one or more computer-readable storage media according to claim 11 the program further comprising:

comparing the captured image with portions of the obscured areas of the document associated with the identified locations.

13. The one or more computer-readable storage media according to claim 12, the program further comprising:

warping the captured image so that the scale and orientation of the captured image matches the scale and orientation of the document.

14. The one or more computer-readable storage media according to claim 13, the program further comprising:

repeating a comparison of the captured image with a particular portion of the document for each 90″

rotation of the captured image.

15. The one or more computer-readable storage media according to claim 12, the program further comprising:

identifying the position of the portion of the document corresponding to the captured image based on the comparisons which produce a correlation of 50% or greater.

16. The one or more computer-readable storage media according to claim 12, the program further comprising:

identifying the position of the captured image to be the portion of the document corresponding to the comparison which produces a correlation at least 10% higher than all other comparison correlation values.

17. The one or more computer-readable storage media according to claim 11, wherein determining obscured areas of the document comprises determining, for a given pixel of the document, whether the amount of the pattern that is obscured in a portion of the document around the given pixel is more than a predetermined amount.

18. The one or more computer-readable storage media according to claim 17, wherein the portion of the document around a given pixel is the same size as the captured image.

19. The one or more computer-readable storage media according to claim 17, wherein the predetermined amount represents an amount of obscuring of the pattern such that the pattern in that portion of the document does not uniquely identify the position of the portion within the document.

20. The one or more computer-readable storage media according to claim 11, wherein the pattern is randomly or pseudorandomly generated.