Embedded interaction code enabled surface type identification

• US 20070041654A1
• Filed: 08/17/2005
• Published: 02/22/2007
• Est. Priority Date: 08/17/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Dang, Yingnong, Wang, Jian, Zhang, Chunhui
• Granted Patent: US 7,817,816 B2
• Time in Patent Office: Days
• US Class Current: 382/260
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/19 by sensing codes defining pattern positions

Abstract

At least one image captured from a printed document and at least one image captured from a display surface other than a printed document are processed. A surface-type-identification module accepts as input both types of images and identifies, for a particular image, the type of surface from which the image was captured. A display-surface-preprocessing module preprocesses the at least one embedded-interaction-code image captured from a display surface other than a printed document. A printed-document-preprocessing module preprocesses the at least one embedded-interaction-code images captured from a printed document. An embedded-interaction-code-processing module outputs location information for at least one of the at least one image captured from a printed document and the at least one image captured from a display surface other than a printed document.

Claims

1. A system that processes at least one image captured from a printed document and that processes at least one image captured from a display surface other than a printed document, the system comprising:

a surface-type-identification module that accepts as input the at least one image captured from a printed document and the at least one image captured from a display surface other than a printed document and that identifies, for a particular image, the type of surface from which the image was captured, wherein the type of surface is selected from the group consisting of;

a printed document and a display surface other than a printed document;

a display-surface-preprocessing module that preprocesses the at least one embedded-interaction-code image captured from a display surface other than a printed document;

a printed-document-preprocessing module that preprocesses the at least one embedded-interaction-code images captured from a printed document; and

an embedded-interaction-code-processing module that outputs location information for at least one of the at least one image captured from a printed document and the at least one image captured from a display surface other than a printed document.

2. The system of claim 1, wherein the surface-type-identification module comprises:

a feature-vector generation module that generates a feature vector that specifies how even the gray level distribution is for the particular image.

3. The system of claim 1, wherein the surface-type-identification module comprises a feature-based surface-type-identification module that identifies, based on the feature vector, the type of surface from which the particular image was captured.

4. The system of claim 1, wherein the surface-type-identification module comprises a LaPlacian-filter module that uses a LaPlacian operator to filter the particular image.

5. The system of claim 1, wherein the surface-type-identification module comprises a LaPlacian-filter-based surface-type-identification module that identifies, based on the filtered image, the type of surface from which the particular image was captured.

6. The system of claim 1, wherein the location information specifies an m-array location for the at least one of the at least one image captured from a printed document and the at least one image captured from a display surface other than a printed document.

7. The system of claim 1, further comprising:

a surface-information-mapping module that maps the location information to surface metadata and x-y position information for the at least one image captured from a display surface other than a printed document.

8. The system of claim 1, further comprising:

an embedded-interaction-code-document-mapping module that maps the location information to document metadata and x-y position information for the at least one image captured from a printed document.

9. A computer-readable medium having computer-executable instructions for performing steps comprising:

generating a feature vector that specifies how even the gray level distribution is for an image that includes embedded-interaction-code information and that was captured from at least one of a printed document and a display surface other than a printed document; and

using the feature vector to identify the surface type from which the image was captured.

10. The computer-readable medium of claim 9, wherein generating the feature vector comprises calculating a horizontal projection vector and a vertical projection vector of the image.

11. The computer-readable medium of claim 10, wherein the horizontal projection vector and the vertical projection vector are calculated as:

$P_{\mathrm{horizontal}}<br><br>\left[i\right]=\underset{k=0}{\overset{w-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>I<br><br>\left(k,i\right)$

and

$P_{\mathrm{vertical}}<br><br>\left[j\right]=\underset{k=0}{\overset{h-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>I<br><br>\left(j,k\right),$

wherein I(x,y) is the gray level of pixel (x,y) in the image I and wherein w and h are the width and height, respectively, of the image.

12. The computer-readable medium of claim 9, having further computer-executable instructions for performing the step of:

defining a horizontal peak width in the horizontal projection vector and a vertical peak width in the vertical projection vector.

13. The computer-readable medium of claim 12, wherein the horizontal peak width and the vertical peak width are defined as:

[v_x v_y] that satisfies the following equations;

$\underset{k=y_{\mathrm{center}}-\frac{v_y}{2}}{\overset{y_{\mathrm{center}}-\frac{v_y}{2}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>P_{\mathrm{horizontal}}<br><br>\left[k\right]=\mathrm{\mu <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>S,<br><br>\underset{k=x_{\mathrm{center}}-\frac{v_x}{2}}{\overset{x_{\mathrm{center}}-\frac{v_x}{2}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>P_{\mathrm{vertical}}<br><br>\left[k\right]=\mathrm{\mu <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>S,<br><br>\mathrm{where}$ $S=\underset{i=0}{\overset{h-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{j=0}{\overset{w-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>I<br><br>\left(i,j\right)$

is a gray level summation of the image, μ

is a constant value, x_center and y_center are the center of the vertical and horizontal projection vector, which are defined as;

$x_{\mathrm{center}}=\frac{\underset{i=0}{\overset{w-1}{\sum <br><br>}}<br><br>{\mathrm{iP}}_{\mathrm{vertical}}<br><br>\left(i\right)}{\underset{i=0}{\overset{w-1}{\sum <br><br>}}<br><br>P_{\mathrm{vertical}}<br><br>\left(i\right)},\text{\hspace{1em}<br><br>}<br><br>y_{\mathrm{center}}=\frac{\underset{i=0}{\overset{w-1}{\sum <br><br>}}<br><br>{\mathrm{iP}}_{\mathrm{horizontal}}<br><br>\left(i\right)}{\underset{i=0}{\overset{w-1}{\sum <br><br>}}<br><br>P_{\mathrm{horizontal}}<br><br>\left(i\right)}.$

14. The computer-readable medium of claim 9, wherein using the feature vector to identify the surface type comprises using a linear classifier to distinguish between a feature vector of an image that was captured from a printed document and a feature vector of an image that was captured from a display surface other than a printed document.

15. The computer-readable medium of claim 14, wherein the linear classifier is a Fisher linear classifier.

16. A computer-readable medium having computer-executable instructions for performing steps comprising:

using a LaPlacian operator to generate a filtered image by filtering an image that includes embedded-interaction-code information and that was captured from at least one of a printed document and a display surface other than a printed document; and

using the filtered image to identify the surface type from which the image was captured.

17. The computer-readable medium of claim 16, wherein I is the image and the filtered image, I₁, is I₁(x,y)=0 if x=0 or x=w−

1or y=0 or y=h−

1, otherwise, I₁(x,y)=−

I(x−

1,y−

1)−

I(x−

1,y)−

I(x−

1,y+1)−

I(x,y−

1)+8I(x,y)−

I(x,y+1)−

I(x+1,y−

1)−

I(x+1,y)−

I(x+1,y+1).

18. The computer-readable medium of claim 16, wherein using the filtered image comprises:

counting a number of positive pixels in I₁ and a number of negative pixels in I₁.

19. The computer-readable medium of claim 18, wherein using the filtered image comprises identifying the surface type from which the image was captured as a printed document when the number of positive pixels in I₁ is greater than the number of negative pixels in I₁.

20. The computer-readable medium of claim 18, wherein using the filtered image comprises identifying the surface type from which the image was captured as a display surface other than a printed document when the number of positive pixels in I₁ is less than the number of negative pixels in I₁.