Embedded interaction code decoding for a liquid crystal display

• US 7,528,848 B2
• Filed: 06/30/2005
• Issued: 05/05/2009
• Est. Priority Date: 06/30/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Dang, Yingnong, Wang, Jian, Chen, Liyong, Zhang, Chunhui, Xu, An
• Primary Examiner(s): Awad; Amr
• Assistant Examiner(s): CERULLO, LILIANA P
• Application Number:
  US11/169,744
• Publication Number:
  US 20070003150A1
• Time in Patent Office: 1,405 Days
• Field of Search: 345/626, 345/87, 345/179, 382/269, 382/314
• US Class Current: 345/626
• CPC Class Codes: G06F 3/03545 Pens or stylus

Abstract

A method and system for decoding data on a medium that is captured by an input device is described. The method decodes a symbol design, such as an embedded interaction code (EIC), on a display screen, such as a liquid crystal display (LCD). An image that is captured by an input device, such as a universal pen-type input device with a camera, is received. The captured image is processed to output an extracted bit stream. Position data and/or metadata information is decoded from the extracted bit stream and a position of the captured image with respect to a screen of the LCD is determined based upon the decoded position data. The EIC pattern may be recognized from LCD pixel grids and a bit stream of EIC bits may be extracted based upon the recognized EIC pattern.

Claims

1. A method for decoding an embedded interaction code (EIC) pattern on a liquid crystal display (LCD), the method comprising steps of:

receiving a captured image;

processing the captured image to output an extracted bit stream; and

decoding position data from the extracted bit stream;

determining a position of the captured image with respect to a screen of the LCD based upon the decoded position data;

wherein the step of processing includes steps of,

preprocessing the captured image to determine pixels corresponding to LCD pixel grids associated with the EIC pattern;

defining the LCD pixel grids based upon the pixels;

recognizing the EIC pattern from the LCD pixel grids;

extracting a bit stream of EIC bits based upon the recognized EIC pattern to obtain the extracted bit stream;

wherein the step of preprocessing includes steps of,

separating pixels that lie on edges and vertexes of the LCD pixel grids as a first mask;

binary eroding the first mask to obtain a second mask, the second mask including fewer pixels than the first mask;

separating connected regions of the second mask in accordance with a first threshold to obtain a third mask and a fourth mask;

binary eroding the fourth mask to obtain a fifth mask;

defining a second threshold corresponding to a region of the fifth mask;

defining a sixth mask as pixels that belong to the regions of the fifth mask; and

merging the third and the sixth masks to define the LCD pixel grids.

2. The method of claim 1, wherein pixels that lie on edges or vertexes of the LCD pixel grids have a mask value of 1.

3. The method of claim 2, wherein the step of binary eroding the first mask includes steps of:

for each pixel of the first mask with a mask value of 1, determining whether at least three neighboring pixels of the pixel have a mask value of 1, and

assigning a mask value of 1 to the pixel in the second mask when the pixel and at least three neighboring pixels have a mask value of 1.

4. The method of claim 1, wherein the step of separating connected regions of the second mask includes steps of:

for each pixel of the second mask determining whether the pixel belongs to a first region or a second region based upon the first threshold, and

assigning a mask value of 1 to the pixel in the third mask and a mask value of 0 to the pixel in the fourth mask when the pixel belongs to the first region.

5. The method of claim 4, wherein the step of separating connected regions of the second mask further includes a step of assigning a mask value of 0 to the pixel in the third mask and a mask value of 1 to the pixel in the fourth mask when the pixel belongs to the second region.

6. The method of claim 1, wherein the step of binary eroding the fourth mask includes steps of:

for each pixel of the fourth mask with a mask value of 1, determining whether at least three neighboring pixels of the pixel have a mask value of 1, and

assigning a mask value of 1 to the pixel in the fifth mask when the pixel and at least three neighboring pixels have a mask value of 1.

7. A software architecture stored on one or more computer readable storage media for processing data representative of an embedded interaction code pattern, comprising:

at least one component configured to process a captured image to output an extracted bit stream, to decode position data from the extracted bit stream, and to determine a position of the captured image with respect to a screen of an LCD; and

at least one application program interface to access the component

wherein the at least one component is further configured to preprocess the captured image to determine pixels corresponding to LCD pixel grids associated with the EIC pattern, to define the LCD pixel grids based upon the pixels, to recognize the EIC pattern from the LCD pixel grids, to extract a bit stream of EIC bits based upon the recognized EIC pattern, to separate pixels that lie on edges and vertexes of the LCD pixel grids as a first mask, to binary erode the first mask to obtain a second mask, the second mask including fewer pixels than the first mask, to separate connected regions of the second mask in accordance with a first threshold to obtain a third mask and a fourth mask, to binary erode the fourth mask to obtain a fifth mask, to define a second threshold corresponding to a region of the fifth mask, to define a sixth mask as pixels that belong to the regions of the fifth mask, and to merge the third and the sixth masks to define the LCD pixel grids.

8. A computer-readable storage medium containing instructions for decoding an embedded interaction code (EIC) pattern on a liquid crystal display (LCD), by a method comprising:

receiving a captured image;

processing the captured image to output an extracted bit stream; and

decoding position data from the extracted bit stream;

determining a position of the captured image with respect to a screen of the LCD based upon the decoded position data;

wherein the step of processing includes steps of,

preprocessing the captured image to determine pixels corresponding to LCD pixel grids associated with the EIC pattern;

defining the LCD pixel grids based upon the pixels;

recognizing the EIC pattern from the LCD pixel grids; and

extracting a bit stream of EIC bits based upon the recognized EIC pattern to obtain the extracted bit stream; and

wherein the step of preprocessing includes steps of,

separating pixels that lie on edges and vertexes of the LCD pixel grids as a first mask;

binary eroding the first mask to obtain a second mask, the second mask including fewer pixels than the first mask;

separating connected regions of the second mask in accordance with a first threshold to obtain a third mask and a fourth mask;

binary eroding the fourth mask to obtain a fifth mask;

defining a second threshold corresponding to a region of the fifth mask;

defining a sixth mask as pixels that belong to the regions of the fifth mask; and

merging the third and the sixth masks to define the LCD pixel grids.

9. The computer-readable storage medium of claim 8, wherein pixels that lie on edges or vertexes of the LCD pixel grids have a mask value of 1.

10. The computer-readable storage medium of claim 9, wherein the step of binary eroding the first mask includes steps of:

for each pixel of the first mask with a mask value of 1, determining whether at least three neighboring pixels of the pixel have a mask value of 1, and

assigning a mask value of 1 to the pixel in the second mask when the pixel and at least three neighboring pixels have a mask value of 1.

11. The computer-readable storage medium of claim 8, wherein the step of separating connected regions of the second mask includes steps of:

for each pixel of the second mask determining whether the pixel belongs to a first region or a second region based upon the first threshold, and

assigning a mask value of 1 to the pixel in the third mask and a mask value of 0 to the pixel in the fourth mask when the pixel belongs to the first region.

12. The computer-readable storage medium of claim 11, wherein the step of separating connected regions of the second mask further includes a step of assigning a mask value of 0 to the pixel in the third mask and a mask value of 1 to the pixel in the fourth mask when the pixel belongs to the second region.

13. The computer-readable storage medium of claim 8, wherein the step of binary eroding the fourth mask includes steps of:

for each pixel of the fourth mask with a mask value of 1, determining whether at least three neighboring pixels of the pixel have a mask value of 1, and

assigning a mask value of 1 to the pixel in the fifth mask when the pixel and at least three neighboring pixels have a mask value of 1.