Embedding a pattern design onto a liquid crystal display

• US 7,619,607 B2
• Filed: 06/30/2005
• Issued: 11/17/2009
• Est. Priority Date: 06/30/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Qiang, Wang, Jian, Zhang, Chunhui
• Primary Examiner(s): Awad; Amr
• Assistant Examiner(s): CERULLO, LILIANA P
• Application Number:
  US11/169,889
• Publication Number:
  US 20070001950A1
• Time in Patent Office: 1,601 Days
• Field of Search: 345/694, 345/695, 345/88, 345/690, 345/87, 345/104, 345/207, 382/314, 382/317, 382/100, 382/182, 382/181, 358/3.28
• US Class Current: 345/104
• CPC Class Codes: G06F 3/037 using the raster scan of a cathode-ray tube [CRT] for detecting the position of the member, e.g. light pens cooperating with CRT monitors

Abstract

A method and system for presenting data on a medium for capture by an input device is described. The method embeds a symbol design, such as an embedded interaction code (EIC), in an image on a display screen, such as a liquid crystal display (LCD). A grid with a plurality of pixels defines a size of an EIC pattern on an LCD. An area of the grid is used to embed position data and/or metadata information. The area may be defined by a sub-pixel of the plurality of pixels and/or the edges of the grid itself. The sub-pixel with the EIC pattern may be configured to have a grey value based upon a color of an underlying image associated with an application program being displayed on the LCD. The method also allows for decoding of the position data and/or metadata information in the EIC pattern on the LCD.

Claims

1. A method for creating a pattern on a liquid crystal display (LCD) and decoding position data from the pattern, the method comprising steps of:

defining a size of the pattern on an LCD by a grid with a plurality of pixels, each pixel having sub-pixels with an adjustable intensity value;

determining an area of the grid for embedding position data, wherein the position data provides an indication of position relative to the LCD and wherein the position data is comprised of a plurality of dots; and

embedding the position data in the determined area of the grid by;

for each dot associated with the position data,

selecting one of the plurality of pixels of the grid corresponding to the position of the dot of the pattern,

selecting a sub-pixel of the selected pixel, and

adjusting the intensity value of the selected sub-pixel to indicate its correspondence with the dot wherein the adjusted intensity value of the selected sub-pixel is based on an underlying image displayed on the LCD;

receiving a captured image;

decoding position data from the pattern, wherein decoding position data includes,

processing the captured image to output an extracted bit stream by,

preprocessing the captured image to determine pixels corresponding to LCD pixel grids associated with the embedded interaction code pattern, wherein preprocessing includes;

separating pixels that lie on edges and vertexes of the LCD pixel grids as a first mask,

binary eroding the first mask to obtain a second mask, the second mask including fewer pixels than the first mask,

separating connected regions of the second mask in accordance with a first threshold to obtain a third mask and a fourth mask,

binary eroding the fourth mask to obtain a fifth mask,

defining a second threshold corresponding to a region of the fifth mask,

defining a sixth mask as pixels that belong to the regions of the fifth mask, and

merging the third and the sixth masks to define the LCD pixel grids,

defining the LCD pixel grids based upon the pixels,

recognizing the embedded interaction code pattern from the LCD pixel grids,

extracting a bit stream of embedded interaction code bits based upon the recognized embedded interaction code pattern to obtain the extracted bit stream;

decoding position data from the extracted bit stream, and

determining a position of the captured image relative to the LCD based upon the decoded position data

so that the position of an image captured from the LCD can be determined relative to the LCD based on the adjusted intensity values of the sub-pixels.

2. The method of claim 1, wherein the determined area of the grid includes a pixel of the plurality of pixels and the step of embedding includes embedding the position data in a sub-pixel of the pixel.

3. The method of claim 2, wherein a pixel includes three sub-pixels including a red channel sub-pixel, a green channel sub-pixel, and a blue channel sub-pixel.

4. The method of claim 3, wherein the step of embedding the position data in the sub-pixel includes embedding the position data in the red channel sub-pixel.

5. The method of claim 4 wherein the grey value of the red channel sub-pixel equals zero and the grey value of the green channel sub-pixel and the blue channel sub-pixel equals 255.

6. The method of claim 4, wherein the intensity value of the red channel sub-pixel is based upon a color of an underlying image displayed on the LCD.

7. The method of claim 1, wherein the step of decoding includes a step of determining the orientation of the pattern.

8. The method of claim 1, wherein the size of the pattern is defined by columns 1 to M and rows 1 to N, M and N being integers.

9. The method of claim 8, wherein M and N are equal in value.

10. The method of claim 8, wherein the step of embedding includes embedding the position data in one of columns 1 to M−

1 and in one of rows 1 to N−

1.

11. The method of claim 1, wherein the position data includes two bits of data to define a position along an x axis and a y axis.

12. The method of claim 1, further comprising a step of displaying the EIC pattern on the LCD by intermittently flashing the EIC pattern.

13. The method of claim 1, further comprising a step of displaying the pattern on the LCD by watermarking an image on the LCD with the pattern.

14. The method of claim 1, wherein the determined area of the grid includes first and second pixels of the plurality of pixels and the step of embedding includes embedding the position data in a sub-pixel of the first pixel and a sub-pixel of the second pixel, wherein the intensity value of the first sub-pixel is different from the intensity value of the second sub-pixel and is based upon a color of an underlying image displayed on the LCD.

15. A system for creating a pattern on a liquid crystal display (LCD), the system comprising:

a design component configured to define a size of the pattern for display on the LCD, the size defined by a grid with a matrix formed by a plurality of pixels arranged in a plurality of columns and a plurality of rows, each pixel having sub-pixels with adjustable intensity values;

a processor component configured to determine an area of the grid for embedding position data relative to the LCD;

an embedding component configured to, when the determined area is a pixel, embed the position data relative to the LCD in the determined area of the grid by adjusting the intensity value of a sub-pixel of at least one of the plurality of pixels;

an embedding component configured to, when the determined area is not a pixel, embed the position data relative to the LCD in the determined area of the grid by adjusting the intensity value of a portion of the matrix, wherein the position data is not based on an underlying image displayed on the LCD; and

an LCD configured to display the position data in the pattern wherein the location of the position data is not based on an underlying image displayed on the LCD.

16. The system of claim 15, further comprising a decoder component configured to decode the embedded position data.

17. The system of claim 16, wherein the decoder component includes a camera configured to capture an image of the pattern and an extracting component configured to extract the embedded position data from the captured image of the pattern.

18. The system of claim 15, wherein the LCD is further configured to display the position data in the EIC pattern by intermittently flashing the EIC pattern on the LCD.