Print user interface system and its applications

• US 20030189731A1
• Filed: 07/01/2002
• Published: 10/09/2003
• Est. Priority Date: 04/06/2002
• Status: Active Grant

• Current Assignee: Celebrate International LLC
• Original Assignee: Kenneth H.P. Chang
• Inventors: Chang, Kenneth H.P.
• Granted Patent: US 7,123,742 B2
• Time in Patent Office: Days
• US Class Current: 358/3.28
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light

Abstract

A print (e.g., on paper) user interface (PUI) device for decoding an embedded message to activate a function is described. The PUI can be used for transforming an embedded message into cognizable human sensory inputs. The device includes a reader and a processor. The reader can sense light from a pattern of pixels on a printed surface and determines the pixel values of the pixels. The pattern of pixels constitutes a foreground visual image that conveys cognizable information to an observer. The primary function of the embedded message need not be to launch a Web-site but to display more information related to the cognizable information. The processor has a program of computer codes readable by the processor to recover embedded message from the pattern of pixels, the program includes codes for determining the embedded message by determining binary values based on the pixel values for pixels from the pattern of pixels and codes for performing a function according to the embedded message to result in resultant information displayable by a display suitable for human sensory reception.

Claims

1. A device for translating prints into human sensory inputs, comprising:

(a) reader for sensing light from a selected pattern of pixels on a printed surface and determining pixel values of pixels in said pattern of pixel, said pattern of pixels constituting a foreground visual image that conveys cognizable information to an observer; and

(b) processor having a program of code means readable by the processor to recover embedded message from the pattern of pixels, the program comprising;

(i) code means for determining the embedded message by determining binary values based on the pixel values for pixels from the pattern of pixels; and

(ii) code means for performing a function according to the embedded message to result in resultant information displayable by a display suitable for human sensory reception.

2. A device according to claim 1 further comprising a display.

3. A device according to claim 1 further comprising a memory containing data and said code means for performing data retrieves data from said memory not via hyperlinking to a Web-site for display.

4. A device according to claim 1 wherein the code means for determining the embedded message divides the pattern of pixels into cells, a plurality of which containing a matrix of pixels to determine said binary values.

5. A device according to claim 1 wherein the code means for determining the embedded message divides the pattern of pixels into tiles, at least some of the tiles each include cells each containing a matrix of pixels, wherein some of the cells represent the location of the corresponding tile in the pattern and some of the cells direct the function being performed.

6. A device according to claim 1 wherein the code means for determining the embedded message determines binary values using contrast of brightness to compare pixel values of selected pixels to the pixel values of pixels neighboring thereto in the pattern of pixels.

7. A device according to claim 6 wherein the reader comprises a light source for illuminating the pattern of pixels to provide light pattern of different pixel values and the reader comprises an image sensor to sense the light from a plurality of tiles simultaneously.

8. A device according to claim 1 wherein the processor directs a display to emit a signal to prompt the observer to further use the device to decode a different portion of the pattern of pixels or a different pattern of pixels, thereby providing interactive operation.

9. A device according to claim 1 wherein the code means for determining finds a central region in a view field when the view field is imaged by the reader to perform the function.

10. A device according to claim 1 wherein the code means for determining can determine from a selected pattern of pixels showing a visual image of a keyboard wherein the embedded data represent a keyboard character such that the code means for performing results in display of a character corresponding to the keyboard character sensed by the reader.

11. A device according to claim 1 wherein the code means for determining is coded to determine from a selected pattern of pixels showing a visual image of graphic or text wherein the embedded data represent pointer to one of audio information, visual information, and a combination thereof connoted by the graphic or text such that the code means for performing displays the connoted information when the reader senses the pattern of pixels showing the visual image.

12. A device according to claim 1 wherein the code means for determining is coded to determine from a selected pattern of pixels showing a visual image of a map wherein the embedded data represent secondary map information such that the code means for performing displays the secondary map information when the reader senses and decodes the pattern of pixels showing the visual image of the map.

13. A device according to claim 1 wherein the processor further contains code means to retrieve data from an Internet Web-site when the processor determines it is unable to locally obtain information requested in the pattern of pixels, wherein the processor does not directly launch a connection to said Web-site when the pattern of pixels is sensed by the reader.

14. A method for translating printed pattern into human sensory reception, comprising:

(a) sensing light from a print showing a pattern of pixels and determining pixel values of pixels in said pattern of pixels, said pattern of pixels constituting a foreground visual image that conveys cognizable information to an observer and containing an embedded message;

(b) recovering an embedded message from the pattern of pixels by determining binary values in the pattern of pixels; and

(c) performing a function according to the embedded message to result in resultant information displayable by a display suitable for human sensory reception.

15. A method according to claim 14 further comprising using contrast of brightness to compare pixel values of selected pixels to the pixel values of pixels neighboring thereto to determine the binary values.

16. A method according to claim 14 further comprising arranging the pixels in said pattern of pixels into cells each having a matrix of pixels, at least some of the cells each having one or more glyph pixels and neighboring background pixels;

and for each glyph pixel determining the binary value of the glyph pixel based on contrasting the glyph pixel'"'"'s brightness to a value derived from the brightness of the background pixels in the cell, and wherein the visual image shows text or an object.

17. A method according to claim 14 wherein the determining of the embedded data comprises dividing the pattern of pixels into tiles, at least some of the tiles each include cells, wherein some of the cells represent the location of the corresponding tile in the pattern and some of the cells direct the function being performed.

18. A method according to claim 14 wherein the determining comprises finding a central region in a view field when the pattern of pixels is sensed by the reader to perform the function.

19. A method according to claim 14 wherein the determining comprises determining data from a selected pattern of pixels showing a visual image of a keyboard wherein the embedded message directs to a keyboard character such that the code means for performing displays the character on the keyboard sensed by the reader.

20. A method according to claim 14 wherein the determining comprises determining data from a selected pattern of pixels showing a visual image of text or graphic wherein the embedded message represents pointer to one of audio information, visual information, and a combination thereof connoted by the text or graphic such that the code means for performing displays the connoted information when the reader senses the pattern of pixels showing the visual image;

and wherein the embedded message does not contain a direct hyperlink to a Web-site.

21. A method according to claim 14 wherein the determining comprises determining data from a selected pattern of pixels showing a visual image of a map wherein the embedded message directs to secondary map information such that the code means for performing displays the secondary map information when the reader senses the pattern of pixels showing the visual image.

22. A portable device for translating printed pattern of pixel into human sensory inputs, comprising:

(a) portable reader for sensing light from a selected pattern of pixels and determining pixel values of at least some of the pixels in said pattern of pixels, said pattern of pixels constituting a foreground visual image that conveys cognizable information that is one of graphic, text, and a combination thereof to an observer; and

(b) processor associated with the reader having a program of code means readable by the processor to recover embedded data from the pattern of pixels, the program comprising;

(i) code means for determining the embedded data by determining binary values based on the pixel values for pixels from the pattern of pixels; and

(ii) code means for performing a function according to the embedded data to result in resultant information displayable by a display suitable for human sensory reception; and

(c) portable display.

23. A device according to claim 22 comprising a removable memory cartridge containing data retrievable by the processor for display for human sensory input.

24. A device according to claim 22 wherein the device is adapted for interactive operation.

25. A device according to claim 22 wherein the display includes a screen capable of graphics and alpha-numeric display.

26. A device according to claim 22 wherein the display and the reader are connected by detachable electrical connection.

27. A device according to claim 22 wherein the display is the display of a portable computer.

28. A device according to claim 22 wherein the display is the display on a palm-top computer and the palm-top computer further contains data and the processor contains code means to retrieve said data from said palm-top computer for display by the processor for human sensory input.

29. A device according to claim 22 wherein the display is the display on a portable telephone, and said telephone further contains data and the processor contains code means to retrieve said data from said portable telephone.

30. A device according to claim 22 wherein the device is a handheld device and wherein the display and the reader are fixedly connected.

31. A device according to claim 22 wherein the processor performs a function according to the embedded data initially attempts to retrieve data from a location that is not a hyperlinked Web-site embedded in the pattern of pixels when the device is applied to the pattern of pixels.

32. A method for translating printed patterns of pixels into human sensory inputs, comprising:

(a) sensing light from view field on a printed surface showing a selected pattern of pixels and determining pixel value of at least some of the pixels in said pattern of pixels, said pattern of pixels constituting a foreground visual image that conveys cognizable information to an observer;

(b) recovering an embedded nonWeb-site-hyperlinked message from the pattern of pixels by determining binary values in the pattern of pixels and recognizing tiles carrying a function, wherein a cell represents a bit of information, said recovering further comprises selecting a function among groups of tiles of different functions by determining weight of the tiles; and

(c) performing a function according to the embedded message to result in resultant information displayable by a portable display suitable for human sensory reception.

33. A handheld device for translating prints into human sensory inputs, comprising:

(a) reader for sensing light from a selected pattern of pixels on a printed surface and determining pixel values of pixels in said pattern of pixels, said pattern of pixels constituting a foreground visual image that conveys cognizable information to an observer and not hyperlinked to a Web-site; and

(b) processor that is adapted to recover embedded message from the pattern of pixels, said processor being adapted to determine the embedded message by determining binary values of pixels in said pattern of pixels for performing a function according to the embedded message to result in resultant information displayable by a display suitable for human sensory reception.

34. A handheld interactive device for translating prints into human sensory inputs, comprising:

(a) reader for sensing light from a selected pattern of pixels on a printed surface and determining pixel values of pixels in said pattern of pixels, said pattern of pixels constituting a foreground visual image that conveys cognizable information to an observer and not hyperlinked to a Web-site;

(b) processor that is adapted to recover embedded message from the pattern of pixels, said processor being adapted to determine the embedded message by determining binary values of pixels in said pattern of pixels for performing a function according to the embedded message to result in resultant information displayable by a display suitable for human sensory reception, and

(c) display for displaying output, wherein the processor is adapted to in response to recovering the embedded message direct the display to prompt the observer to further use the device on another portion of the selected pattern of pixels or another selected pattern of pixels.

35. The device of claim 34 wherein the device will respond to be activated at least at two different portions of pattern of pixels to launch a display of desired information.

36. An interactive print user interface device for translating prints into human sensory inputs, comprising:

(a) document having printed patterns of pixels, said patterns of pixels constituting foreground visual images that convey cognizable information to an observer;

(b) reader for sensing light from selected portions of said patterns of pixels and determining pixel values of pixels in said patterns of pixels;

(c) display for displaying output; and

(d) processor that is adapted to recover an embedded message from a selected pattern of pixels, said processor being adapted to determine the embedded message by determining binary values of pixels in said selected pattern of pixels for performing a function according to the embedded message to result in resultant information displayable by the display suitable for human sensory reception, wherein the document provides patterns of pixels related to the selected patterns of pixels such that the observer can select the related patterns of pixels as a response to the resultant information.

37. The device of claim 36 wherein the related patterns of pixels includes embedded information to control the mechanics of display of the resultant information.

38. The device of claim 36 wherein the related patterns of pixels includes embedded information to control the mechanics of display of the resultant information for one of volume, speed, direction, pause, and exit.

39. The device of claim 36 wherein the related patterns of pixels includes embedded information to provide related resultant information that is a continuation of the resultant information of the selected pattern of pixels.

40. The device of claim 36 wherein resultant information of the selected pattern of pixels prompts the observer to select the related patterns of pixels.

41. The device of claim 36 wherein the printed pattern of pixels have glyph pixels that are printed in one of yellow, infra red, and ultra violet color to render them less conspicuous to human eyes and the reader reads light of only one of yellow, infra red, and ultra violet to recover embedded message from the printed pattern of pixels.

42. The device of claim 36 wherein the embedded message contains parameters on a function to be performed and information on location of the parameters on a function in the pattern of pixels thereby the resultant information rendered is dependent on the parameters on a function and the location of the parameters on a function.

43. A method for embedding a multidimensional coordinate system having three or more dimensions in an image, comprising:

printing pixels as a print of two dimensional matrix having a plurality of tiles, the tiles having cells with glyph pixel surrounded by background pixels, each tile in said plurality of tiles having embedded data of a set of multidimensional coordinates such that each tile has a unique data content and yet portions of the data contents of several tiles on the print are identical;

the tiles being laid in a two dimensional matrix such that a selected tile'"'"'s coordinate location on the print can be calculated by a known two dimensional location of a known tile and a relative offset in location of the selected tile to the known tile on the print.

44. A method for recovering data from a print of pixels which has tiles of cells for embedding data, the tiles having respective multidimensional coordinates, comprising:

(a) sensing pixels within a field of view on the print of pixels and decoding coordinate data of tiles in the field of view;

(b) creating a prototype tile for a selected tile by averaging coordinate data of a group of tiles in the field of view;

(c) finding error cell in the selected tile in the field of view and replacing the error cell with a cell in the prototype tile that corresponds to the error cell;

(d) deriving coordinate location of a center of the field of view by calculating a selected tile'"'"'s coordinate location and the selected tile'"'"'s relative offset from the center of the field of view; and

(e) translating the derived coordinate location into a resource pointer via a predefined mapping table.

45. The method of claim 44 further comprising selecting more than one coordinate and for each one of the selected coordinate averaging data of the coordinate over a selected group of tiles to arrive at one or more prototype tiles for replacing error cell in a selected tile.