Multi-level selection methods and apparatus using context identification for embedded data graphical user interfaces

• US 6,594,406 B1
• Filed: 12/31/1998
• Issued: 07/15/2003
• Est. Priority Date: 12/20/1996
• Status: Expired due to Term

• Current Assignee: Xerox SPA (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Hecht, David L.
• Primary Examiner(s): LEE, TOMMY D
• Application Number:
  US09/223,882
• Time in Patent Office: 1,657 Days
• Field of Search: 382/306, 382/232, 382/243, 235/494, 235/456, 235/487, 235/462.1, 235/454, 235/470, 235/432, 235/375
• US Class Current: 382/306
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06K 19/06037 multi-dimensional coding
• CPC Class Codes: G06K 7/14 using light without selection of wavelength, e.g. sensing reflected white light G06K7/10831 - G06K7/1097 take precedence
• CPC Class Codes: G06K 7/1443 locating of the code in an image

Abstract

An image capture user interface receives an image of an area of a user interface selected by a user and translates the image into operations performable by a computer. The user interface is comprised of graphic entities and embedded code. The user places an image capture device, such as a camera pen, on or near a graphic entity of the user interface, and presses a button on the image capture device indicating selection of the graphic entity. In response to the button, an image is captured that corresponds to the graphic entity selected by the user. The image includes embedded code, which is analyzed to develop an image capture code corresponding to the captured image area. The image capture code is then mapped to a selection code corresponding to the graphic entity selected by the user. The user may then make other selections. The selection codes are processed for a particular syntax, and a computer operation is performed when a selection code, or combination of selection codes, is received which indicate that an operation is to be performed. In other embodiments, mapping of image capture codes to selection codes and syntax processing may be performed in accordance with a particular context.

Claims

1. Apparatus for use with a user interface having sensory indicia and embedded data code, and an image capture device for selecting a portion of the user interface and sending image information corresponding to the portion, comprising:

an image processor for decoding first image information into a first image capture code;

a resolver for mapping the first image capture code into a first selection code; and

a syntax processor for analyzing the first selection code with a second selection code.

2. The apparatus according to claim 1, wherein the resolver comprises:

a plurality of mapping modules for mapping image capture codes to selection codes based on context.

3. The apparatus according to claim 2, wherein the syntax processor comprises:

a plurality of syntax modules for analyzing selection codes based on context.

4. The apparatus according to claim 2, wherein the syntax processor comprises:

a single syntax module for analyzing selection codes.

5. The apparatus according to claim 1, wherein the resolver comprises:

a single mapping module for mapping image capture codes to selection codes.

6. The apparatus according to claim 5, wherein the syntax processor comprises:

a plurality of syntax modules for analyzing selection codes based on context.

7. The apparatus according to claim 5, wherein the syntax processor comprises:

a single syntax module for analyzing selection codes.

8. The apparatus according to claim 1,

wherein the resolver comprises a plurality of mapping modules, one of which determines mapping based on a first context;

and

wherein the syntax processor comprises a plurality of syntax modules, one of which determines an computer system operation based on the first context.

9. The apparatus according to claim 1, wherein the sensory indicia comprises a graphic element.

10. The apparatus according to claim 1, wherein the embedded data code:

comprises glyphs.

11. The apparatus according to claim 1, wherein the embedded data code comprises an address code.

12. The apparatus according to claim 1, wherein the embedded data code comprises a glyph address code.

13. The apparatus according to claim 1, wherein the embedded data code comprises a label code associated with a graphic entity.

14. The apparatus according to claim 1, wherein the user interface comprises:

a first user interface portion including a first graphic entity on a first substrate, wherein the first graphic entity is associated with the first selection code; and

a second user interface portion including a second graphic entity on a second substrate, wherein the second graphic entity is associated with the second selection code.

15. The apparatus according to claim 14, wherein the first user interface portion comprises a dynamic display.

16. The apparatus according to claim 1, wherein the user interface comprises:

a first user interface portion having an address space on a first substrate corresponding to a first context code; and

a second user interface portion having an address space on a second substrate corresponding to a second context code.

17. A method for use with a user interface having sensory indicia and embedded data code, and an image capture device for selecting a portion of the user interface and sending image information corresponding to the portion, comprising:

decoding first image information into a first image capture code;

mapping the first image capture code into a first selection code; and

analyzing the first selection code with a second selection code.

18. The method according to claim 17, wherein the step of mapping comprises:

mapping image capture codes to selection codes based on context using a plurality of mapping modules.

19. The method according to claim 18, wherein the step of analyzing comprises:

analyzing selection codes based on context using a plurality of syntax modules.

20. The method according to claim 18, wherein the step of analyzing comprises:

analyzing selection codes using a single syntax module.

21. The method according to claim 17, wherein the step of mapping comprises:

mapping image capture codes to selection codes using a single mapping module.

22. The method according to claim 21, wherein the step of analyzing comprises:

analyzing selection codes based on context using a plurality of syntax modules.

23. The method according to claim 21, wherein the step of analyzing comprises:

analyzing selection codes using a single syntax module.

24. The method according to claim 17, further comprising:

determining a mapping using a mapping module corresponding to a context; and

determining a computer system operation using a mapping module corresponding to the context.

25. The method according to claim 17, wherein the sensory indicia comprises a graphic element.

26. The method according to claim 17, wherein the embedded data code comprises glyphs.

27. The method according to claim 17, wherein the embedded data code comprises an address code.

28. The method according to claim 17, wherein the embedded data code comprises a glyph address code.

29. The method according to claim 17, wherein the embedded data code comprises a label code associated with a graphic entity.

30. The method according to claim 17, wherein the user interface comprises:

a first user interface portion including a first graphic entity on a first substrate, wherein the first graphic entity is associated with the first selection code; and

a second user interface portion including a second graphic entity on a second substrate, wherein the second graphic entity is associated with a second selection code.

31. The method according to claim 30, wherein the first user interface portion comprises a dynamic display.

32. The method according to claim 17, further comprising:

providing a first address space having an address space on a first substrate corresponding to a first context code; and

providing a second interface having an address space on a second substrate corresponding to a second context code.

33. A graphical user interface for use with a user interface system including a device for capturing a portion of the graphical user interface in response to selection of sensory indicia, and a computer system responsive to embedded code in the portion for performing an operation related to the sensory indicia and a first selection code, comprising:

sensory indicia; and

embedded data codes having a predetermined spatial relationship to the sensory indicia, wherein the embedded data codes comprise a context code.

34. The graphical user interface according to claim 33, wherein the embedded data codes comprise glyphtones.

35. The graphical user interface according to claim 33, wherein the embedded codes comprise:

an address carpet for relating selected locations in the user interface to corresponding computer system entities.

36. The graphical user interface according to claim 33, wherein the embedded codes comprise glyphs.

37. The graphical user interface according to claim 33, wherein the embedded codes comprise:

a label for relating the sensory indicia to a computer system entity.

38. The graphical user interface according to claim 33, wherein the embedded data codes comprise a distributed label context code.

39. Apparatus for determining computer system operations, comprising:

a user interface having sensory indicia and embedded data code;

an image capture device for selecting a portion of the user interface and sending image information corresponding to the portion;

an image processor for decoding image information into a capture code;

a resolver for mapping the image capture code into a selection code; and

a syntax processor for analyzing multiple selection codes to determine corresponding computer system operations.

40. The apparatus according to claim 39, wherein the resolver comprises:

a plurality of mapping modules for mapping image capture codes to selection codes based on context.

41. The apparatus according to claim 40, wherein the syntax processor comprises:

a plurality of syntax modules for analyzing selection codes based on context.

42. The apparatus according to claim 40, wherein the syntax processor comprises:

a single syntax module for analyzing selection codes.

43. The apparatus according to claim 39, wherein the resolver comprises:

a single mapping module for mapping image capture codes to selection codes.

44. The apparatus according to claim 43, wherein the syntax processor comprises:

a plurality of syntax modules for analyzing selection codes based on context.

45. The apparatus according to claim 43, wherein the syntax processor comprises:

a single syntax module for analyzing selection codes.

46. The apparatus according to claim 39,

wherein the resolver comprises a plurality of mapping modules, one of which determines mapping based on a first context;

and

wherein the syntax processor comprises a plurality of syntax modules, one of which determines an computer system operation based on the first context.

47. The apparatus according to claim 39, wherein the sensory indicia comprises:

a graphic element.

48. The apparatus according to claim 39, wherein the embedded data code comprises glyphs.

49. The apparatus according to claim 39, wherein the embedded data code comprises an address code.

50. The apparatus according to claim 39, wherein the embedded data code comprises a glyph address code.

51. The apparatus according to claim 39, wherein the embedded data code comprises a label code associated with a graphic entity.

52. The apparatus according to claim 39, wherein the user interface comprises:

a first user interface portion including a first graphic entity on a first substrate, wherein the first graphic entity is associated with the first selection code; and

a second user interface portion including a second graphic entity on a second substrate, wherein the second graphic entity is associated with a second selection code.

53. The apparatus according to claim 52, wherein the first user interface portion comprises a dynamic display.

54. The apparatus according to claim 39, wherein the user interface comprises:

a first user interface portion having an address space on a first substrate corresponding to a first context code; and

a second user interface portion having an address space on a second substrate corresponding to a second context code.