Hand drawing capture via interface surface

• US 7,456,820 B1
• Filed: 05/23/2000
• Issued: 11/25/2008
• Est. Priority Date: 05/25/1999
• Status: Expired due to Fees

• Current Assignee: Livescribe Incorporated (Anoto Group AB)
• Original Assignee: Silverbrook Research Pty Limited
• Inventors: Lapstun, Paul, Silverbrook, Kia
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): ABDULSELAM, ABBAS I
• Application Number:
  US09/575,172
• Time in Patent Office: 3,108 Days
• Field of Search: 345/156, 345/179, 345/51, 345/52, 345/166, 345/173, 345/177, 345/180, 178/18.01, 715/862, 358/3.28, 717/106
• US Class Current: 345/156
• CPC Class Codes: B41J 13/103 for the sheet feeding section
• CPC Class Codes: B41J 13/106 for the sheet output section
• CPC Class Codes: B41J 2/17503 Ink cartridges
• CPC Class Codes: B41J 2/17513 Inner structure
• CPC Class Codes: B41J 2/17523 Ink connection
• CPC Class Codes: B41J 2/17546 electronically
• CPC Class Codes: B41J 2/17553 Outer structure
• CPC Class Codes: B41J 2/17556 Means for regulating the pressure in the cartridge
• CPC Class Codes: B41J 2/17563 Ink filters
• CPC Class Codes: B41J 2/2114 Ejecting transparent or white coloured liquids, e.g. processing liquids B41J2/211 takes precedence
• CPC Class Codes: B42C 19/02 starting with single sheets
• CPC Class Codes: B42C 9/0006 by applying adhesive to a stack of sheets
• CPC Class Codes: B42C 9/0081 applying adhesive to individual sheets for binding them together by fusing, welding B42C9/0093
• CPC Class Codes: B42P 2261/04 Producing books by printing sheets in following order, e.g. for ordering via the Internet
• CPC Class Codes: B65H 29/34 from supports slid from under the articles
• CPC Class Codes: B65H 37/04 for securing together articles or webs, e.g. by adhesive, stitching or stapling adhering replacement to expiring web during change of web roll B65H19/18
• CPC Class Codes: G06F 21/608 Secure printing
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 2221/2153 Using hardware token as a secondary aspect
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 : Pens or stylus
• CPC Class Codes: G06F 3/03546 : using a rotatable ball at the tip as position detecting member
• CPC Class Codes: G06F 3/038 : Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0386 : for light pen
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1204 : resulting in reduced user or operator actions, e.g. presetting, automatic actions, using hardware token storing data
• CPC Class Codes: G06F 3/1265 : Printing by reference, e.g. retrieving document/image data for a job from a source mentioned in the job
• CPC Class Codes: G06F 3/1284 : Local printer device
• CPC Class Codes: G06K 17/00 : Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 30/1423 : the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/1444 : Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: H04L 2209/56 : Financial cryptography, e.g. electronic payment or e-cash
• CPC Class Codes: H04L 2209/603 : Digital right managament [DRM]
• CPC Class Codes: H04L 2209/805 : Lightweight hardware, e.g. radio-frequency identification [RFID] or sensor
• CPC Class Codes: H04L 9/3247 : involving digital signatures
• CPC Class Codes: H04L 9/3263 : involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements network architectures or network communication protocols for supporting authentication of entities using certificates in a packet data network H04L63/0823
• CPC Class Codes: H04N 1/00127 : Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/00358 : Type of the scanned marks
• CPC Class Codes: H04N 1/00376 : Means for identifying a mark sheet or area
• CPC Class Codes: H04N 1/00567 : Handling of original or reproduction media, e.g. cutting, separating, stacking
• CPC Class Codes: H04N 1/00968 : by scanning marks on a sheet mark-sheet input means H04N1/00355; sensing record carriers G06K7/00
• CPC Class Codes: H04N 1/32101 : Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 1/32122 : in a separate device, e.g. in a memory or on a display separate from image data
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/32771 : in response to a request, e.g. for a particular document
• CPC Class Codes: H04N 1/32778 : using a mark-sheet or machine-readable code request
• CPC Class Codes: H04N 2201/0082 : Image hardcopy reproducer H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3205 : of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 2201/3207 : of an address
• CPC Class Codes: H04N 2201/3226 : of identification information or the like, e.g. ID code, index, title, part of an image, reduced-size image
• CPC Class Codes: H04N 2201/3242 : of processing required or performed, e.g. for reproduction or before recording H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3243 : of type information, e.g. handwritten or text document
• CPC Class Codes: H04N 2201/3247 : Data linking a set of images to one another, e.g. sequence, burst or continuous capture mode
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3269 : of machine readable codes or marks, e.g. bar codes or glyphs
• CPC Class Codes: H04N 2201/327 : which are undetectable to the naked eye, e.g. embedded codes
• CPC Class Codes: H04N 2201/3271 : Printing or stamping
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: Y10S 707/99942 : Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A method and system for enabling user interaction with computer software running in a computer system via an interface surface and a sensing device. The interface surface contains information relating to the computer software and coded data indicative of a drawing field. When placed in an operative position relative to the interface surface, the sensing device senses indicating data indicative of the drawing field and generates movement data indicative of the sensing device'"'"'s movement relative to the interface surface. The indicating and movement data are received in the computer system from the sensing device. The drawing field is identified and then the computer software is operated at least partly in reliance on the movement data, and in accordance with instructions associated with the drawing field.

Claims

1. A method of enabling user interaction with computer software running in a computer system via:

an interface surface containing information relating to the computer software and a drawing input field and having disposed therein or thereon coded data at least partially indicative of the drawing input field; and

a sensing device which, when placed in an operative position relative to the interface surface, senses at least some of the coded data and uses the sensed coded data to generates indicating data at least partially indicative of the drawing field, and generates movement data indicative of the sensing device'"'"'s movement relative to the interface surface;

the method including the steps of, in the computer system;

(a) receiving the indicating data from the sensing device;

(b) receiving the movement data from the sensing device, at least part of said movement data representing drawing input by the user;

(c) identifying the drawing input field at least partially from the indicating data; and

(d) operating the computer software at least partly in reliance on the drawing input, and in accordance with instructions associated with the drawing input field.

2. A method according to claim 1, the method further including the steps of, in the computer system, associating the movement data with the drawing field.

3. A method according to claim 2, wherein the drawing field is associated with a visible drawing zone defined on the interface surface.

4. A method according to claim 1, including the step of sending, in the computer system, data to the computer software indicative of at least the drawing field.

5. A method according to claim 1, wherein the coded data includes at least one tag, each tag being indicative of the drawing field.

6. A method according to claim 5, wherein the tags are also indicative of points within the drawing field.

7. A method according to claim 6, wherein each of the tags includes:

first identity data defining a relative position of that tag; and

second identity data identifying the drawing field.

8. A method according to claim 7, wherein the relative position is defined in relation to the drawing field.

9. A method according to claim 7, wherein the relative position is defined in relation to a plurality of the other tags.

10. A method according to claim 7, wherein the relative position is defined in relation to the interface surface.

11. A method according to claim 7, wherein the first identity data identifies stored information defining the relative position, the stored information not being stored on the interface surface.

12. A method according to claim 11, wherein the first identity data and the second identity data together identify stored information defining the relative position.

15. A method according to any one of the preceding claims, wherein the sensing device includes at least one acceleration measuring device for measuring acceleration of the sensing device as it is used to draw the hand-drawn user input onto the surface, the movement data being generated by periodically sampling the acceleration of the sensing device as it is used to draw the hand-drawn user input onto the surface.

16. A method according to claim 15, further including the step of generating movement data in the form of a locus of the sensing device in relation to the surface, the locus being determined by ascertaining relative displacement of the sensing device.

17. A method according to claim 16, wherein the relative displacement is obtained by doubly integrating the acceleration with respect to time.

18. A method according to claim 16 or 17, wherein the acceleration measuring device includes one or more accelerometers configured to measure at least two orthogonal components of acceleration.

19. A method according to any one of claims 1 to 14, wherein position elements are disposed on the interface surface, the sensing device being configured to periodically sense position elements as it is used to draw the hand-drawn user input onto the surface, the method including the step of generating the movement data in the form of a locus of the sensing device in relation to the surface by ascertaining relative displacement of the sensing device over time with respect to at least one of the position elements.

20. A method according to claim 19, wherein the position elements are disposed on the surface as a regular array of dots, lines or other formations.

21. A method according to claim 19, wherein the position elements are disposed on the surface stochastically.

22. A method according to any one of claims 1 to 14, wherein the movement data is generated by ascertaining relative movement of one or more motion sensing elements rotatably mounted to the sensing device for contact with the surface while the sensing device is used to draw the hand-drawn user input thereon.

23. A method according to claim 22, wherein the motion sensing elements include one or more rollerballs mounted for rotation within a constraining housing disposed substantially within the sensing device.

24. A method according to claim 23, wherein components of rotation of the rollerball, due to movement of the sensing device when drawing the hand-drawn user input onto the surface, are periodically measured.

25. A method according to claim 24, wherein the components of rotation of the rollerball due to movement of the sensing device by the user when drawing the hand-drawn user input onto the surface are measured by means of:

rollers disposed within the constraining housing for rotation, the rollers being configured to be driven by contact with the rotating rollerball;

or

optical sensing of rotation of the rollerball with respect to the constraining housing.

29. The method of claim 1 or claim 13, the sensing device generating at least the movement data in accordance with movement of the sensing device with respect to the coded data.

30. The method of claim 1 or claim 13, the method including printing the interface surface with a printer.

31. The method of claim 1 or claim 13, the indicating data representing at least an identity of the interface surface, the method including identifying the drawing input field in the computer system by:

determining the identity of the interface surface;

using the identity to determine a description of the interface surface; and

,

identifying the drawing input field from the description and at least the movement data.

13. A method of enabling user interaction with computer software running in a computer system via:

an interface surface containing information relating to the computer software and having coded data disposed therein or thereon, the coded data being at least partially indicative of an identity of the interface surface; and

a sensing device which, when placed in an operative position relative to the interface surface, senses at least some of the coded data and uses the sensed coded data to generates indicating data at least partially indicative of the identity of the interface surface, and generates movement data indicative of the sensing device'"'"'s movement relative to the interface surface;

the method including the steps of, in the computer system;

(a) receiving the indicating data from the sensing device;

b) receiving the movement data from the sensing device, at least part of the said movement data representing written gesture input by the user;

(c) performing written gesture recognition in relation to at least some of the movement data; and

(d) in the event that a written gesture is recognised, operating the computer system in accordance with instructions associated with the written gesture and the interface surface.

14. A method according to claim 13, wherein the selection gesture includes circumscribing or underlining at least some of the information.

26. A method according to claim 13, wherein the coded data includes at least one tag, each tag being indicative of the drawing field.

27. A method according to claim 26, wherein the tags are also indicative of points of the interface surface.

28. A method according to claim 27, wherein each of the tags includes:

first identity data defining a relative position of that tag; and

second identity data identifying the interface surface.

32. A system for enabling user interaction with computer software running in a computer system via:

an interface surface containing information relating to the computer software and a drawing input field and having disposed therein or thereon coded data at least partially indicative of the drawing input field; and

a sensing device which, when placed in an operative position relative to the interface surface, senses at least some of the coded data and uses the sensed coded data to generates indicating data at least partially indicative of the drawing input field, and generates movement data indicative of the sensing device'"'"'s movement relative to the interface surface;

the computer system being configured to;

(a) receive the indicating data from the sensing device;

(b) receive the movement data from the sensing device, at least part of said movement data representing drawing input by the user;

(c) identify the drawing input field at least partially from the indicating data; and

(d) operate the computer software at least partly in reliance on the drawing input, and in accordance with instructions associated with the drawing input field.

33. A system according to claim 32, the computer system further being configured to associate the movement data with the drawing field.

34. A system according to claim 33, wherein the drawing field is associated with a visible drawing zone defined on the interface surface.

35. A system according to claim 32, the computer system being configured to send data to the computer software indicative of at least the drawing field.

36. A system according to claim 32, wherein the coded data includes at least one tag, each tag being indicative of the drawing field.

37. A system according to claim 36, wherein the tags are also indicative of points within the drawing field.

38. A system according to claim 37, wherein each of the tags includes:

first identity data defining a relative position of that tag; and

second identity data identifying the drawing field.

39. A system according to claim 38, wherein the relative position is defined in relation to the drawing field.

40. A system according to claim 38, wherein the relative position is defined in relation to a plurality of the other tags.

41. A system according to claim 38, wherein the relative position is defined in relation to the interface surface.

42. A system according to claim 38, wherein the first identity data identifies stored information defining the relative position, the stored information not being stored on the interface surface.

43. A system according to claim 42, wherein the first identity data and the second identity data together identify stored information defining the relative position.

46. A system according to any one of claims 32 to 45, wherein the sensing device includes at least one acceleration measuring device for measuring acceleration of the sensing device as it is used to draw the hand-drawn user input onto the surface, the movement data being generated by periodically sampling the acceleration of the sensing device as it is used to draw the hand-drawn user input onto the surface.

47. A system according to claim 46, wherein the computer system is configured to generate the movement data in the form of a locus of the sensing device in relation to the surface, the locus being determined by ascertaining relative displacement of the sensing device.

48. A system according to claim 47, wherein the relative displacement is obtained by doubly integrating the acceleration with respect to time.

49. A system according to claim 47 or 48, wherein the acceleration measuring device includes one or more accelerometers configured to measure at least two orthogonal components of acceleration.

50. A system according to any one of claims 32 to 49, wherein position elements are disposed on the interface surface, the sensing device being configured to periodically sense position elements as it is used to draw the hand-drawn user input onto the surface, the computer system being configured to generate the movement data in the form of a locus of the sensing device in relation to the surface by ascertaining relative displacement of the sensing device over time with respect to at least one of the position elements.

51. A system according to claim 50, wherein the position elements are disposed on the surface as a regular array of dots, lines or other formations.

52. A system according to claim 50, wherein the position elements are disposed on the surface stochastically.

53. A system according to any one of claims 32 to 52, wherein the movement data is generated by ascertaining relative movement of one or more motion sensing elements rotatably mounted to the sensing device for contact with the surface while the sensing device is used to draw the hand-drawn user input thereon.

54. A system according to claim 53, wherein the motion sensing elements include one or more rollerballs mounted for rotation within a constraining housing disposed substantially within the sensing device.

55. A system according to claim 54, wherein components of rotation of the rollerball, due to movement of the sensing device when drawing the hand-drawn user input onto the surface, are periodically measured.

56. A system according to claim 55, wherein the components of rotation of the rollerball due to movement of the sensing device by the user when drawing the hand-drawn user input onto the surface are measured by means of:

rollers disposed within the constraining housing for rotation, the rollers being configured to be driven by contact with the rotating rollerball;

or

optical sensing of rotation of the rollerball with respect to the constraining housing.

60. The system of claim 32 or claim 44, the sensing device generating at least the movement data in accordance with movement of the sensing device with respect to the coded data.

61. The system of claim 32 or claim 44, the system including a printer for printing the interface surface.

62. The system of claim 32 or claim 44, the indicating data representing at least an identity of the interface surface, the method including identifying the drawing input field in the computer system by:

determining the identity of the interface surface;

using the identity to determine a description of the interface surface; and

,

identifying the drawing input field from the description and at least the movement data.

44. A system of enabling user interaction with computer software running in a computer system via:

an interface surface containing information relating to the computer software and having coded data disposed therein or thereon, the coded data being at least partially indicative of an identity of the interface surface; and

a sensing device which, when placed in an operative position relative to the interface surface, senses at least some of the coded data and uses the sensed coded data to generates indicating data at least partially indicative of the identity of the interface surface, and generates movement data indicative of the sensing device'"'"'s movement relative to the interface surface;

the computer system being configured to;

(a) receive the indicating data from the sensing device;

(b) receive the movement data from the sensing device, at least part of the said movement data representing written gesture input by the user;

(c) perform written gesture recognition in relation to at least some of the movement data; and

(d) in the event that a written gesture is recognised, operate the computer software in accordance with instructions associated with the written gesture and the interface surface.

45. A system according to claim 44, wherein the written gesture includes circumscribing or underlining at least some of the information.

57. A system according to claim 44, wherein the coded data includes at least one tag, each tag being indicative of the interface surface.

58. A system according to claim 57, wherein the tags are also indicative of points on the interface surface.

59. A system according to claim 58, wherein each of the tags includes:

first identity data defining a relative position of that tag; and

second identity data identifying the drawing field.

63. A method to enable a user to interact with computer software running in a computer system utilizing a form printed onto a surface, the form including information relating to the computer software and a drawing input field, and having disposed therein or thereon coded data at least partially indicative of the drawing input field, the method including the steps of:

providing the form to the user;

receiving, in the computer system, indicating data and movement data from a sensing device, the indicating data at least partially indicative of the drawing input field, the movement data indicative of movement of the sensing device relative to the form, device at least part of said movement data representing drawing input by the user, wherein the sensing device, when placed operatively relative to the drawing field, senses at least some of the coded data and uses the sensed coded data to generate the indicating data and substantially simultaneously generate the movement data; and

identifying, in the computer system and at least partially from the indicating data, the drawing input field, and

operating the computer software at least partially in reliance on the drawing input and on instructions associated with the drawing input field.

64. The method of claim 63, which includes printing the form in response to receiving, in the computer system, a request for the form.

65. The method of claim 63, which includes causing the form and the coded data to be printed onto the surface substantially simultaneously.

66. The method of claim 63 further including, in the computer system, associating the movement data with the drawing field.

67. The method of claim 63, including the step of performing written gesture recognition on at least some of the movement data.

68. The method of claim 67 wherein, in the event that the written gesture is recognized, operating the computer system in accordance with instructions associated with the written gesture.

69. The method of claim 67 wherein the selection gesture includes circumscribing or underlining at least some of the information.

70. The method of claim 63 further including the step of, in the computer system, recognizing whether the movement data is indicative of a written gesture drawn on the surface by the user.

71. The method of claim 70 wherein the written gesture includes a selection gesture.

72. The method of claim 63, including the step of identifying the user.

73. The method of claim 72, wherein the step of identifying the user includes using the movement data.

74. The method of claim 72, further including the step of receiving, in the computer system, data indicative of an identity of the user.

75. The method of claim 63, further including the step of receiving, in the computer system, data from a memory of the sensing device, the data being indicative of an identity of the user.

76. The method of claim 63, including the step of sending, in the computer system, data to the computer software indicative of at least the drawing field.

77. The method of claim 63, wherein the drawing field is associated with a visible drawing zone defined on the surface.

78. The method of claim 63, wherein the sensing device includes at least one acceleration measuring device for measuring acceleration of the sensing device as it is used on the surface, the movement data being generated by periodically sampling the acceleration of the sensing device as it is used on the surface.

79. The method of claim 78, wherein the acceleration measuring device includes one or more accelerometers configured to measure at least two orthogonal components of acceleration.

80. The method of claim 63, further including the step of generating movement data in the form of a locus of the sensing device in relation to the surface, the locus being determined by ascertaining relative displacement of the sensing device.

81. The method of claim 80, wherein the relative displacement is obtained by doubly integrating the acceleration with respect to time.

82. The method of claim 63, wherein elements are disposed on the surface, the sensing device being configured to periodically sense the elements as it is used on the surface, the method including the step of generating the movement data by ascertaining relative displacement of the sensing means over time with respect to at least one of the elements.

83. The method of claim 82, wherein the elements are disposed on the surface as a regular array of dots, lines or other formations.

84. The method of claim 82, wherein the elements are disposed on the surface stochastically.

85. The method according to claim 82, wherein the coded data includes the elements.

86. The method of claim 63, wherein the movement data is generated by ascertaining relative rotation of one or more motion sensing elements rotatably mounted to the sensing device for contact with the surface while the sensing device is used thereon.

87. The method of claim 86 wherein the motion sensing elements include one or more rollerballs mounted for rotation within a constraining housing disposed substantially within the sensing device.

88. The method of claim 87, wherein components of rotation of the rollerball, due to movement of the sensing device on the surface, are periodically measured.

89. The method of claim 88, wherein the components of rotation of the rollerball due to movement of the sensing device on the surface are measured by means of:

rollers disposed within the constraining housing for roation, the rollers being configured to be driven by contact with the rotating rollerball;

or

optical sensing of rotation of the rollerball with respect to the constraining housing.

90. The method of claim 63, wherein the coded data is indicative of a plurality of positions, in or associated with the drawing field.

91. The method of claim 63, wherein the coded data includes a plurality of tags, each of which includes tag data.

92. The method of claim 91, including deriving the relative position of at least one tag using at least some of the at least one tag'"'"'s tag data.

93. The method of claim 92, wherein the relative position is defined in relation to one of the group comprising:

a plurality of other tags, the surface, the drawing field and a zone associated with the drawing field.

94. The method of claim 91, including deriving an identity of the drawing field using at least some of the tag data of at least one tag.

95. The method of claim 94, wherein the data from which the identity of the drawing field is derived is the same in all tags of the same drawing field.

96. The method of claim 63, the sensing device generating at least the movement data in accordance with movement of the sensing device with respect to the coded data.

97. The method of claim 63, the method including printing the surface with a printer.

98. The method of claim 63, the indicating data representing at least an identity of the surface, the method including identifying the drawing input field in the computer system by:

determining the identity of the interface surface;

using the identity to determine a description of the surface; and

,

identifying the drawing input field from the description and at least some of the movement data.

99. A system for enabling a user to interact with computer software running in a computer system utilizing a form printed onto a surface, the form including information relating to the computer software and a drawing input field, and having disposed therein or thereon coded data at least partially indicative of the drawing input field, the system including:

a computer system which;

receives indicating data and movement data from a sensing device, the indicating data at least partially indicative of a drawing input field relating to the computer software and the movement data indicative of movement of the sensing device relative to the form, device at least part of said movement data representing drawing input by the user, wherein the sensing device, when placed operatively relative to the drawing field, senses at least some of the coded data and uses the sensed coded data to generate the indicating data and substantially simultaneously generate the movement data;

identifies, at least partially from the indicating data, the drawing input field, and

operates the computer software at least partially based on reliance on the drawing input and instructions associated with the drawing input field.

100. The system of claim 99 including a printer for printing the form in response to receiving, in the computer system, a request for the form.

101. The system of claim 99 including a printer for printing the form and in which the printer prints the coded data at substantially the same time as the form.

102. The system of claim 99 wherein the computer system is configured to associate the movement data with the drawing field.

103. The system of claim 99, wherein the computer system is configured to perform written gesture recognition on at least some of the movement data.

104. The system of claim 103 wherein the computer system is configured, in the event that the written gesture is recognized, to operate the computer software in accordance with instructions associated with the written gesture.

105. The system of claim 99 wherein the computer system is configured to recognize whether the movement data is indicative of a written gesture drawn on the surface by the user.

106. The system of claim 105 wherein the written gesture includes a selection gesture.

107. The system of claim 106 wherein the selection gesture includes circumscribing or underlining at least some of the information.

108. The system of claim 99, wherein the computer system is configured to identify the user.

109. The system of claim 108, wherein the computer system is configured to identify the user by using the movement data.

110. The system of claim 108, wherein the computer system is configured to receive data indicative of an identity of the user.

111. The system of claim 99, wherein the computer system is configured to receive identity data from a memory of the sensing device, the identity data being indicative of an identity of the user.

112. The system of claim 99, wherein the computer system is configured to send data to the computer software indicative of at least the drawing field.

113. The system of claim 99, wherein the drawing field is associated with a visible drawing zone defined on the surface.

114. The system of claim 99 which includes the sensing device.

115. The system of claim 114, wherein the sensing device includes at least one acceleration measuring device for measuring acceleration of the sensing device as it is used on the surface, the movement data being generated by periodically sampling the acceleration of the sensing device as it is used on the surface.

116. The system of claim 115, wherein the acceleration measuring device includes one or more accelerometers configured to measure at least two orthogonal components of acceleration.

117. The system of claim 114, wherein the sensing device generates movement data in the form of a locus of the sensing means in relation to the surface, the locus being determined by ascertaining relative displacement of the sensing device.

118. The system of claim 117, wherein the relative displacement is obtained by doubly integrating acceleration with respect to time.

119. The system of claim 114, wherein elements are disposed on the surface, the sensing device being configured to periodically sense elements as it is used on the surface, the movement data being generated by ascertaining relative displacement of the sensing means over time with respect to at least one of the elements.

120. The system of claim 119, wherein the elements are disposed on the surface as a regular array of dots, lines or other formations.

121. The system of claim 119, wherein the elements are disposed on the surface stochastically.

122. The system of claim 119 wherein the coded data includes the elements.

123. The system of claim 114, wherein the movement data is generated by ascertaining relative movement of one or more motion sensing elements rotatably mounted to the sensing device for contact with the surface while the sensing device is used thereon.

124. The system of claim 123, wherein the motion sensing elements include one or more rollerballs mounted for rotation within a constraining housing disposed substantially within the sensing device.

125. The system of claim 124, wherein components of rotation of the rollerball, due to movement of the sensing device when used on the surface, are periodically measured.

126. The system of claim 125, wherein the components of rotation of the rollerball due to movement of the sensing device by the user when used on the surface are measured by means of:

rollers disposed within the constraining housing for rotation, the rollers being configured to be driven by contact with the rotating rollerball;

or

optical sensing of rotation of the rollerball with respect to the constraining housing.

127. The system of claim 114 wherein the sensing device includes a marking nib.

128. The system of claim 114 wherein the sensing device contains identity information which imparts a unique identity to the sensing device.

129. The system of claim 128, wherein the identity data includes the identity of the sensing device.

130. The system of claim 99, wherein the coded data is indicative of a plurality of positions in or associated with the drawing field.

131. The system of claim 130, where the computer system is configured to derive an identity of the drawing field using at least some of the tag data of at least one tag.

132. The system of claim 131, wherein the data from which the identity of the drawing field is derived is the same in all the tags of the same drawing field.

133. The system of claim 99, wherein the coded data includes a plurality of tags, each of which includes tag data.

134. The system of claim 133, wherein the computer system is configured to derive the relative position of at least one tag using at least some of the tag data of the at least one tag.

135. The system of claim 134, wherein the relative position is defined in relation to one of the following group:

a plurality of other tags, the surface, the drawing field, and a zone associated with the drawing field.

136. The system of claim 99, the sensing device generating at least the movement data in accordance with movement of the sensing device with respect to the coded data.

137. The system of claim 99, the system including a printer for printing the surface.

138. The system of claim 99, the indicating data representing at least an identity of the surface, the method including identifying the drawing input field in the computer system by:

determining the identity of the surface;

using the identity to determine a description of the surface; and

,

identifying the drawing input field from the description and at least the movement data.