Method and system for collaborative document markup using processing sensor

• US 7,044,363 B2
• Filed: 11/12/2002
• Issued: 05/16/2006
• Est. Priority Date: 06/30/1999
• Status: Expired due to Fees

• Current Assignee: Silverbrook Pty, Ltd.
• Original Assignee: Silverbrook Research Pty Limited
• Inventors: Silverbrook, Kia, Lapstun, Paul, Lapstun, Jacqueline Anne
• Primary Examiner(s): Lee, Michael G.
• Assistant Examiner(s): TAYLOR, APRIL ALICIA
• Application Number:
  US10/291,820
• Publication Number:
  US 20030106024A1
• Time in Patent Office: 1,281 Days
• Field of Search: 235/375, 235/454, 235/472.03, 235/487
• US Class Current: 235/375
• CPC Class Codes: B41J 13/103 for the sheet feeding section
• CPC Class Codes: B41J 13/106 for the sheet output section
• CPC Class Codes: B41J 2/17503 Ink cartridges
• CPC Class Codes: B41J 2/17513 Inner structure
• CPC Class Codes: B41J 2/17523 Ink connection
• CPC Class Codes: B41J 2/17546 electronically
• CPC Class Codes: B41J 2/17553 Outer structure
• CPC Class Codes: B41J 2/17556 Means for regulating the pressure in the cartridge
• CPC Class Codes: B41J 2/17563 Ink filters
• CPC Class Codes: B41J 2/2114 Ejecting transparent or white coloured liquids, e.g. processing liquids B41J2/211 takes precedence
• CPC Class Codes: B42C 19/02 starting with single sheets
• CPC Class Codes: B42C 9/0006 by applying adhesive to a stack of sheets
• CPC Class Codes: B42C 9/0081 applying adhesive to individual sheets for binding them together by fusing, welding B42C9/0093
• CPC Class Codes: B42P 2261/04 Producing books by printing sheets in following order, e.g. for ordering via the Internet
• CPC Class Codes: B65H 29/34 from supports slid from under the articles
• CPC Class Codes: B65H 37/04 for securing together articles or webs, e.g. by adhesive, stitching or stapling adhering replacement to expiring web during change of web roll B65H19/18
• CPC Class Codes: G06F 2203/0337 Status LEDs integrated in the mouse to provide visual feedback to the user about the status of the input device, the PC, or the user
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1204 resulting in reduced user or operator actions, e.g. presetting, automatic actions, using hardware token storing data
• CPC Class Codes: G06F 3/1243 : Variable data printing, e.g. document forms, templates, labels, coupons, advertisements, logos, watermarks, transactional printing, fixed content versioning
• CPC Class Codes: G06F 3/1284 : Local printer device
• CPC Class Codes: G06F 40/117 : Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/171 : by use of digital ink
• CPC Class Codes: G06K 17/00 : Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
• CPC Class Codes: G06V 30/1423 : the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: H04L 12/1827 : Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04N 1/00127 : Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/00358 : Type of the scanned marks
• CPC Class Codes: H04N 1/00376 : Means for identifying a mark sheet or area
• CPC Class Codes: H04N 1/00567 : Handling of original or reproduction media, e.g. cutting, separating, stacking
• CPC Class Codes: H04N 1/00968 : by scanning marks on a sheet mark-sheet input means H04N1/00355; sensing record carriers G06K7/00
• CPC Class Codes: H04N 1/32101 : Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 1/32122 : in a separate device, e.g. in a memory or on a display separate from image data
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/32771 : in response to a request, e.g. for a particular document
• CPC Class Codes: H04N 1/32778 : using a mark-sheet or machine-readable code request
• CPC Class Codes: H04N 2201/0082 : Image hardcopy reproducer H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3205 : of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 2201/3207 : of an address
• CPC Class Codes: H04N 2201/3226 : of identification information or the like, e.g. ID code, index, title, part of an image, reduced-size image
• CPC Class Codes: H04N 2201/3242 : of processing required or performed, e.g. for reproduction or before recording H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3243 : of type information, e.g. handwritten or text document
• CPC Class Codes: H04N 2201/3247 : Data linking a set of images to one another, e.g. sequence, burst or continuous capture mode
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3269 : of machine readable codes or marks, e.g. bar codes or glyphs
• CPC Class Codes: H04N 2201/327 : which are undetectable to the naked eye, e.g. embedded codes
• CPC Class Codes: H04N 2201/3271 : Printing or stamping
• CPC Class Codes: H04N 2201/3278 : Transmission

Abstract

A method and system is described for enabling collaborative document markup, through the use of one or more forms which support interaction with a computer system or network. Each form is printed on sheet material such as paper and includes coded data which allows it to be used to interact with the computer system via a sensing device operated by a user. The method and system described can be employed by geographically distributed participants to mark up a shared document using interactive surface media, while engaged in a telephone conference or similar. Every time a participant adds a markup to the document the other participants can be provided with a printed copy of the updated page.

Claims

1. A method of enabling collaborative document markup between participants, the method including the steps of:

providing a participant with at least one document markup form containing information relating to a collaborative document markup session and including coded data indicative of an identity of the document markup form and of at least one reference point of the document markup form;

receiving, in a computer system, indicating data from a sensing device operated by said participant involved in the document markup session, said indicating data regarding the identity of the document markup form and at least one of a position and a movement of the sensing device relative to the document markup form;

identifying in the computer system and from the indicating data, at least one parameter relating to the document markup session; and

providing at least one other participant remote from said participant with at least one document markup form containing information representative of the at least one parameter,

wherein the sensing device comprises;

(a) an image sensor for capturing images of at least some of the coded data when the sensing device is placed in an operative position relative to the form; and

(b) a processor for;

(i) identifying at least some of the coded data from one or more of the captured images;

(ii) determining an orientation, within the captured images, of at least some of the coded data;

(iii) decoding at least some of the coded data; and

(iv) generating the indicating data using at least some of the decoded coded data.

2. The method of claim 1 in which said at least one parameter relating to the document markup session is associated with at least one zone of the document markup form, and in which the method includes identifying, in the computer system and from the zone relative to which the sensing device is positioned or moved, said at least one parameter.

4. The method of claim 1 or claim 3, in which the parameter is an action parameter of the collaborative document markup session, the method including effecting, in the computer system, an operation in respect of the action parameter.

5. The method of claim 4, in which the action parameter of the collaborative document markup session is selected from the group comprising:

sending an invitation to a document markup session;

starting a document markup session;

ending a document markup session;

sending a message to a participant during a document markup session;

viewing messages sent during a document markup session;

viewing the page history of a document markup session;

ordering the printing of a document markup page;

updating a document markup page; and

ordering the reprinting of a new document markup page.

6. The method of claim 1 or claim 3, wherein the processor is housed within the sensing device.

7. The method of claim 1 or claim 3, wherein the processor operates at processing speeds ranging from 1 instruction per second to 1×

10¹² instructions per second.

8. The method of claim 7 wherein the processor operates at processing speeds ranging from 10 instructions per second to 1×

10⁶ instructions per second.

9. The method of claim 1 or claim 3, wherein the image sensor is housed within the sensing device.

10. The method of claim 1 or claim 3, wherein the image sensor captures images at rates ranging from 1 image per second to 1,000 images per second.

11. The method of claim 10 wherein the image sensor captures images at rates ranging from 10 images per second to 200 images per second.

12. The method of claim 1 or claim 3, wherein the coded data comprises non-barcode coded data.

13. The method of claim 1 or claim 3 in which the parameter is an option parameter of the collaborative document markup session, the method including identifying, in the computer system, that the participant has entered a hand-drawn mark by means of a sensing device and effecting, in the computer system, an operation associated with the option parameter.

14. The method of claim 13 in which the option parameter is associated with at least one of:

an access level;

a message recipient; and

a line color.

15. The method of claim 1 or claim 3 in which the parameter is a text parameter of the collaborative document markup session, the method including identifying, in the computer system, that the participant has entered handwritten text data by means of the sensing device and effecting, in the computer system, an operation associated with the text parameter.

16. The method of claim 15 which includes converting, in the computer system, the handwritten text data to computer text.

17. The method of claim 16 which includes converting, in the computer system, the handwritten text data to computer text using an online handwriting recognition program.

18. The method of claim 15 in which the text parameter is associated with at least one of:

a document markup session description;

a document markup session purpose;

a document markup session start date;

a start time;

an end time;

an attachment document identification;

a participant identification;

a message subject description; and

a message text.

19. The method of claim 1 or claim 3 in which the parameter is an authorization parameter of the collaborative document markup session, the method including identifying, in the computer system, that the participant has entered a handwritten signature by means of the sensing device and effecting, in the computer system, an operation associated with the authorization parameter.

20. The method of claim 19 which includes verifying, in the computer system, that the signature is that of the participant.

21. The method of claim 20 in which the authorization parameter is associated with updating a document markup page.

22. The method of claim 1 or claim 3 in which the parameter is a graphical parameter of the collaborative document markup session, the method including identifying, in the computer system, that the participant has entered hand-drawn graphical information by means of the sensing device and effecting, in the computer system, an operation associated with the graphical parameter.

23. The method of claim 22 in which the graphical parameter is associated with a markup to a document markup page.

24. The method of claim 1 or claim 3 in which at least one of the one or more document markup form contains information relating to at least one of:

booking a document markup session;

controlling a document markup session;

selecting invitees to a document markup session;

inviting participants to a document markup session;

composing messages in a document markup session;

sending messages in a document markup session;

viewing messages in a document markup session;

selecting a line color in a document markup session;

viewing the history of a document markup session;

printing document markup pages;

updating document markup pages;

reprinting document markup pages; and

ending a document markup session.

25. The method of claim 1 or claim 3 in which the document markup session participants are provided with a telephonic communicator for simultaneous use during the collaborative document markup session.

26. The method of claim 25, in which the telephonic communicator includes a video communication link between the participants.

27. The method of claim 25, the telephonic communicator being controlled by said computer system.

3. A method of enabling document markup between participants, the method including the steps of:

providing a participant with at least one document markup form containing information relating to a collaborative document markup session and including coded data indicative of at least one parameter of the document markup session;

receiving, in a computer system, data from a sensing device operated by said participant regarding said at least one parameter and regarding at least one of a movement and a position of the sensing device relative to the document markup form;

interpreting, in the computer system, at least one of said movement and position of the sensing device as the sensing device relates to said at least one parameter; and

providing at least one other participant remote from said participant with at least one document markup form containing information representative of the at least one parameter,

wherein the sensing device comprises;

(a) an image sensor for capturing images of at least some of the coded data when the sensing device is placed in an operative position relative to the form; and

(b) a processor for;

(i) identifying at least some of the coded data from one or more of the captured images;

(ii) determining an orientation, within the captured images, of at least some of the coded data;

(iii) decoding at least some of the coded data;

(iv) generating the data regarding said at least one parameter using at least some of the decoded coded data; and

(v) generating the data regarding at least one of the movement and position of the sensing device relative to the document markup form using at least some of the decoded coded data.

28. The method according to claim 3, wherein any 10 millimeter diameter area of the coded data indicative of the at least one parameter of the document markup session includes sufficient information to identify the at least one parameter.

29. A system for enabling collaborative document markup between participants, the system including:

at least one document markup form containing information relating to a collaborative document markup session and including coded data indicative of an identity of the document markup form and of at least one reference point of the document markup form;

a computer system for receiving indicating data from a sensing device operated by a participant involved in the document markup session said indicating data regarding the identity of the document markup form and at least one of a position and a movement of the sensing device relative to the document markup form, said computer system including means for identifying, from the indicating data, at least one parameter relating to the document markup session; and

a provider for providing at least one other participant remote from said participant with at least one document markup form containing information representative of the at least one parameter,

wherein the sensing device comprises;

(a) an image sensor for capturing images of at least some of the coded data when the sensing device is placed in an operative position relative to the form; and

(b) a processor for;

(i) identifying at least some of the coded data from one or more of the captured images;

(ii) determining an orientation, within the captured images, of at least some of the coded data;

(iii) decoding at least some of the coded data; and

(iv) generating the indicating data using at least some of the decoded coded data.

30. The system of claim 29 in which said at least one parameter relating to the document markup session is associated with at least one zone of the document markup form.

31. The system of claim 29 including the sensing device, the sensing device sensing at least one of the movement and position of the sensing device relative to the document markup form, using at least some of the coded data.

34. The system of claim 29 or claim 32, wherein the processor is housed within the sensing device.

35. The system of claim 29 or claim 32, wherein the processor operates at processing speeds ranging from 1 instruction per second to 1×

10¹² instructions per second.

36. The system of claim 35 wherein the processor operates at processing speeds ranging from 10 instructions per second to 1×

10⁶ instructions per second.

37. The system of claim 29 or claim 32, wherein the image sensor is housed within the sensing device.

38. The system of claim 29 or claim 32, wherein the image sensor captures images at rates ranging from 1 image per second to 1,000 images per second.

39. The system of claim 38 wherein the image sensor captures images at rates ranging from 10 images per second to 200 images per second.

40. The system of claim 29 or claim 32, wherein the coded data comprises non-barcode coded data.

41. The system of claim 29 or claim 32 including the sensing device.

42. The system of claim 29 or claim 32 in which said at least one parameter of the document markup session is selected from the group comprising an action parameter of the document markup session, an option parameter of the document markup session, a text parameter of the document markup session, an authorization parameter of the document markup session, and a graphical parameter of the document markup session.

43. The system of claim 42 in which the action parameter of the document markup session is selected from the group comprising:

sending an invitation to a document markup session;

starting a document markup session;

ending a document markup session;

sending a message to a participant during a document markup session;

viewing messages sent during a document markup session;

viewing the page history of a document markup session;

ordering the printing of a document markup page;

updating a document markup page; and

ordering the reprinting of a document markup page.

44. The system of claim 42 in which the option parameter is associated with at least one of:

an access level;

a message recipient; and

a line color.

45. The system of claim 42 in which the text parameter is associated with at least one of:

a document markup session description;

a document markup session purpose;

a document markup start date;

a start time;

an end time;

an attachment document identification;

a participant identification;

a document markup session purpose;

a message subject description; and

a message text.

46. The system of claim 42 in which the authorization parameter is associated with updating a document markup page.

47. The system of claim 42 in which the graphical parameter is associated with a markup to a document markup page.

48. The system of claim 29 or claim 32, including a telephone communicator for simultaneous use during the collaborative document markup session.

49. The system of claim 48, including a video communicator for use during the collaborative document markup session.

32. A system for enabling collaborative document markup between participants, the system including:

at least one document markup form containing information relating to a collaborative document markup session and including coded data indicative of at least one parameter of the document markup session;

a computer system for receiving data from a sensing device operated by the participant involved in the document markup session, said data regarding at least one of a movement and a position of the sensing device relative to the document markup form, and for interpreting at least one of said movement and said position of the sensing device as the sensing device relates to said at least one parameter; and

a provider for providing at least one other participant remote from the participant with at least one document markup form containing information representative of the at least one parameter,

wherein the sensing device comprises;

(a) an image sensor for capturing images of at least some of the coded data when the sensing device is placed in an operative position relative to the form; and

(b) a processor for;

(i) identifying at least some of the coded data from one or more of the captured images;

(ii) determining an orientation, within the captured images, of at least some of the coded data;

(iii) decoding at least some of the coded data;

(iv) generating the data regarding said at least one parameter using at least some of the decoded coded data; and

(v) generating the data regarding at least one of the movement and position of the sensing device relative to the document markup form using at least some of the decoded coded data.

33. The system of claim 32 in which said at least one parameter relating to the document markup session is associated with at least one zone of the document markup form.

50. The system according to claim 32, wherein any 10 millimeter diameter area of the coded data indicative of the at least one parameter of the document markup session includes sufficient information to identify the at least one parameter.