Method and system for instruction of a computer using processing sensor

• US 7,034,953 B2
• Filed: 11/12/2002
• Issued: 04/25/2006
• Est. Priority Date: 09/17/1999
• Status: Expired due to Fees

• Current Assignee: Silverbrook Research Pty Limited
• Original Assignee: Silverbrook Research Pty Limited
• Inventors: Silverbrook, Kia, Lapstun, Paul
• Primary Examiner(s): HASKINS, TWYLER LAMB
• Application Number:
  US10/291,487
• Publication Number:
  US 20030151772A1
• Time in Patent Office: 1,260 Days
• Field of Search: 358/1.15, 358/1.18, 235/462.01, 235/487, 435/179
• US Class Current: 358/1.15
• CPC Class Codes: G06F 2203/0337 Status LEDs integrated in the mouse to provide visual feedback to the user about the status of the input device, the PC, or the user
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/171 by use of digital ink
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture

Abstract

The present invention provides a method for enabling instruction of a computer to perform tasks, the method including the steps of: providing a user with at least one form printable on a surface to provide one or more first viewable information zones relating to one or more available commands and one or more second viewable information zones relating to one or more objects; receiving, in a computer system, indicating data from a sensing device operated by the user regarding movement of the sensing device relative to the surface, said movement including a stroke of part of the sensing device on or relative to said surface; determining, in the computer system and from the indicating data, if the stroke substantially links one or more of said first viewable information zones with one or more of said second viewable information zones, and thereby interpreting the stroke as designating (i) a respective one or more of said available commands and (ii) a respective one or more of said objects; and applying in the computer system the designated one or more of said available commands to the designated one or more of said objects.

Claims

1. A method for enabling instruction of a computer to perform tasks, the method including the steps of:

providing a user with at least one form printable on a surface to provide one or more first viewable information zones relating to one or more available commands and one or more second viewable information zones relating to one or more objects, the form including coded data indicative of an identity of the form and of at least one reference point of the form;

receiving, in a computer system, indicating data from a sensing device operated by the user regarding at least one of a movement and a position of the sensing device relative to the surface, at least one of the movement and the position including a stroke of part of the sensing device on or relative to said surface;

determining, in the computer system and from the indicating data, if the stroke substantially links one or more of said first viewable information zones with one or more of said second viewable information zones, and thereby interpreting the stroke as designating (i) a respective one or more of said available commands and (ii) a respective one or more of said objects; and

applying in the computer system the designated one or more of said available commands to the designated one or more of said objects,

wherein the sensing device comprises;

(a) an image sensor adapted to capture images of at least some of the coded data when the sensing device is placed in an operative position relative to the surface; and

(b) a processor adapted to;

(i) identify at least some of the coded data from one or more of the captured images;

(ii) determine an orientation, within the captured images, of at least some of the coded data;

(iii) decode at least some of the coded data; and

(iv) generate the indicating data using at least some of the decoded coded data.

2. The method of claim 1, wherein the processor is housed within the sensing device.

3. The method of claim 1, wherein the processor is adapted to operate at processing speeds ranging from 1 instruction per second to 1×

10¹² instructions per second.

4. The method of claim 3 wherein the processor is adapted to operate at processing speeds ranging from 10 instructions per second to 1×

10⁶ instructions per second.

5. The method of claim 1, wherein the image sensor is housed within the sensing device.

6. The method of claim 1, wherein the image sensor is adapted to capture images at rates ranging from 1 image per second to 1,000 images per second.

7. The method of claim 6 wherein the image sensor is adapted to capture images at rates ranging from 10 images per second to 200 images per second.

8. The method of claim 1, wherein the coded data comprises non-barcode coded data.

9. The method of claim 1, wherein said indicating data regarding at least one of the movement and the position of the sensing device relative to the surface is generated by the sensing device sensing at least one of the movement and the position of the sensing device relative to the surface using at least some of the coded data.

10. The method of claim 1, including the step of identifying, in the computer system and from said indicating data, if the stroke has encircled one or more of said second viewable zones relating to one or more objects, thereby designating said one or more objects.

11. The method of claim 1, including the step of identifying, in the computer system and from said indicating data, if the stroke has intersected one or more of said second viewable zones relating to one or more objects, thereby designating said one or more objects.

12. The method of claim 11, at least one of said second viewable zones having a viewable boundary, and including the step of identifying, in the computer system and from said indicating data, if the stroke has crossed said boundary more than once, and to apply a different one or more of said available commands if such an occurrence is identified.

13. The method of claim 1, including the step of identifying, in the computer system and from the indicating data, if a user has effected the stroke to designate one or more of said second viewable zones and extend to intersect with one or more of said first viewable zones.

14. The method of claim 1 for designation of a feature of said one or more objects such that said designated one or more of the available commands is carried out with respect to that feature.

15. The method of claim 14, wherein said designated feature is a color attribute of said designated one or more objects, and the method includes the step of setting the value of said color attribute according to the designated one or more commands.

16. The method of claim 15, including the step of providing the user with a further form printable on a surface including one or more viewable information zones relating to said designated one or more of said objects, said one or more zones including representations of the designated one or more objects rendered according to the value of the color attribute.

17. A system for enabling instruction of a computer to perform tasks, the system including:

at least one form printable on a surface to provide one or more first viewable information zones relating to one or more available commands and one or more second viewable information zones relating to one or more objects, the form including coded data indicative of an identity of the form and of at least one reference point of the form;

a computer system for receiving indicating data from a sensing device operated by a user regarding at least one of a movement and a position of the sensing device relative to the surface, at least one of the movement and the position including a stroke of part of the sensing device on or relative to said surface,

the computer system adapted to determine, from the indicating data received, if the sensing device stroke substantially links one or more of said first viewable information zones with one or more of said second viewable information zones and thereby designates (i) a respective one or more of said available commands and (ii) a respective one or more of said objects one or more of said available commands,

the computer system adapted to apply the designated one or more of said available commands to the designated one or more of said objects,

wherein the sensing device comprises;

(a) an image sensor adapted to capture images of at least some of the coded data when the sensing device is placed in an operative position relative to the surface; and

(b) a processor adapted to;

(i) identify at least some of the coded data from one or more of the captured images;

(ii) determine an orientation, within the captured images, of at least some of the coded data;

(iii) decode at least some of the coded data; and

(iv) generate the indicating data using at least some of the decoded coded data.

18. The system of claim 17, wherein the processor is housed within the sensing device.

19. The system of claim 17, wherein the processor is adapted to operate at processing speeds ranging from 1 instruction per second to 1×

10¹² instructions per second.

20. The system of claim 19 wherein the processor is adapted to operate at processing speeds ranging from 10 instructions per second to 1×

10⁶ instructions per second.

21. The system of claim 17, wherein the image sensor is housed within the sensing device.

22. The system of claim 17, wherein the image sensor is adapted to capture images at rates ranging from 1 image per second to 1,000 images per second.

23. The system of claim 22 wherein the image sensor is adapted to capture images at rates ranging from 10 images per second to 200 images per second.

24. The system of claim 17, wherein the coded data comprises non-barcode coded data.

25. The system of claim 17, wherein said indicating data regarding at least one of the movement and the position of the sensing device relative to the surface is generated by the sensing device sensing at least one of the movement and the position of the sensing device relative to the surface using at least some of the coded data.

26. The system of claim 17, the computer system adapted to identify from said indicating data if the stroke has encircled one or more of said second viewable zones relating to one or more objects, thereby designating said one or more objects.

27. The system of claim 17, the computer system adapted to identify from said indicating data if the stroke has intersected one or more of said second viewable zones relating to one or more objects, thereby designating said one or more objects.

28. The system of claim 27, at least one of said second viewable zones having a viewable boundary, the computer system adapted to identify from said indicating data if the stroke has crossed said boundary more than once, and to apply a different one or more of said available commands if such an occurrence is identified.

29. The system of claim 17 including the sensing device.