Document including computer graphical user interface element, method of preparing same, computer system and method including same

• US 20040032428A1
• Filed: 06/13/2003
• Published: 02/19/2004
• Est. Priority Date: 06/13/2002
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Pilu, Maurizio, Frohlich, David Mark, Pollard, Stephen Bernard
• Application Number:
  US10/460,675
• Publication Number:
  US 20040032428A1
• Time in Patent Office: Days
• US Class Current: 345/764
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]

Abstract

A printed document is an interface with a computer. A camera generates a video signal representing an image of the printed document. A processor linked to the camera processes an image of the printed document and a finger or pointing implement pointing to a region of the printed document. The processor recognizes when a graphical user interface element within the captured image is selected by the user pointing to the user interface element on the printed document by determining the region on the page that is pointed to by the finger or pointing implement and determining from a memory the identity of the user interface element, if any, that corresponds to the region of the printed page pointed to by the finger or pointing implement and then triggers an operation represented by that graphical user interface element.

Claims

1. A method of processing information associated with content on a document, the document also including at least one graphical user interface element for controlling a computer function, different graphical user interface elements being associated with controlling different computer functions, the method comprising:

converting an optical image of the document into a signal representing the graphical user interface element and other content on the document; and

processing at least some information associated with the other content of the document as included in the signal based on a selected graphical user interface element on the document, as included in the signal, the processing being performed by the computer in response to the signal.

2. The method of claim 1 wherein the document includes a plurality of the graphical user interface elements, the method further comprising:

(a) selecting one of the graphical user interface elements by pointing, the signal including an indication of the pointing, and (b) processing, with the computer, at least some of the other content in response to the pointed to graphical user interface element, as included in the signal.

3. The method of claim 2 wherein the pointing step is performed with a pointer, the signal including an indication of the location of the pointer, the processing being responsive to the indication of the pointer location so information associated with at least some of the other content of the document is processed based on the location of the pointer on the selected graphical user interface element and the selected graphical user interface element.

4. The method of claim 2 wherein the pointing step is performed by a finger of a user, so that the signal includes an indication of the location of the finger, the processing being responsive to the indication of the finger location so information associated with at least some of the other content of the document is processed based on the location of the finger on the selected graphical user interface element and the selected graphical user interface element.

5. The method of claim 1 wherein the at least one graphical user interface element is printed in an area outside margins associated with the other content of the document, and further including processing graphical user interface elements only in the portion of the signal associated with regions outside the margin.

6. The method of claim 5 wherein the document includes a plurality of distinct position indicating visual indicia that are included in the signal, determining the location of the area outside the location of the margin where the at least one graphical user interface is located by processing the portion of the signal indicative of the position indicating indicia.

7. The method of claim 1 wherein the at least one graphical user interface element is printed in an area inside margins including the other content of the document, and causing the computer to process information associated with a portion of the other content adjacent the graphical user interface and element.

8. In combination,

a document including (a) at least one graphical user interface element for controlling a computer function and (b) other content, different graphical user interface elements being associated with controlling different computer functions,

an optical image converter for generating a signal in response to optical images on the document, the signal representing the graphical user interface element and the other content of the document;

and

a processor adapted to be responsive to the signal for processing information associated with at least some of the other content of the document based on a selected graphical user interface element of the document, as included in the signal.

9. The combination of claim 8 wherein the document includes a plurality of the graphical user interface elements, one of the graphical user interface elements being adapted to be selected by pointing, the optical image converter signal when derived, including an indication of the pointing, wherein the processor is arranged to be responsive to the portion of the optical image converter signal including the pointing for causing the processor to process information associated with at least some of the other content in response to the pointed to graphical user interface element, as included in the signal.

10. The combination of claim 9 wherein the apparatus includes a pointer for selecting one of the graphical user interface elements, the signal when derived, including an indication of the location of the pointer, the processor being arranged to be responsive to the indication of the pointer location to cause the processor to process information associated with at least some of the other content of the document based on the location of the pointer on the selected graphical user interface element and the selected graphical user interface element.

11. The combination of claim 9 wherein the selected graphical user interface element is adapted to be selected by a finger of a user, and wherein the optical image converter is arranged so that the signal generated thereby includes an indication of the location of the finger, the processor being arranged to be responsive to the indication of the finger location to cause the processor to process information associated with at least some of the other content of the document based on the location of the finger on the selected graphical user interface element and the selected graphical user interface element.

12. The combination of claim 8 wherein the at least one graphical user interface element is printed in an area outside margins associated with the other content of the document, and wherein the processor is arranged to respond to graphical user interface elements only in the portion of the signal associated with regions outside the margin.

13. The combination of claim 12 wherein the document includes a plurality of distinct position indicating visual indicia adapted to be included in the signal, the processor being arranged to be responsive to the portion of the signal indicative of the position indicating indicia for determining the location of the area outside the location of the margin where the at least one graphical user interface element is located.

14. The combination of claim 8 wherein the at least one graphical user interface element is printed in an area inside margins including the other content of the document, the processor being arranged to process information associated with a portion of the other content adjacent the graphical user interface element.

15. Apparatus for use with plural documents, each including at least one graphical user interface element for controlling a computer function and other content, different graphical user interface elements being associated with controlling different computer functions, the apparatus comprising:

an optical image converter for generating a signal in response to optical images on the document, the signal representing the graphical user interface element and the other content of the document; and

a processor adapted to be responsive to the signal for processing information associated with at least some of the other content of the document based on a selected graphical user interface element of the document, as included in the signal.

16. The apparatus of claim 15 wherein the document includes a plurality of the graphical user interface elements, one of the graphical user interface elements being adapted to be selected by pointing, the optical image converter signal including an indication of the pointing, wherein the processor is adapted to be responsive to the portion of the optical image converter signal including the pointing for causing the processor to process information associated with at least some of the other content in response to the pointed to graphical user interface element, as included in the signal.

17. The apparatus of claim 16 wherein the apparatus includes a pointer for selecting one of the graphical user interface elements, the signal when derived, including an indication of the location of the pointer, the processor being arranged to be responsive to the indication of the pointer location to cause the processor to process information associated with at least some of the other content of the document based on the location of the pointer on the selected graphical user interface element and the selected graphical user interface element.

18. The apparatus of claim 16 wherein the selected graphical user interface element is adapted to be selected by a finger of a user, and wherein the optical image converter is arranged so that the signal generated thereby includes an indication of the location of the finger, the processor being arranged to be responsive to the indication of the finger location to cause the processor to process information associated with at least some of the other content of the document based on the location of the finger on the selected graphical user element interface and the selected graphical user interface element.

19. The apparatus of claim 15 wherein the at least one graphical user interface is printed in an area outside margins associated with the other content of the document, and wherein the processor is arranged to respond to graphical user interface elements only in the portion of the signal associated with regions outside the margin.

20. The apparatus of claim 19 wherein the document includes a plurality of distinct position indicating visual indicia adapted to be included in the signal, the processor being arranged to be responsive to the portion of the signal indicative of the position indicating indicia for determining the location of the area outside the location of the margin where the at least one graphical user interface element is located.

21. The apparatus of claim 15 wherein the at least one graphical user interface element is printed in an area inside margins including the other content of the document, the processor being arranged to process information associated with a portion of the other content adjacent the graphical user interface element.

22. A method of preparing a document including visual content to be processed by a computer and at least one graphical user interface element for controlling processing by the computer of information associated with the visual content, the method comprising the steps of

applying the visual content to the document, and

applying at least one graphical user interface element to the document, the at least one graphical user interface element being selected from a plurality of graphical user interface elements, each associated with a different function which can be performed by the computer on information associated with the visual content included in the document.

23. The method of claim 22 wherein the at least one graphical user interface element is applied to a portion of the document beyond margins of the visual content.

24. The process of claim 22 wherein the at least one graphical user interface element is applied to a portion of the document within margins of the visual content.

25. The method of claim 22 wherein a plurality of different graphical user interface elements are applied to the document.

26. The method of claim 22 further including applying subregions to the at least one graphical user interface element, the applied subregions being arranged for manual selection by application of a marking instrument.

27. The method of claim 22 further including applying distinct visual position indicating indicia to predetermined locations of the document, the location indicia being different from the visual content and the at least one graphical user interface element.

28. A document for use with a computer, the document comprising

at least one graphical user interface element, and

a visual content portion;

the graphical user interface element being visually distinct from the visual content portion and being such as to provide information to the computer of how the computer is to process information associated with the visual content on the document.

29. The document of claim 28 wherein the document includes a plurality of the graphical user interface elements that are different from each other and associated with different processing functions by the computer of the information associated with the visual content included in the document.

30. The document of claim 29 wherein the at least one graphical user interface element is beyond margins on the document for the visual content.

31. The document of claim 30 wherein the document includes location indicia at predetermined locations of the document, the location indicia being different from the visual content and the at least one graphical user interface element.

32. The document of claim 29 wherein the at least one graphical user interface element is within margins on the document for the visual content.