Method and apparatus for assisting the reading of a document

• US 20030081014A1
• Filed: 10/30/2002
• Published: 05/01/2003
• Est. Priority Date: 10/31/2001
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Frohlich, David Mark
• Application Number:
  US10/283,135
• Publication Number:
  US 20030081014A1
• Time in Patent Office: Days
• US Class Current: 345/856
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G08C 21/00 Systems for transmitting the position of an object with respect to a predetermined reference system, e.g. tele-autographic system
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Computer apparatus for assisting the reading of a document by displaying upon it a chronologic sequence of pointing gestures, comprises a camera and data processor for imaging the document and for tracking the movement of a pointer, such as a person'"'"'s finger, over the document to associate that movement with the imaged document as the said sequence, and a computer with a projector for displaying the sequence in association with a reproduction of the same document, which may be printed paper or an on-screen image, normally at a remote location for conferencing.

A photograph on a two dimensional medium comprises a computer-readable representation of such a chronologic sequence of pointing gestures associated with the photograph.

Claims

1. Apparatus for assisting the reading of a document by displaying upon it a chronologic sequence of pointing gestures, comprising means for imaging said document to form an imaged document and for tracking the movement of a pointer over said document to associate that movement with said imaged document as said sequence, and means for displaying said sequence in association with a reproduction of the same document.

2. Apparatus according to claim 1, in which the means for imaging the document comprises a camera focussed on a real two dimensional medium on which the document is printed or scribed, and a processor linked to the camera for processing an image captured by the camera, and for orientating the image using predetermined registration criteria.

3. Apparatus according to claim 2, in which the image processing means is arranged to identify the position of the pointer in the captured image and thereby to track it.

4. Apparatus according to claim 1, comprising a synchronous communications system and source and destination processors, linked by said communications system, the source processor being arranged to read the document and track the sequence of pointing gestures, and the destination processor being arranged to display the said sequence over the reproduction of the same document.

5. Apparatus according to claim 4, in which the destination processor has means for receiving an electronic version of the document and for printing it as the said reproduction.

6. Apparatus according to claim 4, in which the destination processor has means for receiving an electronic version of the document and for displaying it visually as the said reproduction.

7. Apparatus according to claim 1, comprising a memory for storing the image of the document and the said associated sequence, and a processor for reproducing the stored sequence on the reproduction of the document.

8. Apparatus according to claim 1, comprising means for transmitting the image of the document and the associated sequence to a remote memory, such as an internet website, for use by a processor for reproducing the stored sequence on the reproduction of the document.

20. Apparatus according to claim 1, for assisting the reading of a multi-page document, some or all of which pages may have a respective said sequence of gestures associated with them, comprising means responsive to the presence or absence of such sequence to provide an indication whether there is such a sequence.

21. Apparatus according to claim 1 for assisting the reading of a multi-page document, some or all of which pages may have a respective said sequence of gestures associated with them, in which the said sequence or sequences comprise an indication of when to turn to the next page.

22. Apparatus according to claim 20, in which the said sequence comprises a trigger for turning automatically to the next page.

23. A computer program arranged to control a processor associated with the apparatus of claim 1, the program controlling the imaging of the document and the tracking of the movement of the pointer, and the association of that movement with the imaged document as the said sequence.

24. A computer program arranged to control a processor in apparatus according to claim 1, the program controlling the displaying of the sequence in association with a reproduction of the said document.

9. A method of assisting the reading of a document by displaying upon it a chronologic sequence of pointing gestures, comprising imaging the document and tracking the movement of a pointer over the document to associate that movement with the imaged document as the said sequence, and displaying the sequence in association with a reproduction of the same document.

10. A method according to claim 9, for synchronous electronic conferencing over a communications system, comprising transmitting the said sequence of pointing gestures over the communications system.

11. A method according to claim 10, further comprising transmitting an image of the document over the communications system.

12. A method according to claim 11, further comprising printing a reproduction of the document from the transmitted image of that document, and then projecting the said chronologic sequence of pointing gestures onto the printed reproduction.

13. A document on a two dimensional medium comprising a computer-readable representation of a chronologic sequence of pointing gestures associated with the document.

14. A document according to claim 13, further comprising an audio memory on the two-dimensional medium, capable of aural reproduction when viewing the document and whilst displaying the said sequence over the document.

15. A document according to claim 14, in which the medium comprises a transducer for generating the aural reproduction.

16. A document according to claim 13, including a code representative of an audio sequence stored elsewhere, and capable of aural reproduction when viewing the document and whilst displaying the said sequence over the document.

18. Apparatus for viewing a document or a photograph according to any of claims 13 to 17, comprising a projector for displaying the said sequence over the document or photograph.

19. Apparatus according to claim 18, comprising means for simultaneously playing back the aural reproduction from the audio memory, in association with the said sequence.

17. A photograph on a two dimensional medium comprising a computer-readable representation of a chronologic sequence of pointing gestures associated with the photograph.

25. Apparatus for assisting the reading of a paper document, comprising means for imaging the paper document to form an imaged document and for tracking the movement of a pointer over the paper document to associate that movement with the imaged document as a chronologic sequence of pointing gestures, and means for displaying said chronologic sequence in association with a reproduction of the same document.

26. Apparatus for assisting the reading of a paper document, comprising means for imaging the paper document to form an imaged document and for tradking the movement of a pointer over the paper document to associate that movement with the imaged document as a chronologic sequence of pointing gestures, and means for displaying said chronologic sequence on a reproduction on paper of the same document.

27. Apparatus for assisting the reading of a document by displaying upon it a chronologic sequence of pointing gestures, comprising optical apparatus for imaging the document to form an imaged document, and for tracking the movement of a pointer over the document, a processor responsive to said optical apparatus to associate said movement with said imaged document as said sequence, and computer apparatus for displaying said sequence in association with a reproduction of the same document.

28. Apparatus according to claim 27, in which the optical apparatus for imaging the document comprises a camera focussed on a real two dimensional medium on which the document is printed or scribed, and a processor linked to the camera for processing an image captured by the camera, and for orientating the image using predetermined registration criteria.

29. Apparatus according to claim 28, in which the image processing means is arranged to identify the position of the pointer in the captured image and thereby to track it.

30. Apparatus according to claim 27, comprising a synchronous communications system and source and destination processors linked by said communications system, the source processor being arranged to read the document and track the sequence of pointing gestures, and the destination processor being arranged to display the said sequence over the reproduction of the same document.

31. Apparatus according to claim 30, in which the destination processor has means for receiving an electronic version of the document and for printing it as the said reproduction.

32. Apparatus according to claim 30, in which the destination processor has means for receiving an electronic version of the document and for displaying it visually as the said reproduction.

33. A method of assisting the reading of a document by displaying upon it a chronologic sequence of pointing gestures, comprising imaging, at a first location, the document and tracking the movement of a pointer over the document to associate that movement with the imaged document as the said sequence, and displaying, at a second location remote from said first location, the sequence in association with a reproduction of the same document.

34. A method according to claim 33, for synchronous electronic conferencing over a communications system, comprising transmitting the said sequence of pointing gestures over the communications system between said first and second locations.

35. A method according to claim 34, further comprising transmitting an image of the document over the communications system.

36. A method according to claim 35, further comprising printing a reproduction of the document from the transmitted image of that document, and then projecting the said chronologic sequence of pointing gestures onto the printed reproduction.

37. Apparatus for assisting the reading of a paper document, comprising optical apparatus for imaging the paper document to form an imaged document and for tracking the movement of a pointer over the paper document to associate that movement with the imaged document as a chronologic sequence of pointing gestures, and computer apparatus for displaying said chronologic sequence in association with a reproduction of the same document.

38. Apparatus for assisting the reading of a paper document, comprising optical apparatus for imaging the paper document to form an imaged document and for tracking the movement of a pointer over the paper document to associate that movement with the imaged document as a chronologic sequence of pointing gestures, and computer apparatus for displaying said chronologic sequence on a reproduction on paper of the same document.