SYSTEMS AND METHODS FOR INTERACTIONS WITH DOCUMENTS ACROSS PAPER AND COMPUTERS

• US 20120042288A1
• Filed: 08/16/2010
• Published: 02/16/2012
• Est. Priority Date: 08/16/2010
• Status: Abandoned Application

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Liu, Qiong, Chiu, Patrick, Chen, Francine, LIAO, Chunyuan, Tang, Hao
• Application Number:
  US12/857,497
• Publication Number:
  US 20120042288A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 1/00129 with a display device, e.g. CRT or LCD monitor with a television apparatus H04N1/00283; arrangements for control of display devices G09G
• CPC Class Codes: H04N 1/00241 using an image reading device as a local input to a computer input via network, e.g. network scanners H04N1/00225
• CPC Class Codes: H04N 1/195 the array comprising a two-dimensional array or a combination of two-dimensional arrays
• CPC Class Codes: H04N 1/19594 using a television camera or a still video camera
• CPC Class Codes: H04N 2201/0081 Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0436 Scanning a picture-bearing surface lying face up on a support H04N2201/0434 takes precedence; using cylindrical picture bearing surfaces H04N1/06; with manual scanning H04N1/107

Abstract

Systems and methods provide for mixed use of physical documents and a computer, and more specifically provide for detailed interactions with fine-grained content of physical documents that are integrated with operations on a computer to provide for improved user interactions between the physical documents and the computer. The system includes a camera which processes the physical documents and detects gestures made by a user with respect to the physical documents, a projector which provides visual feedback on the physical document, and a computer with a display to coordinate the interactions of the user with the computer and the interactions of the user with the physical document. The system, which can be portable, is capable of detecting interactions with fine-grained content of the physical document and translating interactions at the physical document with the computer display, and vice versa.

Claims

1. A system for interacting with physical documents and at least one computer, comprising:

a camera processing module which processes the content of at least one physical document and detects user interactions on the at least one physical document;

a projector processing module which provides visual feedback on the at least one physical document; and

a computer with a screen which coordinates the user interactions on the at least one physical document with an action on the computer.

2. The system of claim 1, wherein the camera processing module processes fine-grained content of the at least one physical document, including individual words, characters and graphics, and wherein the camera processing module detects user interactions relating to the fine-grained content.

3. The system of claim 1, wherein the visual feedback provided by the projector processing module is based on user interactions on the physical document.

4. The system of claim 1, wherein the user interactions further include gestures made on the at least one physical document which correspond to actions on the computer.

5. The system of claim 4, wherein the gestures correspond to pre-configured commands which result in a specific type of visual feedback.

6. The system of claim 1, wherein a user interaction on the computer is translated into visual feedback provided by the projector to the at least one physical document.

7. The system of claim 1, wherein the projector processing module provides visual feedback on a physical surface other than the physical document.

8. The system of claim 1, further comprising a portable, integrated camera and projector with a foldable frame and at least one mirror, the mirror attached to the frame and positioned over the at least one physical document to reflect an optical path of the camera and projector onto the at least one physical document.

9. The system of claim 1, wherein the camera processing module processes the content of the at least one physical document and obtains a corresponding digital document to display on the computer screen.

10. The system of claim 9, wherein the user interactions on the at least one physical document result in corresponding interactions on the corresponding digital document.

11. The system of claim 1, wherein the camera processing module processes the content of the at least one physical document and obtains digital content which relates to the at least one physical document.

12. A method for interacting with at least one physical document and at least one computer, comprising:

processing the at least one physical document;

detecting user interactions with the at least one physical document;

providing visual feedback on the at least one physical document; and

coordinating the user interactions on the at least one physical document with interactions on a computer with a screen.

13. The method of claim 12, further comprising:

processing the at least one physical document to identify fine-grained content, including individual words, characters and graphics; and

detecting user interactions relating to the fine-grained content.

14. The method of claim 12, wherein the visual feedback is based on user interactions on the physical document.

15. The method of claim 12, wherein the user interactions further include gestures made on the at least one physical document which correspond to actions on the computer.

16. The method of claim 15, wherein the gestures correspond to pre-configured commands which result in a specific type of visual feedback.

17. The method of claim 12, further comprising providing visual feedback on a physical surface other than the physical document.

18. The method of claim 12, further comprising translating a user interaction on the computer into visual feedback on the at least one physical document.

19. The method of claim 18, further comprising translating user interaction with the at least one physical document with simultaneous user interaction on the computer to manipulate detailed content of the at least one physical document.

20. The method of claim 12, wherein detailed content of the physical document is manipulated by user interactions using a first hand to interact with the at least one physical document and a second hand to interact with the computer.

21. The method of claim 12, wherein detailed content of the digital document is manipulated by user interactions using a first hand to interact with the at least one physical document and a second hand to interact with the computer.

22. The method of claim 12, further comprising synchronously manipulating detailed content of the physical document and a digital document on the computer using a first hand to interact with the at least one physical document and a second hand to interact with the digital document.

23. The method of claim 12, further comprising processing the content of the at least one physical document and obtaining a corresponding digital document to display on the computer screen.

24. The method of claim 23, wherein the user interactions on the at least one physical document result in corresponding interactions on the corresponding digital document.

25. The method of claim 12, further comprising processing the content of the at least one physical document and obtaining digital content which relates to the at least one physical document.

26. A computer program product for interacting with at least one physical document and a computer, the computer program product embodied on a computer readable storage medium and when executed by a computer, performs the method comprising:

processing the at least one physical document;

detecting user interactions with the at least one physical document;

providing visual feedback on the at least one physical document; and

coordinating the user interactions on the at least one physical document with interactions on a computer with a screen.