CYCLICAL CREATION, TRANSFER AND ENHANCEMENT OF MULTI-MODAL INFORMATION BETWEEN PAPER AND DIGITAL DOMAINS

• US 20090021493A1
• Filed: 05/29/2008
• Published: 01/22/2009
• Est. Priority Date: 05/29/2007
• Status: Active Grant

• Current Assignee: Livescribe Incorporated (Anoto Group AB)
• Original Assignee: Livescribe Incorporated (Anoto Group AB)
• Inventors: Edgecomb, Tracy L., Marggraff, Jim
• Granted Patent: US 8,416,218 B2
• Time in Patent Office: Days
• US Class Current: 345/179
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 40/171 by use of digital ink

Abstract

In a pen-based computing system, multi-modal data is transferred between a paper domain and a digital domain. Data initially generated in the paper domain is captured by a smart pen and a digital file including the captured data is generated. For example, a computing system coupled to the smart pen generates a digital file including the captured data. A paper representation of the digital file is subsequently generated. The digital file can subsequently be modified by editing the paper representation of the digital file using the smart pen. Edits to the paper representation of the digital file are captured by the smart pen and converted to the digital domain where they are used to edit the content of the digital file.

Claims

1. A computer program product for transferring data between a paper domain and a digital domain, the computer program product comprising a computer-readable storage medium containing computer program code for:

receiving data captured by a smart pen from the paper domain;

generating a digital file including the data captured from the paper domain;

generating a paper representation of the digital file;

receiving editing data captured by the smart pen from the paper domain, the editing data modifying a content of the paper representation of the digital file; and

responsive to the editing data captured by the smart pen, modifying the digital file to generate a modified digital file.

2. The computer program product of claim 1, wherein the data captured by the smart pen comprises at least one of:

writing content, image content, audio content, video content, data or sensed motion of the smart pen.

3. The computer program product of claim 1, further comprising:

receiving annotation data captured by the smart pen, the annotation data associated with a subset of the paper representation of the digital file;

associating the annotation data with a subset of the digital file corresponding to the subset of the paper representation of the digital file.

4. The computer program product of claim 1, wherein generating the digital file including the data captured from the paper domain comprises:

applying a content recognition algorithm to identify content included in the data captured from the paper domain; and

formatting the identified content included in the data captured from the paper domain into the digital file.

5. A pen-based computing system for transferring data between a paper domain and a digital domain the system comprising:

a smart pen device capturing data from the paper domain;

a computing system coupled to the smart pen device, the computing device receiving data captured from the paper domain and generating a digital file including the data captured from the paper domain; and

a printer coupled to the computing system, the printer generating a paper representation of the digital file;

6. The pen-based computing system of claim 5, wherein:

the smart pen device generates and captures editing data modifying the paper representation of the digital file; and

the computing system receives the editing data modifying the paper representation of the digital file and generates a modified digital file responsive to the editing data.

7. The pen-based computing system of claim 6, wherein:

the printer generates a paper representation of the modified digital file.

8. The pen-based computing system of claim 5 wherein the data captured from the paper domain comprises at least one of:

writing content, image content, audio content, video content, data or sensed motion of the smart pen.

9. The pen-based computing system of claim 5 wherein the editing data comprises at least one of:

writing content, image content, audio content, video content, data or sensed motion of the smart pen.

10. The pen-based computing system of claim 5, wherein:

the smart pen device captures annotation data, the annotation data associated with a subset of the paper representation of the digital file; and

the computing system receives the annotation data and associating the annotation data with a subset of the digital file corresponding to the subset of the paper representation of the digital file.