Paper-to-computer interfaces

• US 7,317,557 B2
• Filed: 07/29/2002
• Issued: 01/08/2008
• Est. Priority Date: 07/27/2001
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Pollard, Stephen Bernard, Grosvenor, David Arthur, Pilu, Maurizio
• Primary Examiner(s): Safaipour; Houshang
• Application Number:
  US10/206,350
• Publication Number:
  US 20030025951A1
• Time in Patent Office: 1,989 Days
• Field of Search: 358/448, 358/474, 358/505, 358/497, 358/1.9, 358/498, 358/100, 358/462.11, 358/462.14, 382/100, 235/462.11, 235/462.14
• US Class Current: 358/448
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 1/32133 on the same paper sheet, e.g. a facsimile page header
• CPC Class Codes: H04N 2201/0436 Scanning a picture-bearing surface lying face up on a support H04N2201/0434 takes precedence; using cylindrical picture bearing surfaces H04N1/06; with manual scanning H04N1/107
• CPC Class Codes: H04N 2201/3226 of identification information or the like, e.g. ID code, index, title, part of an image, reduced-size image
• CPC Class Codes: H04N 2201/3269 of machine readable codes or marks, e.g. bar codes or glyphs
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 2201/328 Processing of the additional information

Abstract

The present invention provides an apparatus and method which facilitates the use of a printed or scribed document as an interface with a computer. The apparatus in one aspect comprises: a printed or scribed document bearing specialized calibration marks, the document being positioned on a work surface; a camera focused on the document for generating video signals representing the document in electronic form; and a processor linked to the camera for processing an image captured by the camera and configured to identify the calibration marks of the document in the captured image and then determine from the location of the calibration marks in the image a transformation between co-ordinates of features. In the image and corresponding co-ordinates of features in the document that compensates for the freely variable positioning of the document on the work surface.

Claims

1. An apparatus which facilitates the use of a printed or scribed document as an interface with a computer and which comprises:

a. a work surface configured to receive a printed or scribed document bearing specialized calibration marks, the document being freely, variably and arbitrarily positioned on the work surface;

b. a camera for focusing on the document for generating video signals representing the document in electronic form; and

c. a processor linked to the camera for processing an image captured by the camera and configured to identify specialized calibration marks of the document in the captured image and then determine from the location of the specialized calibration marks in the image a transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document, and that compensates for the positioning of the document on the work surface.

2. An apparatus as claimed in claim 1, wherein the printed or scribed document is positioned flat on the work surface at an angle of orientation on the work surface that is freely and arbitrarily variable and the processor is configured to determine from the location of the specialized calibration marks in the image a transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document that compensates for the angle of orientation of the document.

3. An apparatus as claimed in claim 2, wherein the processor is configured to determine a transformation of the form:

4. An apparatus as claimed in claim 1, wherein the printed or scribed document may be positioned on the work surface in an oblique position relative to the camera and the processor is configured to determine from the location of the specialized calibration marks in the image a transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document that compensates for a said oblique position of the document.

5. An apparatus as claimed in claim 1 which holds in a memory information concerning the content of the document at specified co-ordinates of the document.

6. An apparatus as claimed in claim 1, wherein the document is further marked with a symbol which identifies the document page.

7. An apparatus as claimed in claim 6, wherein the document page identification symbol is a machine-readable code.

8. An apparatus as claimed in claim 7, wherein the document page identification symbol is a bar code.

9. An apparatus as claimed in claim 5, wherein the document is further marked with a symbol which identifies the document page, and wherein the memory holds information on the co-ordinates where a page identification symbol is to be found on the document, such that once the processor has determined the correspondence between co-ordinates of the image and co-ordinates of the document, the co-ordinates within the image at which the symbol should be found may be processed to determine the identity of the document page.

10. An apparatus as claimed in claim 1, wherein four spaced apart calibration marks are provided on the document.

11. An apparatus as claimed in claim 10 wherein the specialized calibration marks are in the form of a rectangle, one approximately at each respective corner of the document.

12. An apparatus as claimed in claim 1 wherein at least one of the specialized calibration marks has the form of a target comprising a plurality of concentric rings or a bulls-eye with at least one concentric ring.

13. An apparatus as claimed in claim 1 wherein at least one of the specialized calibration marks has a different form from the other calibration mark(s).

14. A method of using a printed or scribed document as an interface with a computer, which method comprises:

a. providing said printed or scribed document bearing specialized calibration marks on a work surface that is capable of receiving the printed or scribed document in a freely, variable and arbitrary position, and a camera focused on the document for generating video signals representing the document in electronic form and a processor linked to the camera for processing an image captured by the camera and configured to identify the specialized calibration marks of the document in the captured image and then determine from the location of the specialized calibration marks in the image a transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document that compensates for positioning of the document on the work surface;

b. capturing an image of the document with the camera; and

c. identifying, by the processor, the specialized calibration marks of the document in the captured image and then determining, from the location of the specialized calibration marks in the image, the transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document that compensates for the positioning of the document on the work surface.

15. A processor for use in an apparatus which facilitates the use of a printed or scribed document as an interface with a computer, the processor being programmed for processing an image captured by a camera of a printed or scribed document bearing specialized calibration marks and freely, variably and arbitrarily positioned on a work surface to identify the specialized calibration marks of the document in the captured image and then determine from the location of the specialized calibration marks in the captured image a transformation between co-ordinates of features in the captured image and corresponding co-ordinates of features in the document that compensates for the positioning of the document on the work surface.

16. A method for facilitating the use of a printed or scribed document, the method comprising the steps of:

processing an image captured by a camera of a printed or scribed document bearing specialized calibration marks and freely variably and arbitrarily positioned on a work surface to identify the specialized calibration marks of the document in the captured image; and

determining, from the location of the specialized calibration marks in the captured image, a transformation between co-ordinates of features in the captured image and corresponding co-ordinates of features in the document that compensates for the positioning of the document on the work surface.

17. An apparatus which facilitates the use of a printed or scribed document as an interface with a computer and which comprises:

a. a work table configured to receive a printed or scribed document bearing specialized calibration marks, the document being freely, variably and arbitrarily positioned on the work table;

b. a camera mounted on a stand viewing down over the work table for focusing on the document for generating video signals representing the document in electronic form; and

c. a processor linked to the camera for processing an image captured by the camera and configured to identify the specialized calibration marks of the document in the captured image and then determine from the location of the specialized calibration marks in the image a transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document, and that compensates for the positioning of the document on the work surface.

18. An apparatus as claimed in claim 17, wherein the printed or scribed document is positioned flat on the work surface at an angle of orientation on the work surface that is freely and arbitrarily variable and the processor is configured to determine from the location of the specialized calibration marks in the image a transformation between co-ordinates of features in the image and corresponding co-ordinates of features in the document that compensates for the angle of orientation of the document.

19. An apparatus as claimed in claim 18, wherein the processor is configured to determine a transformation of the form: