Object identification method and system for an augmented-reality display

• US 6,898,307 B1
• Filed: 09/22/1999
• Issued: 05/24/2005
• Est. Priority Date: 09/22/1999
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Harrington, Steven J.
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/400,755
• Time in Patent Office: 2,071 Days
• Field of Search: 382/190, 382/203, 382/218, 382/219, 382/286, 382/288, 382/291, 382/106, 382/154, 348/14.16, 348/42, 348/47, 345/7, 345/619, 345/629, 345/633
• US Class Current: 382/154
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

A method and apparatus selectively displays a processor generated image in accordance with user instructions detected by a processor from a processor viewing device disposed to view a real reference item controlled by the user. The real item, preferably a page of paper, is disposed as a reference frame for an electronic image to be displayed. The orientation of the page is identified by the system. Corresponding real world coordinates are computed for the captured image and compared with predetermined standards to verify the nature of the object.

Claims

1. A method of verifying a projected image within a three-dimensional view plane of an augmented-reality display system as a preselected movable real object disposed in three-dimensional free space, whereby the preselected movable real object may be employed as an interface tool for the system, comprising steps of:

identifying a representative characteristic of the preselected movable real object within the three-dimensional view plane, the preselected movable real object is comprised of three equidistant line points disposed in free space, and wherein the representative characteristic comprises shape and location of the preselected movable real object and is exclusive of preselected marked standards and printed identifiers;

determining dimensional aspects of the preselected movable real object from the projected image including the detection of projected dimensions of the three equidistant line points;

computing a corresponding dimensional identity and location of the preselected movable real object at an object point relative to the view plane including calculating X, Y and Z coordinates in real space of the preselected movable real object at the object point based on the projected dimensions of the three equidistant line points in the view plane and known augmented-reality display system geometric dimensions; and

verifying whether the dimensional identity and location are reasonably consistent with predetermined standards for the preselected movable real object.

2. The method as claimed in claim 1 wherein the preselected movable real object comprises a reference panel such as a screen, tablet or piece of paper and the identifying includes recognizing a corner of the panel.

3. The method as claimed in claim 2 wherein the determining comprises calculating distances between corners and a center point of the reference panel.

4. The method as claimed in claim 3 wherein the computing comprises converting the calculated distances to the dimensional identity and location based on an assumption that the reference panel is structurally flat.

5. The method as defined in claim 1 wherein the verifying includes testing from at least one of the tests of (a) whether the preselected movable real object has expected dimensions or proportions, (b) whether the corners are right angles, (c) whether a center point matches when calculated from distinct sets of the corners, (d) whether the corners are generally within a common plane, and (e) whether the preselected movable real object lies within an expected viewing range.

6. A method for identifying a movable real piece of paper disposed in free space in a variable three-dimensional viewing area of an augmented-reality display system comprising steps of:

identifying a movable real object at a viewing plane in the three-dimensional viewing area having a characteristic representative of the piece of paper wherein the characteristic is exclusive of preselected registration marks and printed identifiers;

locating a plurality of corners of the movable real object;

calculating a dimensional representation of the movable real object in the viewing plane from the locations of the corners;

unprojecting the dimensional representation to calculate a plurality of the movable real object coordinates representative of a size of the movable real object and a distance of the movable real object from the viewing plane including calculating X, Y and Z coordinates in real space of the movable real object based on dimensions of the movable real object in the viewing plane and display system geometries; and

comparing the movable real object coordinates with predetermined standards indicative of the piece of paper for verifying the object as the piece of paper.

7. The method as defined in claim 6, wherein the calculating includes identifying a diagonal between the corners comprised of three equidistant line points.

8. The method as defined in claim 7 wherein the identifying includes identifying a center point of the diagonal.

9. The method as defined in claim 6 wherein the comparing includes testing from at least one of the tests of (a) whether the object has expected dimensions or proportions, (b) whether the corners are right angles, (c) whether a center point matches when calculated from distinct sets of the corners, (d) whether the corners are generally within a common plane, and (e) whether the object lies within an expected viewing range.

10. The method as defined in claim 6 wherein the unprojecting comprises unprojecting a plurality of dimensional representations of the movable real object attributable to three-dimensional movement of the movable real object in the variable viewing area.