Method and system for a user-following interface

• US 7,530,019 B2
• Filed: 09/19/2006
• Issued: 05/05/2009
• Est. Priority Date: 08/23/2002
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kjeldsen, Frederik Carl Moesgaard, Pinhanez, Claudio Santos, Pingali, Gopal Sarma, Levas, Anthony
• Primary Examiner(s): Vu; Kieu D.
• Assistant Examiner(s): Basom; Blaine T.
• Application Number:
  US11/523,484
• Publication Number:
  US 20070013716A1
• Time in Patent Office: 959 Days
• Field of Search: 715/730, 715/848, 715/757, 715/761, 715/850, 715/852, 715/731, 715/732, 715/863, 348/208.14, 353/48, 353/49, 353/71, 353/89, 353/94, 353/82, 353/28, 702/150, 702/152
• US Class Current: 715/730
• CPC Class Codes: G03B 21/28 Reflectors in projection beam in illumination beam G03B21/2066
• CPC Class Codes: H04N 5/74 Projection arrangements for image reproduction, e.g. using eidophor
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

Techniques are disclosed for projecting an image onto a surface suitable for interaction with a user while avoiding user occlusion, and while correcting for distortion due to oblique projection. The displayed image moves to a suitable surface at a suitable size and orientation as a user moves around an environment, resulting in a user-following interface. Surfaces are selected in which the projected interface is not occluded by the user or other objects in the environment. Displayed images may be interactive, and moved into an interaction area on a suitable surface that is convenient for the user. The interaction area may or may not coincide with the display area. Adaptation of the projected interface is allowed so that the content of the display and the style of interaction widgets are modified based on distance from the user and orientation of the user with respect to a projected interface.

Claims

1. A user-following interface system comprising:

one or more projectors adapted to project one or more images onto one or more selected surfaces in a real environment, wherein the one or more selected surfaces are selected from a plurality of surfaces;

a user tracking module adapted to track positions of one or more users in the environment;

a controller coupled to the one or more projectors and the user tracking module, the controller adapted to select the one or more surfaces based on the positions of the one or more users in the real environment, and to provide information to the one or more projectors suitable for allowing the one or more projectors to project the one or more images onto the one or more selected surfaces; and

a sensing mechanism adapted to detect user interactions with a selected image projected onto one of the selected surfaces,

wherein the controller is further adapted to select an interaction area for an image by;

mapping position and orientation of a user into a three-dimensional environment model;

determining a volume in the three-dimensional model that is reachable by the user;

determining a set of surfaces available for interaction in the reachable volume;

determining available interaction areas on each surface;

determining a distance from an appendage of the user to a center of an available interaction area;

eliminating interaction areas that are occluded by the user to create remaining interaction areas; and

selecting one of the remaining interaction areas, within the reachable volume, that is closest to the user while being approximately orthogonal to a viewing direction of the user.