Projectors and depth cameras for deviceless augmented reality and interaction
First Claim
1. A multi-dimensional modeling system, comprising:
- an arrangement of depth-sensing cameras that sense objects in a physical space and output sensed information;
an arrangement of projectors that impose projections on surfaces in the physical space; and
a control component that processes the sensed information and the projections to map a virtual interactive space to the physical space to emulate an interactive display for interaction with the objects and the surfaces,the projections interrelated such that objects in at least two projections are cross-referenced to establish connectivity in the physical space,at least a portion of the connectivity computed by constructing tables that index connecting components from one view to another view.
2 Assignments
0 Petitions
Accused Products
Abstract
Architecture that combines multiple depth cameras and multiple projectors to cover a specified space (e.g., a room). The cameras and projectors are calibrated, allowing the development of a multi-dimensional (e.g., 3D) model of the objects in the space, as well as the ability to project graphics in a controlled fashion on the same objects. The architecture incorporates the depth data from all depth cameras, as well as color information, into a unified multi-dimensional model in combination with calibrated projectors. In order to provide visual continuity when transferring objects between different locations in the space, the user'"'"'s body can provide a canvas on which to project this interaction. As the user moves body parts in the space, without any other object, the body parts can serve as temporary “screens” for “in-transit” data.
-
Citations
20 Claims
-
1. A multi-dimensional modeling system, comprising:
-
an arrangement of depth-sensing cameras that sense objects in a physical space and output sensed information; an arrangement of projectors that impose projections on surfaces in the physical space; and a control component that processes the sensed information and the projections to map a virtual interactive space to the physical space to emulate an interactive display for interaction with the objects and the surfaces, the projections interrelated such that objects in at least two projections are cross-referenced to establish connectivity in the physical space, at least a portion of the connectivity computed by constructing tables that index connecting components from one view to another view. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A multi-dimensional modeling system, comprising:
-
an arrangement of depth-sensing cameras that sense a distance value per pixel of objects in a physical space and output 3D sensed information, and an arrangement of projectors that impose 2D projections on surfaces in the physical space; and a control component that processes the 3D sensed information and the projections into virtual camera images as created from perspective views of virtual cameras, the virtual camera images employed to emulate an interactive display for user interaction with the objects and surfaces in the physical space, the projections interrelated such that objects in at least two projections are cross-referenced to establish connectivity in the physical space, at least a portion of the connectivity computed by constructing tables that index connecting components from one view to another view. - View Dependent Claims (11, 12, 13, 14)
-
-
15. A multi-dimensional modeling method, comprising:
-
sensing objects within a physical space to output sensed information using an arrangement of depth-sensing cameras; projecting images on surfaces in the physical space using an arrangement of projectors; creating and overlaying a virtual interactive space onto the physical space based on the sensed information and the images; and presenting interactions with an object relative to a surface in the physical space, the projections interrelated such that objects in at least two projections are cross-referenced to establish connectivity in the physical space, at least a portion of the connectivity computed by constructing tables that index connecting components from one view to another view. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification