Providing an interactive experience using a 3D depth camera and a 3D projector
First Claim
1. A method for providing an interactive experience to a user using an interaction system, comprising:
- capturing a depth image of a physical object placed on, or in proximity to, a surface using a depth camera, the depth image comprising information regarding the distance of different parts of the physical object to the depth camera;
capturing a video image of the physical object using a video camera, wherein the depth camera is not the video camera, and wherein the video image is captured in conjunction with the depth image;
generating a 3D virtual object which is a counterpart of the physical object based on the depth image and the video image; and
projecting, using a 3D projector, the 3D virtual object on, or in proximity to, the surface.
2 Assignments
0 Petitions
Accused Products
Abstract
An interaction system is described which uses a depth camera to capture a depth image of a physical object placed on, or in vicinity to, an interactive surface. The interaction system also uses a video camera to capture a video image of the physical object. The interaction system can then generate a 3D virtual object based on the depth image and video image. The interaction system then uses a 3D projector to project the 3D virtual object back onto the interactive surface, e.g., in a mirrored relationship to the physical object. A user may then capture and manipulate the 3D virtual object in any manner. Further, the user may construct a composite model based on smaller component 3D virtual objects. The interaction system uses a projective texturing technique to present a realistic-looking 3D virtual object on a surface having any geometry.
-
Citations
20 Claims
-
1. A method for providing an interactive experience to a user using an interaction system, comprising:
-
capturing a depth image of a physical object placed on, or in proximity to, a surface using a depth camera, the depth image comprising information regarding the distance of different parts of the physical object to the depth camera; capturing a video image of the physical object using a video camera, wherein the depth camera is not the video camera, and wherein the video image is captured in conjunction with the depth image; generating a 3D virtual object which is a counterpart of the physical object based on the depth image and the video image; and projecting, using a 3D projector, the 3D virtual object on, or in proximity to, the surface. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. An interaction system, comprising:
-
an illumination source for projecting radiation onto a surface; a depth camera for capturing a depth image of at least one physical object by receiving radiation, emitted by the illumination source, the depth image comprising information regarding the distance of different parts of the physical object to the depth camera, which illuminates the at least one physical object on, or in proximity to, the surface; a video camera for capturing a video image of said at least one physical object, wherein the depth camera and the video camera are separate cameras, and wherein the video camera captures the video image of the at least one physical object in conjunction with the depth camera capturing the depth image of the at least one physical object; a virtual scene generation module for generating at least one 3D virtual object based on the depth image and the video image; and a stereoscopic projector for projecting said at least one 3D virtual object on, or in proximity to, the surface. - View Dependent Claims (17, 18)
-
-
19. A computer readable storage device for storing computer readable instructions, the computer readable instructions providing a virtual scene generation module when executed by one or more processing devices, the computer readable instructions comprising:
-
logic configured to receive a depth image from a depth camera of a physical object placed on, or in proximity to, a surface, the depth image comprising information regarding the distance of different parts of the physical object to the depth camera; logic configured to receive a video image from a video camera of the physical object, wherein the depth camera is not the video camera, and wherein the video camera captures the video image of the physical object in conjunction with the depth camera capturing the depth image of the physical object; logic configured to generate a 3D virtual object, the 3D virtual object being a 3D representation of the physical object based on the depth image and the video image; and logic configured to apply a texture, obtained from the video image, to the 3D virtual object in a form of a perspective-adjusted 3D image, the perspective-adjusted 3D image being projected, by a 3D projector, to a location on, or in proximity to, the surface. - View Dependent Claims (20)
-
Specification