Foreground analysis on parametric background surfaces
First Claim
Patent Images
1. A system comprising:
- one or more processors;
an image sensor;
a projector;
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;
capturing one or more images of an environment using the image sensor;
determining a location of a display medium within the environment based at least in part on the one or more images, wherein the display medium has a known parametric description;
projecting content onto the display medium using the projector;
determining an observed depth map of at least a portion of the environment that contains the display medium, wherein the observed depth map includes first depth information associated with the display medium and an object within the environment that at least partially occludes the display medium;
computing a display depth map of the display medium based at least in part on the location and the known parametric description of the display medium, wherein the display depth map includes second depth information associated with the display medium, the second depth information including a first depth for a first point of the display medium that is unobscured in the one or more images and a second depth for a second point of the display medium that is obscured by the object and that is not depicted in the one or more images; and
producing a foreground depth map based at least in part on differences between the second depth information of the display depth map and the first depth information of the observed depth map, the foreground depth map indicating a difference in depth between a third point associated with the observed depth map and a corresponding third point associated with the display depth map.
2 Assignments
0 Petitions
Accused Products
Abstract
Techniques are described for projecting content onto a display surface and for analyzing the hand of a user that is placed over the display surface, such as when the user is making a command gesture. Based on the location and known shape of the display surface, a surface depth map is calculated corresponding to the display surface. Points of the calculated surface depth map are then subtracted from respective points an observed depth map to produce a foreground depth map corresponding to the user hand. The hand may be further characterized and tracked based on the foreground depth map.
-
Citations
25 Claims
-
1. A system comprising:
-
one or more processors; an image sensor; a projector; one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising; capturing one or more images of an environment using the image sensor; determining a location of a display medium within the environment based at least in part on the one or more images, wherein the display medium has a known parametric description; projecting content onto the display medium using the projector; determining an observed depth map of at least a portion of the environment that contains the display medium, wherein the observed depth map includes first depth information associated with the display medium and an object within the environment that at least partially occludes the display medium; computing a display depth map of the display medium based at least in part on the location and the known parametric description of the display medium, wherein the display depth map includes second depth information associated with the display medium, the second depth information including a first depth for a first point of the display medium that is unobscured in the one or more images and a second depth for a second point of the display medium that is obscured by the object and that is not depicted in the one or more images; and producing a foreground depth map based at least in part on differences between the second depth information of the display depth map and the first depth information of the observed depth map, the foreground depth map indicating a difference in depth between a third point associated with the observed depth map and a corresponding third point associated with the display depth map. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method, comprising:
-
capturing one or more images of an environment that contain a first representation of a target object; determining an observed depth map of at least a portion of the environment, wherein the observed depth map includes first depth information associated with the target object; computing a target depth map based on a known location of the target object and a known parametric description associated with the target object, wherein the target depth map includes second depth information associated with the target object, the second depth information including a first depth for a first point of the target object that is unobscured in the one or more images and a second depth for a second point of the target object that is obscured by a foreground object in the one or more images; and determining a foreground depth map based at least in part on differences between the second depth information of the target depth map and the first depth information of the observed depth map, the foreground depth map indicating a difference in depth between a third point associated with the observed depth map and a corresponding third point associated with the target depth map. - View Dependent Claims (9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:
-
determining, from one or more images of an environment, a location of a target object within the environment, wherein the target object is partially occluded by a hand of a user; determining an observed depth map of at least a portion of the environment that contains the target object, wherein the observed depth map includes first depth information associated with the at least the portion of the environment; computing a target depth map of the target object based on the location and a known parametric description of the target object, wherein the target depth map includes second depth information associated with the target object, the second depth information including a first depth for a first point of the target object that is unobscured in the one or more images and a second depth for a second point of the target object that is obscured by the hand of the user in the one or more images; and producing a foreground depth map corresponding to the hand of the user based at least in part on differences between the second depth information associated with the target depth map and the first depth information associated with the observed depth map, the foreground depth map indicating a difference in depth between a third point associated with the observed depth map and a corresponding third point associated with the target depth map. - View Dependent Claims (20, 21, 22, 23, 24, 25)
-
Specification