Obtaining input from a virtual user interface
First Claim
Patent Images
1. A system comprising:
- one or more processors;
one or more cameras to obtain at least a first 3-dimensional (3D) image and a second 3D image of an environment, the first image and the second image including a display object and an operating object;
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;
generating first 2-dimensional (2D) contours of the display object from the first 3D image and second 2D contours of the operating object from the first 3D image;
generating first 1-dimensional (1D) curvature information from the first 2D contours of the display object and second 1D curvature information from the second 2D contours of the operating object;
generating third 2D contours from the second 3D image, wherein the third 2D contours include at least a portion of the first 2D contours of the display object and a first portion of the second 2D contours of the operating object, and a second portion of the second 2D contours of the operating object is absent from the third 2D contours;
generating third 1D curvature information from the third 2D contours; and
determining a location in the second 3D image of the second portion of the second 2D contours of the operating object that is absent from the third 2D contours based, at least in part, on the second 1D curvature information and the third 1D curvature information.
2 Assignments
0 Petitions
Accused Products
Abstract
A virtual user interface in a 3D environment is projected onto a display object, such as an open hand. An operating object, such as a stylus or a second hand with an index finger extended, may be used to select input features of the projected user interface. In one scenario, for example, a telephone keypad may be projected onto the open hand and the index finger may be used to select a particular number on the keypad. Images of the hands may be segmented to produce contours of the finger and open hand, and these contours are used to determine the location of the finger relative to the open hand.
-
Citations
27 Claims
-
1. A system comprising:
-
one or more processors; one or more cameras to obtain at least a first 3-dimensional (3D) image and a second 3D image of an environment, the first image and the second image including a display object and an operating object; one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising; generating first 2-dimensional (2D) contours of the display object from the first 3D image and second 2D contours of the operating object from the first 3D image; generating first 1-dimensional (1D) curvature information from the first 2D contours of the display object and second 1D curvature information from the second 2D contours of the operating object; generating third 2D contours from the second 3D image, wherein the third 2D contours include at least a portion of the first 2D contours of the display object and a first portion of the second 2D contours of the operating object, and a second portion of the second 2D contours of the operating object is absent from the third 2D contours; generating third 1D curvature information from the third 2D contours; and determining a location in the second 3D image of the second portion of the second 2D contours of the operating object that is absent from the third 2D contours based, at least in part, on the second 1D curvature information and the third 1D curvature information. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method comprising:
-
obtaining data from an environment, the data including a plurality of images, wherein each of the plurality of images includes one or more objects; identifying a display object and an operating object in a first image of the plurality of images, wherein the operating object includes an input portion; projecting a user interface onto the display object; determining when a portion of the operating object overlaps with a portion of the display object in a second image to produce a merged contour, wherein the portion of the operating object that overlaps with the portion of the display object includes the input portion, and particular contours of the input portion are absent from the merged contour; and determining a location of the input portion in the second image. - View Dependent Claims (9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A method comprising:
-
identifying a first hand and a second hand in an image, wherein the first hand functions as a display surface for a projected user interface and the second hand functions as a pointing device to interact with the user interface; segmenting the image to produce contours of the first and second hands; determining, from the contours, when the first and second hands overlap such that part of the second hand forming the pointing device is obscured; and deriving a location of the part of the second hand relative to the first hand to identify an interaction with the user interface. - View Dependent Claims (20, 21, 22)
-
-
23. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:
-
identifying a display hand and an operating hand in a first image; generating first contours of the display hand and second contours of the operating hand from the first image; generating first curvature information from the first contours and second curvature information from the second contours; identifying one or more continuous surfaces in a second image; generating respective contours for each of the one or more continuous surfaces in the second image; identifying contours of a particular continuous surface in the second image that include at least a portion of the first contours and at least a portion of the second contours; and determining a location of a particular portion of the second contours that is absent from the contours of the particular continuous surface based at least in part on the second curvature information. - View Dependent Claims (24, 25, 26, 27)
-
Specification