Near touch interaction with structured light
First Claim
1. A system for determining near touch interaction, comprising:
- a display for displaying one or more elements thereon;
one or more stereo camera pairs positioned adjacent to the display; and
a processor for causing the display to flash one or more structured light sequences using the light sources that illuminate the display on a user pointer placed adjacent to the display, determining a depth map in accordance with information acquired by the stereo camera pair, and if desired, employing the one or more structured light sequences to possibly refine the depth map, wherein the display produces patterns targeting only regions of the field-of view where the depth refinement is determined to be beneficial, and determining a location corresponding to one or more of the one or more elements displayed on the display to be selected in accordance with the determined depth map.
1 Assignment
0 Petitions
Accused Products
Abstract
A near-touch interface is provided that utilizes stereo cameras and a series of targeted structured light tessellations, emanating from the screen as a light source and incident on objects in the field-of-view. After radial distortion from a series of wide-angle lenses is mitigated, a surface-based spatio-temporal stereo algorithm is utilized to estimate initial depth values. Once these values are calculated, a subsequent refinement step may be applied in which light source tessellations are used to flash a structure onto targeted components of the scene, where initial near-interaction disparity values have been calculated. The combination of a spherical stereo algorithm, and smoothing with structured light source tessellations, provides for a very reliable and fast near-field depth engine, and resolves issues that are associated with depth estimates for embedded solutions of this approach.
-
Citations
25 Claims
-
1. A system for determining near touch interaction, comprising:
-
a display for displaying one or more elements thereon; one or more stereo camera pairs positioned adjacent to the display; and a processor for causing the display to flash one or more structured light sequences using the light sources that illuminate the display on a user pointer placed adjacent to the display, determining a depth map in accordance with information acquired by the stereo camera pair, and if desired, employing the one or more structured light sequences to possibly refine the depth map, wherein the display produces patterns targeting only regions of the field-of view where the depth refinement is determined to be beneficial, and determining a location corresponding to one or more of the one or more elements displayed on the display to be selected in accordance with the determined depth map. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for determining near touch interaction by a user object, comprising the steps of:
-
displaying on a display one or more objects; acquiring one or more images of the user object by one or more pairs of stereo cameras located adjacent to the display; dewarping the one or more acquired images; computing a disparity based upon the dewarped images; identifying one or more target regions of the display adjacent to one or more corresponding portions of the disparity that may benefit from refinement, and displaying one or more screen tessellations therein using the light sources that illuminate the display, wherein the display produces patterns targeting only regions of the field-of view where the depth refinement is determined to be beneficial; refining the computed disparity in accordance with additional data received in accordance with the displayed screen tessellations; and determining one or more of the displayed objects to be selected by the user object in accordance with the redefined computer disparity. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24)
-
-
25. A computer program stored on a non-transitory computer medium, the computer program causing a processor to implement an API to perform the steps of:
-
generating or refining a coarse depth map; determining one or more critical data points to assist in refining the coarse depth map; providing control over fine-tuning one or more light tessellations to be displayed on at least a portion of a display using the light sources that illuminate the display to aid in acquisition of the one or more critical data points, wherein the display produces patterns of the tessellations targeting only regions of the field-of view where the depth refinement is determined to be beneficial; and performing scene recognition and analysis in accordance with one or more artificial intelligence algorithms based upon the coarse depth map as refined in accordance with the acquired one or more critical data points.
-
Specification