Hover detection in an interactive display device

• US 9,535,537 B2
• Filed: 02/23/2011
• Issued: 01/03/2017
• Est. Priority Date: 11/18/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Large, Timothy, Powell, Karlton, Bathiche, Steven
• Primary Examiner(s): PARDO, THUY N
• Application Number:
  US13/033,529
• Publication Number:
  US 20120127128A1
• Time in Patent Office: 2,141 Days
• Field of Search: 345/175, 345/173, 345/174, 345/156, 345/32, 345/690, 345/8, 345/633
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Embodiments are disclosed that relate to hover detection in interactive display devices. One embodiment provides an interactive display device comprising a display panel configured to display an image on an interactive surface, an imaging optical wedge disposed adjacent to the display panel, an image sensor configured to capture an image of an object located in front of the interactive surface and spaced from the interactive surface by capturing the image through the imaging optical wedge, a logic subsystem, and a data-holding subsystem comprising instructions executable by the logic subsystem to operate the display panel and the image sensor, and to detect a hover input based upon one or more images received from the image sensor.

Claims

1. An interactive display device, comprising:

a display panel configured to display an image on an interactive surface;

an imaging optical wedge disposed adjacent to the display panel;

an image sensor configured to capture an image of an object located in front of the interactive surface and spaced from the interactive surface by capturing the image through the imaging optical wedge and the display panel;

a variable diffuser comprising a diffusivity that is controllable between two or more states;

a logic subsystem; and

a data-holding subsystem comprising instructions executable by the logic subsystem to receive one or more images from the image sensor, detect a hover input based upon the one or more images received from the image sensor, distinguish blur arising due to hover distance from blur arising due to motion by deconvoluting the image using a predetermined diffusivity profile, and display a response to the hover input via the display panel.

2. The interactive display device of claim 1, wherein the image sensor comprises a three-dimensional image sensor.

3. The interactive display device of claim 2, wherein the three-dimensional image sensor comprises a time-of-flight depth sensor.

4. The interactive display device of claim 2, wherein the three-dimensional image sensor comprises a stereo camera system comprising two or more image sensors.

5. The interactive display device of claim 4, wherein the instructions are executable to acquire images with a first image sensor and a second image sensor of the stereo camera system at different diffusivities.

6. The interactive display device of claim 4, wherein the instructions are executable to acquire images with a first image sensor and a second image sensor of the stereo camera system at a same diffusivity.

7. The interactive display device of claim 2, wherein the three-dimensional image sensor comprises a structured light depth sensor.

8. The interactive display device of claim 7, wherein the structured light depth sensor is configured to project a structured light pattern through the imaging optical wedge.

9. The interactive display device of claim 7, wherein the structured light depth sensor is configured to project a structured light pattern via the display panel.

10. The interactive display device of claim 1, further comprising a diffuser positioned between the imaging optical wedge and the display panel, wherein the liquid crystal display panel is a modulating display panel, and wherein the instructions are executable to form a plurality of apertures via the display panel to focus a plurality of images on the diffuser and to capture an image of the diffuser through the optical wedge.

11. The interactive display device of claim 1, wherein the image sensor is a first image sensor, and further comprising a second image sensor located such that images are not captured through the display panel, wherein the first image sensor is configured to detect objects in a first, closer range of distances from the interactive surface, and wherein the second image sensor is configured to detect objects in a second, farther range of distances from the interactive surface.

12. The interactive display device of claim 1, wherein the instructions are executable to acquire images at a plurality of different diffusivities to determine a distance of an object from the interactive surface.

13. The interactive display device of claim 1, wherein the instructions are executable to acquire images at a plurality of different diffusivities to determine a three-dimensional configuration of an object positioned on or above the interactive surface.

14. The interactive display device of claim 1, wherein the instructions are executable to compare a gradient determined from deconvolution of the acquired image to expected range of gradients and assigning higher merit to gradients in the expected range.

15. The interactive display device of claim 1, wherein the instructions are executable to distinguish blur arising from hover distance from blur arising from motion based on the gradient determined from deconvolution of the acquired image.