HOVER DETECTION IN AN INTERACTIVE DISPLAY DEVICE

• US 20120127128A1
• Filed: 02/23/2011
• Published: 05/24/2012
• Est. Priority Date: 11/18/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Large, Timothy, Powell, Karlton, Bathiche, Steven
• Granted Patent: US 9,535,537 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Embodiments are disclosed that relate to hover detection in interactive display devices. One embodiment provides an interactive display device comprising a display panel configured to display an image on an interactive surface, an imaging optical wedge disposed adjacent to the display panel, an image sensor configured to capture an image of an object located in front of the interactive surface and spaced from the interactive surface by capturing the image through the imaging optical wedge, a logic subsystem, and a data-holding subsystem comprising instructions executable by the logic subsystem to operate the display panel and the image sensor, and to detect a hover input based upon one or more images received from the image sensor.

Claims

1. An interactive display device, comprising:

a display panel configured to display an image on an interactive surface;

an imaging optical wedge disposed adjacent to the display panel;

an image sensor configured to capture an image of an object located in front of the interactive surface and spaced from the interactive surface by capturing the image through the imaging optical wedge;

a logic subsystem; and

a data-holding subsystem comprising instructions executable by the logic subsystem to receive one or more images from the image sensor, to detect a hover input based upon the one or more images received from the image sensor, and to display a response to the hover input via the display panel.

2. The interactive display device of claim 1, wherein the image sensor comprises a three-dimensional image sensor.

3. The interactive display device of claim 2, wherein the three-dimensional image sensor comprises a time-of-flight depth sensor.

4. The interactive display device of claim 2, wherein the three-dimensional image sensor comprises a stereo camera system comprising two or more image sensors.

5. The interactive display device of claim 4, further comprising a variable diffuser, and wherein the instructions are executable to acquire images with a first image sensor and a second image sensor of the stereo camera system at different diffusivities.

6. The interactive display device of claim 4, wherein the instructions are executable to acquire images with a first image sensor and a second image sensor of the stereo camera system at a same diffusivity.

7. The interactive display device of claim 2, wherein the three-dimensional image sensor comprises a structured light depth sensor.

8. The interactive display device of claim 7, wherein the structured light depth sensor is configured to project a structured light pattern through the imaging optical wedge.

9. The interactive display device of claim 7, wherein the controller is configured to project a structured light pattern via the display panel.

10. The interactive display device of claim 1, further comprising a diffuser positioned between the optical wedge and the display panel, wherein the display panel is a modulating display panel, and wherein the instructions are executable to form a plurality of apertures via the display panel to focus a plurality of images on the diffuser and to capture an image of the diffuser through the optical wedge.

11. The interactive display device of claim 1, wherein the image sensor is a first image sensor, and further comprising a second image sensor located such that images are not captured through the display panel, wherein the first image sensor is configured to detect objects in a first, closer range of distances from the interactive surface, and wherein the second image sensor is configured to detect objects in a second, farther range of distances from the interactive surface.

12. An interactive display device, comprising:

a sensor-in-pixel display system, comprising

an image-producing element,

a large-format image sensor positioned behind the image-producing element relative to an interactive surface of the interactive display device, and

a lens array configured to focus different perspectives of an object on different portions of the large-format image sensor;

a logic subsystem; and

a data-holding subsystem comprising instructions executable by the logic subsystem to operate the image-producing element and the large-format sensor, to acquire an image from the large-format image sensor, to construct a stereo image of an object from two or more sub-areas of the image acquired from the large-format image sensor, to detect a hover input via the stereo image, and to display a response to the hover input via the image-producing element.

13. The interactive display device of claim 12, wherein the lens array is spaced from the image-producing element by a gap.

14. The interactive display device of claim 12, wherein the lenslet array is infrared-reflective and visibly transmissive.

15. An interactive display device, comprising:

a display panel configured to display an image on an interactive surface;

a first sensor configured to detect touch inputs ;

a second sensor configured to capture an image of an object located in front of the interactive surface and spaced from the interactive surface; and

a computing device comprising a logic subsystem and a data-holding subsystem comprising instructions stored thereon that are executable by the logic subsystem to

operate the display panel, the first sensor, and the image sensor,

detect one or more of a touch input and a hover input within a first range of distances from the display panel via the first sensor,

display a response via the display panel to the one or more of the touch input and the hover input within the first range of distances;

detect a hover input within a second range of distances from the display panel via an image received from the second sensor, and

display a response via the display panel to the hover input within the second range of distances.

16. The interactive display device of claim 15, wherein the first sensor is a large-format image sensor and forms, with the display panel, a sensor-in-pixel display panel.

17. The interactive display device of claim 16, wherein the second sensor comprises a 3D image sensor positioned adjacent to the display panel such that images are not captured through the sensor-in-pixel display panel.

18. The interactive display device of claim 16, further comprising an imaging optical wedge, wherein the second sensor is a 2D or 3D image sensor configured to capture an image through the imaging optical wedge and the display panel.

19. The interactive display device of claim 15, wherein the first sensor comprises a capacitive touch sensor and/or a resistive touch sensor.

20. The interactive display device of claim 15, further comprising an imaging optical wedge, and wherein the first sensor is an image sensor configured to capture an image through the display panel and the optical wedge.