Display interaction detection

• US 10,101,817 B2
• Filed: 03/03/2015
• Issued: 10/16/2018
• Est. Priority Date: 03/03/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Hsin, Chih-Fan, Graumann, David L.
• Primary Examiner(s): Holder, Anner N
• Application Number:
  US14/636,247
• Publication Number:
  US 20160259418A1
• Time in Patent Office: 1,323 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Techniques for interaction detection are described herein. The techniques may include detecting, via a depth sensing module, a physical object at a plane of a display. The techniques may also include detecting, via the depth sensing module, a reflection of the physical object at the plane of the display. An interaction module may identify an interaction of the physical object with the display based on a meeting point of the detected physical object and the detected reflection.

Claims

1. A system, comprising one or more modules at least partially comprising hardware logic, comprising:

a depth sensing module to;

detect a physical object at a plane of a display;

detect a reflection of the physical object in the plane of the display

an interaction module to identify an interaction of the physical object with the display based on a meeting point of the physical object and the reflection of the physical object in the plane of the display.

2. The system of claim 1, wherein the depth sensing module comprises an infrared emitter configured to:

illuminate the physical object and the reflection of the physical object; and

detect a location of the interaction with respect to the infrared emitter.

3. The system of claim 2, further comprising a visible light sensor to enhance the identification of the interaction by detecting the physical object and the reflection of the physical object in a light spectrum that is different from a light spectrum capability of the infrared emitter.

4. The system of claim 3, wherein the visible light sensor is a first visible light sensor, further comprising a second visible light sensor to enhance location detection of the interaction by triangulation of image data captured by the first visible light sensor and image data captured by the second visible light sensor.

5. The system of claim 4, wherein the first visible light sensor and the second visible light sensor are configured to be disposed at different locations at an edge of the display.

6. The system of claim 4, wherein the first and second visible light sensors are configured to reduce a potential error in identifying the interaction caused by an occlusion of the physical object by another physical object.

7. The system of claim 4, further comprising a calibration disruption module configured to detect movement of the infrared emitter, the first visible light sensor, the second visible light sensor, or any combination thereof.

8. The system of claim 4, wherein the display is configured to emit light in a direction of propagation, and wherein the depth sensor, the first visible light sensor, the second visible light sensor, or any combination thereof, are each physically attached to the display and configured to face the direction of propagation of light emitted from the display.

9. The system of claim 8, further comprising a reflective component configured to:

direct at least a portion of the projection of the depth sensor;

direct at least a portion of a field of view of the first visible light sensor;

direct at least a portion of a field of view of the second visible light sensor;

or

any combination thereof.

10. The system of claim 1, wherein the depth sensing module is configured to be disposed on an edge of the display.

26. The system of claim 1, comprising a reflective component to direct projection and a field of view, wherein the reflective component is a transition metal switchable mirror physically attached to the display.

11. A method, comprising:

detecting, via a depth sensing module, a physical object at a plane of a display;

detecting, via the depth sensing module, a reflection of the physical object in the plane of the display;

identifying, via an interaction module, an interaction of the physical object with the display based on a meeting point of the physical object and the reflection of the physical object in the plane of the display.

12. The method of claim 11, the method further comprising:

illuminating the physical object and the reflection of the physical object via an infrared emitter; and

detecting a location of the interaction with respect to the infrared emitter.

13. The method of claim 12, further comprising enhancing, via a visible light sensor, the identification of the interaction by detecting the physical object and the reflection of the physical object in a light spectrum that is different from a light spectrum capability of the infrared emitter.

14. The method of claim 13, wherein the visible light sensor is a first visible light sensor, further comprising enhancing, via a second visible light sensor, location detection of the interaction by triangulation of image data captured by the first visible light sensor and image data captured by the second visible light sensor.

15. The method of claim 14, wherein the first visible light sensor and the second visible light sensor are configured to be disposed at different locations at an edge of the display.

16. The method of claim 14, further comprising reducing, via the first and second visible light sensors, a potential error in identifying the interaction caused by an occlusion of the physical object by another physical object.

17. The method of claim 14, further comprising detecting, via a calibration disruption module, movement of the infrared emitter, the first visible light sensor, the second visible light sensor, or any combination thereof.

18. The method of claim 14, wherein the display is configured to emit light in a direction of propagation, and wherein the depth sensor, the first visible light sensor, the second visible light sensor, or any combination thereof, are each physically attached to the display and configured to face the direction of propagation of light emitted from the display.

19. The method of claim 18, further comprising:

directing at least a portion of the projection of the depth sensor via a reflective component;

directing at least a portion of a field of view of the first visible light sensor via the reflective component;

directing at least a portion of a field of view of the second visible light sensor via the reflective component;

or

any combination thereof.

20. The method of claim 11, further comprising disposing the depth sensing module on an edge of the display.

21. A tangible, non-transitory, computer-readable medium comprising instructions that, in response to execution by a processor, direct the processor to:

detect a physical object at a plane of a display;

detect a reflection of the physical object in the plane of the display; and

identify an interaction of the physical object with the display based on a meeting point of the physical object and the reflection of the physical object in the plane of the display.

22. The computer-readable medium of claim 21, comprising instructions that, in response to execution by a processor, direct the processor to:

illuminate the physical object and the reflection of the physical object via an infrared emitter; and

detect a location of the interaction with respect to the infrared emitter.

23. The computer-readable medium of claim 22, comprising instructions that, in response to execution by a processor, direct the processor to:

enhance, via a first visible light sensor, the identification of the interaction by detecting the physical object and the reflection of the physical object in a light spectrum that is different from a light spectrum capability of the infrared emitter, wherein the visible light sensor is a first visible light sensor;

enhance, via a second visible light sensor, location detection of the interaction by triangulation of image data captured by the first visible light sensor and image data captured by the second visible light sensor; and

reduce, via the first and second visible light sensors, a potential error in identifying the interaction caused by an occlusion of the physical object by another physical object.

24. The computer-readable medium of claim 23, comprising instructions that, in response to execution by a processor, direct the processor to detect, via a calibration disruption module, movement of the infrared emitter, the first visible light sensor, the second visible light sensor, or any combination thereof.

25. The computer-readable medium of claim 21, comprising instructions that, in response to execution by a processor, direct the processor to:

direct at least a portion of the projection of the depth sensor via a reflective component;

direct at least a portion of a field of view of the first visible light sensor via the reflective component;

direct at least a portion of a field of view of the second visible light sensor via the reflective component;

or

any combination thereof.