AUTO-ALIGNED ILLUMINATION FOR INTERACTIVE SENSING IN RETRO-REFLECTIVE IMAGING APPLICATIONS

• US 20160005219A1
• Filed: 07/01/2014
• Published: 01/07/2016
• Est. Priority Date: 07/01/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Powell, Karlton, Lutian, John, Large, Timothy
• Granted Patent: US 9,612,687 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 19/009 for use with infrared radiation
• CPC Class Codes: G02B 5/32 Holograms used as optical elements
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00

Abstract

Interactive sensing in retro-reflective imaging systems may be enabled through auto-alignment of an infrared (IR) illumination region with a floating image region by transmitting IR light, added on or within a display, through the same optical path as the visible floating image light. The IR illumination at the display may be either a flood of light or structured light. In some implementations, stereo vision based imaging techniques may be employed in conjunction with an auto-aligned IR floating image to provide illumination for a stereo imaging system and to provide position sensing for interaction.

Claims

1. A method comprising:

determining an optical path of light forming a floating image on a display plane;

providing infrared (IR) illumination to the display plane through the optical path to automatically align a region of the IR illumination and a region of the floating image; and

providing position sensing of the floating image through a sensor employing stereo vision to enable user interaction with the floating image in response to the sensor detecting the aligned region of the IR illumination.

2. The method of claim 1, wherein providing the IR illumination comprises:

flooding the display plane with IR light.

3. The method of claim 1, wherein providing the IR illumination comprises:

illuminating the display plane with structured light.

4. The method of claim 3, wherein the structured light is structured in one of a position space and an angle space.

5. The method of claim 4, comprising:

employing one or more IR absorbing dyes printed over an illumination layer to provide the structured light in the position space.

6. The method of claim 4, comprising:

employing an IR laser to provide the structured light in the angle space.

7. The method of claim 1, further comprising:

mitigating distortion created by the stereo vision by employing off-set imaging or rectilinear imaging.

8. The method of claim 7, wherein employing the off-set imaging comprises:

employing one of transmissive or reflective optics.

9. The method of claim 7, wherein employing the rectilinear imaging comprises:

employing a Theia lens.

10. A computing device comprising:

a retro-reflective display;

an illumination source;

one or more sensors;

a memory configured to store instructions;

a processor coupled to the memory, the retro-reflective display, the one or more sensors, and the illumination source, the processor operable to execute an interactive sensing application in conjunction with the instructions stored in the memory, wherein the interactive sensing application is configured to;

determine an optical path of light forming a floating image on a display plane;

provide infrared (IR) illumination from the illumination source to the display plane through the optical path to automatically align a region of the IR illumination and a region of the floating image; and

provide position sensing of the floating image through the one or more sensors to enable user interaction with the floating image in response to the one or more sensors detecting the aligned region of the IR illumination.

11. The computing device of claim 10, wherein the interactive sensing application is configured to enable re-convergence of the IR illumination and visible display content at a common floating image plane through the automatic alignment of the region of the IR illumination and the region of the floating image.

12. The computing device of claim 10, wherein the IR illumination is provided within a backlight unit (BLU) of the display plane.

13. The computing device of claim 10, wherein the illumination source is a backlight unit (BLU) of the display and is configured to provide the IR illumination by one of an interstitial placement of both IR light emitting diodes (LEDs) and white LEDs on a common delivery light guide or a separate guide for each of the white LEDs and the IR LEDs.

14. The computing device of claim 10, wherein the interactive sensing application is configured to correct distortion created by the stereo vision through Scheimpflug principle-based imaging.

15. The computing device of claim 10, wherein the illumination source is configured to one of:

project an IR laser beam with diffractive optical elements (DOEs) positioned at a top center of the display pane emanating an angular output;

project a divergent IR laser beam interacting with a hologram positioned at one of a beam splitter and over a display plane; and

project an IR pattern in an angle space through a holographic flat lens.

16. The computing device of claim 15, wherein the holographic flat lens is one of telecentric, convergent, or divergent.

17. The computing device of claim 10, wherein the floating image is one of a two-dimensional (2D) image or a three-dimensional (3D) image formed using a retro-reflective array and a beam splitter.

18. The computing device of claim 10, wherein the one or more sensors include two offset stereo vision cameras in conjunction with non-structured light, a single camera in conjunction with structured light, and a sensor-in-pixel (SIP) array in conjunction with unstructured light.

19. A method comprising:

projecting infrared (IR) light into a display, wherein the IR light is partially transmitted through a panel of the display, scattered, and reflected back from an active region of the panel, wherein the active region is a region of the IR light aligned with a region of a floating image; and

employing the active region of the panel to register the floating image with one or more sensors associated with the display.

20. The method of claim 19, further comprising:

employing one of a screen or a test board with a plurality of touch points to calibrate the one or more sensors.