Transparent display device

• US 9,152,173 B2
• Filed: 10/09/2012
• Issued: 10/06/2015
• Est. Priority Date: 10/09/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lee, Jinha, Boulanger, Catherine N., Bathiche, Steven Nabil
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): CHEN, FRANK S
• Application Number:
  US13/647,507
• Publication Number:
  US 20140098085A1
• Time in Patent Office: 1,092 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser

Abstract

This document describes techniques and apparatuses for implementing a transparent display device. A transparent display device includes a transparent or translucent screen to render images on the screen, and to render virtual objects that appear to be in a three-dimensional (3D) space behind the screen. The transparent display device also includes a hand tracker to sense movement of a user'"'"'s hands to interact with one or more of the virtual objects, and to generate 3D-input based on the movement. The transparent or translucent screen enables the user to see the user'"'"'s hands behind the screen as the user'"'"'s hands interact with the one or more virtual objects. The transparent display device is controlled to modify the rendering of the images on the screen or the virtual objects behind the screen based on the 3D-input.

Claims

1. A transparent display device comprising:

a transparent screen configured to render two-dimensional (2D) images on the transparent screen, and to render one or more virtual three-dimensional (3D) volumes comprising stacked windows that appear in a 3D space behind the transparent screen, the transparent screen configured to employ a grid to cause the 3D volumes to appear to be in 3D space, each window of the stacked windows selectable to launch, an associated application, file, or page, each of the one or more virtual 3D volumes initially rendered in a closed state;

a hand tracker configured to sense movement of a user'"'"'s hands and to enable a user interaction with the one or more virtual 3D volumes in the 3D space behind the transparent screen;

a processor in communication with the transparent screen and the hand tracker, the processor configured to;

first select a virtual 3D volume of the one or more virtual 3D volumes based on the movement; and

second select a window of the selected virtual 3D volume;

a controller configured to;

render, responsive to the first selection, the windows of the selected virtual 3D volume in an open state; and

transition, responsive to the second selection, the transparent screen from a 3D display mode to a 2D display mode by sliding the application, file, or page associated with the selected window in 2D on the transparent screen while simultaneously ceasing to render the one or more virtual 3D volumes to create an appearance of covering the one or more virtual 3D volumes.

2. The transparent display device of claim 1, wherein the transparent screen is configured to be transparent when rendering the one or more 3D volumes that appear in a 3D space behind the transparent screen and the transparent screen is configured to be at least partially opaque when rendering 2D images.

3. The transparent display device of claim 1, further comprising a head tracker that is configured to track a position of a user'"'"'s head or eyes relative to the transparent screen, and wherein the controller is configured to modify the rendering of the one or more virtual 3D volumes that appear to be in the 3D space behind the transparent screen based on the position of the user'"'"'s head or eyes.

4. The transparent display device of claim 1, wherein the transparent screen is a liquid crystal display (LCD).

5. A method comprising:

rendering an image on a transparent screen of a computing device;

receiving two-dimensional (2D) input from a user via an input panel positioned behind the transparent screen;

modifying the rendering of the image on the transparent screen based on the 2D-input;

sensing movement of one of the user'"'"'s hands to reach into a three-dimensional (3D) space behind the transparent screen;

generating 3D-input based on the movement of the user'"'"'s hand;

responsive to sensing the movement of the one of the user'"'"'s hands to reach into the 3D space behind the transparent screen and without further user input, transitioning the transparent screen from a 2D display mode to a 3D display mode by simultaneously ceasing to render the image on the transparent screen and revealing one or more virtual 3D volumes comprising stacked windows that appear to be in the 3D space behind the transparent screen, each window of the stacked windows representing, and selectable to launch, an associated application, file, or page, each of the one or more virtual 3D volumes initially rendered in a closed state;

sensing additional movement of the one of the user'"'"'s hands to reach towards the input panel; and

responsive to sensing the additional movement and without further user input, sliding the rendering of the image onto the transparent screen effective to transition the transparent screen from the 3D display mode to the 2D display mode.

6. A method comprising:

rendering multiple windows stacked in one or more virtual three-dimensional (3D) volumes that appear to be in a 3D space behind a transparent screen, each window of the multiple windows associated with, and selectable to launch, an associated application, file, or page, the virtual 3D volumes being rendered in a closed state;

sensing movement of a user'"'"'s hand behind the transparent screen to select one of the one or more virtual 3D volumes;

generating 3D-input based on the movement;

rendering the selected virtual 3D volume in an opened state to enable the user to interact with the multiple windows stacked in the selected 3D volume responsive to receiving the 3D-input;

sensing additional movement of the user'"'"'s hand behind the transparent screen to select a window of the selected virtual 3D volume;

generating additional 3D-input based on the additional movement; and

responsive to sensing the additional movement of the user'"'"'s hand selecting a window of the selected virtual 3D volume and without further user input, transitioning the transparent screen from a 3D display mode to a two-dimensional (2D) display mode by simultaneously;

presenting a sliding effect comprising sliding an application, file, or page associated with the selected window in 2D onto the transparent screen; and

simultaneously ceasing to present the one or more virtual 3D volumes.

7. The method of claim 6, further comprising, responsive to sensing movement of the user'"'"'s hand behind the transparent screen, rendering feedback on the transparent screen based on the location of the user'"'"'s hand in the 3D space.

8. The method of claim 7, wherein rendering the feedback further comprises causing the transparent screen to glow based on the location of the user'"'"'s hand in the 3D space.

9. The method of claim 7, wherein rendering the feedback further comprises causing display of a cursor on the transparent screen based on the location of the user'"'"'s hand in the 3D space.

10. The method of claim 6, wherein the transparent screen enables the user to see the user'"'"'s hand behind the transparent screen.

11. The method of claim 6, further comprising sensing a position of the user'"'"'s head or eyes relative to the transparent screen, and wherein the rendering the multiple windows comprises rendering the multiple windows as stacked in the one or more virtual 3D volumes that appear to be in the 3D space behind the transparent screen based on the position of the user'"'"'s head or eyes relative to the transparent screen.

12. The method of claim 6, wherein rendering the open state of the selected virtual 3D volume renders each window of the multiple windows as having an increased distance in the 3D space from each other window of the multiple windows, the increased distance being greater than a distance between each window of the multiple windows as rendered in the closed state, the increased distance effective to:

give the multiple windows an expanded appearance; and

increase exposure of two or more of the multiple windows.

13. The method of claim 12, wherein rendering the open state of the selected virtual 3D volume concurrently renders the open state of the selected virtual 3D volume and a closed state of at least one other virtual 3D volume of the one or more virtual 3D volumes.

14. The method of claim 6, wherein the movement of the user'"'"'s hand to select one of the multiple windows causes an area around each window of the multiple windows to glow progressively as the user'"'"'s hand reaches the respective window, only one window glowing at any given time to indicate a current position of the user'"'"'s hand with respect to the multiple windows.

15. The method of claim 6, wherein simultaneously presenting the sliding effect and ceasing to present the one or more 3D volumes creates an effect of covering the one or more virtual 3D volumes with the application, file, or pages associated with the selected window.