Dynamic projected user interface

• US 8,022,942 B2
• Filed: 01/25/2007
• Issued: 09/20/2011
• Est. Priority Date: 01/25/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bathiche, Steven N., Wilson, Andrew D.
• Primary Examiner(s): Mengistu; Amare
• Assistant Examiner(s): Sharifi-Tafreshi; Koosha
• Application Number:
  US11/658,000
• Publication Number:
  US 20080180654A1
• Time in Patent Office: 1,699 Days
• Field of Search: 345/156, 345/173, 345/175, 345/176, 715/773, 700/84, 341/22
• US Class Current: 345/176
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A dynamic projected user interface device is disclosed, that includes a projector, a projection controller, and an imaging sensor. The projection controller is configured to receive instructions from a computing device, and to provide display images via the projector onto display surfaces. The display images are indicative of a first set of input controls when the computing device is in a first operating context, and a second set of input controls when the computing device is in a second operating context. The imaging sensor is configured to optically detect physical contacts with the one or more display surfaces.

Claims

1. A device comprising:

a projector;

a keypad comprising one or more keys;

a waveguide comprising an image portion having a first surface forming a first boundary of the image portion and a second surface spaced apart from the first surface and forming a second boundary of the image portion, wherein the first and second surfaces of the image portion converge along a projection path from the projector and are configured to reflect rays of electromagnetic radiation by internal reflection through a first portion of the image portion to a second portion of the image portion, and wherein the one or more keys have a first key surface facing the second portion of the image portion and a second key surface opposite the first key surface;

a projection controller, configured to receive instructions from a computing device and to provide one or more display images via the projector onto one or more display surfaces, the one or more display surfaces comprising the second key surface of one or more of the keys, the waveguide being disposed between the projector and the one or more display surfaces and the waveguide comprising an expansion portion with boundaries that diverge along the projection path away from the projector, the projector being positioned in interface with the expansion portion and the image portion being positioned in interface with the one or more display surfaces, such that rays emitted by the projector are internally reflected throughout the expansion portion and are transmitted from the image portion to the one or more display surfaces, the waveguide including a waveguide nexus configured to reflect images from the projector onto respective image paths in the expansion portion, the projector being positioned adjacent the waveguide nexus and configured to project the images directly to the waveguide nexus, and wherein the display images are indicative of a first set of input controls when the computing device is in a first operating context, and a second set of input controls when the computing device is in a second operating context; and

an imaging sensor, configured to optically detect physical contacts between an object and the one or more display surfaces.

2. The device of claim 1, wherein the keypad comprises at least one of a keyboard, a numeric keypad, a game controller, and a musical keyboard.

3. The device of claim 1, wherein the imaging sensor is disposed in connection with the waveguide and configured to detect physical contacts with the one or more display surfaces by imaging the physical contacts through the waveguide.

4. The device of claim 1, further comprising one or more additional projectors and waveguides, such that each of the waveguides is disposed between one of the projectors and a portion of the one or more display surfaces.

5. The device of claim 1, wherein the projector is configured to emit non-visible light, and the one or more display surfaces comprise a composition adapted to re-emit visible light responsively to absorption of the non-visible light.

6. The device of claim 1, wherein the projector is configured to emit non-visible light, and wherein the imaging sensor is configured to obtain images from reflections of the non-visible light through the one or more display surfaces.

7. The device of claim 1, wherein the projector comprises a spatial light modulator, and one or more laser emitters configured to project the display images in a rasterized scanning beam format.

8. The device of claim 1, further configured to detect a plurality of different modes of physical contact with the one or more display surfaces, such that a plurality of different inputs are enabled for a single one of the display surfaces, as a function of characteristics of the physical contact with that display surface.

10. The device of claim 1, wherein the one or more keys comprise a plurality of keys disposed on at least a portion of the keypad, the portion of the keypad being disposed adjacent the first surface of the image portion, wherein the plurality of keys are arranged on the portion of the keypad in a direction of the projection path and the first and second surfaces of the image portion are configured to direct rays of electromagnetic radiation, emitted from the projector, along at least a portion of the projection path and onto each of the plurality of keys.

11. The device of claim 1, wherein the one or more keys comprise a plurality of keys arranged on the keypad in a plurality of rows, wherein the image portion is configured to direct rays of electromagnetic radiation, emitted from the projector, along the projection path and onto a number of keys in each of the plurality of rows.

15. The device of claim 1, wherein the first portion of the image portion is configured to reflect rays of electromagnetic radiation with total internal reflection and the second portion of the image portion is configured to direct rays of electromagnetic radiation to the first surface of the image portion at an angle past a critical angle.

9. A device comprising:

a keypad, comprising one or more displaceable keys;

an imaging sensor configured for entering input to a computing system;

a substantially flat, tapered waveguide disposed between the keypad and the imaging sensor, the waveguide configured to enable the imaging sensor to detect physical displacement of the one or more keys by imaging at least a portion of the one or more keys, the keys including at least a partially transparent portion such that the imaging sensor is further configured to detect physical contact with the keys and to convey to the computing system a plurality of characteristics of the physical contact with the keys, thereby enabling a plurality of different inputs for each of the keys as a function of the characteristics of the physical contact; and

a projector operatively connected to an output interface of the computing system, the projector being configured for dynamically generating images as a function of output from the computing system, and projecting the dynamically generated images through the waveguide and onto the one or more keys.

16. The device of claim 9, wherein each of the one or more keys of the keypad have a first surface facing the waveguide and a second surface opposite the first surface, and wherein the waveguide is configured to enable the imaging sensor to detect physical contact with the second surface of the one or more keys and to detect physical displacement of the one or more keys, thereby enabling a plurality of different inputs for each of the one or more keys.

12. A device comprising:

a projector;

a keypad comprising one or more keys having a first key surface facing a waveguide and a second key surface opposite the first surface;

a projection controller configured to receive instructions from a computing device and to provide one or more display images via the projector onto one or more display surfaces, the one or more display surfaces comprising the second key surface of one or more of the keys, wherein the display images are indicative of a first set of input controls when the computing device is in a first operating context, and a second set of input controls when the computing device is in a second operating context; and

an imaging sensor configured to optically detect physical contacts between an object and the one or more display surfaces, and further configured to detect a plurality of different modes of physical contact with the one or more display surfaces, such that a plurality of different inputs are enabled for a single one of the display surfaces, as a function of characteristics of the physical contact with that display surface.

13. The device of claim 12, wherein the plurality of different modes of physical contact comprise at least one mode in which at least one of the plurality of keys is stroked in a lateral direction across the second surface of the at least one key, and at least one mode in which the at least one key is physically displaced in a direction toward the waveguide.

14. The device of claim 12, wherein the keypad comprises a plurality of keys disposed on at least a portion of the keypad, and wherein the waveguide comprises:

an image portion having a first surface forming a first boundary of the image portion and a second surface spaced apart from the first surface and forming a second boundary of the image portion, the first and second boundaries converging along a projection path away from the projector, wherein the plurality of the keys have a first key surface facing the image portion and a second key surface opposite the first surface, and wherein the image portion is configured to direct rays of electromagnetic radiation, emitted from the projector, along the projection path and onto each of the plurality of the keys.