Dynamic projected user interface

• US 8,493,366 B2
• Filed: 07/13/2011
• Issued: 07/23/2013
• Est. Priority Date: 01/25/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bathiche, Steven N., Wilson, Andrew D.
• Primary Examiner(s): Sharifi-Tafreshi, Koosha
• Application Number:
  US13/181,931
• Publication Number:
  US 20110285633A1
• Time in Patent Office: 741 Days
• Field of Search: 345156-184, 178 1801- 1809, 340/407.1, 341/13, 341/14, 341/22, 708/142, 715/773, 359/34
• US Class Current: 345/176
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A dynamic projected user interface device is disclosed, that includes a projector, a projection controller, and an imaging sensor. The projection controller is configured to receive instructions from a computing device, and to provide display images via the projector onto display surfaces. The display images are indicative of a first set of input controls when the computing device is in a first operating context, and a second set of input controls when the computing device is in a second operating context. The imaging sensor is configured to optically detect physical contacts with the one or more display surfaces.

Claims

1. A device comprising:

a projector;

a keypad comprising at least first and second keys disposed on a portion of the keypad;

a waveguide comprising an image portion having a first surface forming a first boundary and a second surface forming a second boundary, the first and second surfaces of the waveguide image portion converging along the projection path from the projector, wherein the first key has a key surface facing a first region of the waveguide image portion, and the second key has a key surface facing a second region of the waveguide image portion, and wherein the first and second surfaces of the waveguide image portion are configured to direct first rays of electromagnetic radiation emitted from the projector onto the key surface of the first key at the first region of the waveguide image portion, and the first and second surfaces of the waveguide image portion are configured to direct second rays of electromagnetic radiation emitted from the projector by internal reflection therebetween through the first region of the waveguide image portion and onto the key surface of the second key at the second region of the image portion;

a projection controller, configured to receive instructions from a computing device and to provide the first rays of electromagnetic radiation via the projector onto the first key to form a first display image thereon, and provide the second rays of electromagnetic radiation via the projector onto the second key to form a second display image thereon, wherein the first and second display images are indicative of a first set of input controls when the computing device is in a first operating context, and a second set of input controls when the computing device is in a second operating context; and

an imaging sensor, configured to optically detect physical contacts between an object and the first and second keys.

2. The device of claim 1, wherein the keypad comprises at least one of a keyboard, a numeric keypad, a game controller, and a musical keyboard.

3. The device of claim 1, wherein the waveguide is disposed between the projector and the first and second keys.

4. The device of claim 3, wherein the waveguide comprises an expansion portion with boundaries that diverge along the projection path away from the projector, wherein the projector is positioned in interface with the expansion portion and the image portion is positioned in interface with the first and second keys, such that rays emitted by the projector are internally reflected throughout the expansion portion and are transmitted from the image portion to the first and second keys.

5. The device of claim 4, wherein the waveguide comprises a waveguide nexus configured to reflect images from the projector onto respective image paths in the expansion portion, wherein the projector is positioned adjacent the waveguide nexus and configured to project the images directly to the waveguide nexus.

6. The device of claim 1, wherein the imaging sensor is disposed in connection with the waveguide and configured to detect physical contacts with the first and second keys by imaging the physical contacts through the waveguide.

7. The device of claim 1, further comprising one or more additional projectors and waveguides, such that each of the waveguides is disposed between one of the projectors and one or more keys of the keypad.

8. The device of claim 1, wherein the projector is configured to emit non-visible light, and the first and second keys comprise a composition adapted to re-emit visible light responsively to absorption of the non-visible light.

9. The device of claim 1, wherein the projector is configured to emit non-visible light, and wherein the imaging sensor is configured to obtain images from reflections of the non-visible light through the first and second keys.

10. The device of claim 1, wherein the projector comprises a spatial light modulator, and one or more laser emitters configured to project the first and second display images in a rasterized scanning beam format.

11. The device of claim 1, further configured to detect a plurality of different modes of physical contact with the first key, such that a plurality of different inputs are enabled for the first key, as a function of characteristics of the physical contact with the first key.

12. The device of claim 1, wherein the first region of the image portion is configured to reflect the second rays of electromagnetic radiation with total internal reflection and the second region of the image portion is configured to direct the second rays of electromagnetic radiation to the first surface of the image portion at an angle past a critical angle.

13. The device of claim 1, wherein the portion of the keypad is disposed adjacent the first surface of the image portion.

14. A device comprising:

a keypad, comprising one or more displaceable keys each being movable between a first, non-depressed position and a second, depressed position;

an imaging sensor configured for entering input to a computing system;

a substantially flat, tapered waveguide disposed along a path between the keypad and the imaging sensor, the imaging sensor being configured to detect physical displacement of each key of the one or more keys from the first, non-depressed position by imaging, through the waveguide, at least a portion of the key to determine a spatial position of the key; and

a projector operatively connected to an output interface of the computing system, the projector being configured for dynamically generating images as a function of output from the computing system, and projecting the dynamically generated images through the waveguide and onto the one or more keys.

15. The device of claim 14, wherein each key of the one or more keys of the keypad has a first surface facing the waveguide and a second surface opposite the first surface, and wherein the imaging sensor is configured to detect:

a first touch input comprising a physical contact with the second surface of the key when the key is in the first, non-depressed position; and

a second press input comprising physical displacement of the key to the second, depressed position, thereby enabling a plurality of different inputs for the key.

16. The device of claim 14, wherein the waveguide is disposed between the projector and keypad.

17. The device of claim 16, wherein the waveguide comprises an expansion portion with boundaries that diverge along the projection path away from the projector, wherein the projector is positioned in interface with the expansion portion and the image portion is positioned in interface with the keypad, such that rays emitted by the projector are internally reflected throughout the expansion portion and are transmitted from the image portion to the one or more keys.

18. A device comprising:

a projector;

a keypad comprising a plurality of keys disposed on at least a portion of the keypad and including at least first and second keys;

a single waveguide comprising a waveguide image portion having a first surface forming a first boundary of the waveguide image portion and a second surface that is spaced apart from the first surface and forms a second boundary of the waveguide image portion, the first and second surfaces of the waveguide image portion converging along a projection path from the projector and being configured to internally reflect rays of electromagnetic radiation, emitted from the projector, therebetween along a portion of the projection path and onto each of the first and second keys, the first and second keys being arranged in a direction of the projection path from the projector and each having a key surface facing the waveguide image portion;

a projection controller, configured to receive instructions from a computing device and to provide, via rays of electromagnetic radiation emitted by the projector through the single waveguide, a first display image on the first key and a second display image on the second key; and

an imaging sensor, configured to optically detect physical contacts between an object and the first and second keys.

19. The device of claim 18, wherein the plurality of keys are arranged on the keypad in a plurality of rows, the plurality of keys including the first key in a first row and the second key in a second row.

20. The device of claim 18, further comprising one or more additional waveguides configured to direct rays of electromagnetic radiation onto one or more keys.