Wearable display and method of controlling the wearable display generating a user interface according to that of an external device

• US 9,804,686 B2
• Filed: 06/25/2013
• Issued: 10/31/2017
• Est. Priority Date: 11/20/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Park, Hyorim
• Primary Examiner(s): Spar, Ilana
• Assistant Examiner(s): HERMANN, KIRK W
• Application Number:
  US13/926,648
• Publication Number:
  US 20140139439A1
• Time in Patent Office: 1,589 Days
• Field of Search: None
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/42 using separate channels for security data
• CPC Class Codes: G06F 21/82 Protecting input, output or interconnection devices
• CPC Class Codes: G06F 2221/032 Protect output to user by software means
• CPC Class Codes: G06F 3/0227 Cooperation and interconnection of the input arrangement with other functional units of a computer G06F3/023 - G06F3/037 take precedence
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A Head Mount Display (HMD) and a method for controlling the HMD are disclosed. The method includes detecting an external device in an image captured by the HMD, recognizing at least one component from a first User Interface (UI) of the external device, the at least one component including a keypad or a window for outputting data corresponding to an input signal to the keypad, generating a second UI including the recognized at least one component, displaying the second UI in the HMD, wherein the at least one component of the first UI being displayed on the external device is overlaid with the displayed second UI using the image from which the external device is detected, and displaying data corresponding to an input signal currently received at the external device on the second UI in the HMD.

Claims

1. A Head Mount Display (HIVID) comprising:

a processor configured to control an operation of the HIVID;

a display unit;

a communication unit configured to communicate with an external device; and

a sensor unit configured to capture an image of the external device and provide the image to the processor,

wherein the processor is further configured to;

detect the external device from the image,

recognize a first keyboard component and a first window component from the external device, wherein the first window component of the external device outputs hidden data corresponding to an input signal currently received at the external device via the first keyboard component having a first keyboard layout,

switch a normal mode of the external device to a protection mode of the external device if the first keyboard component and the first window component from the external device are recognized, wherein the protection mode causes the display of a protection layer over one or more displayed components of the external device,

generate a second keyboard component corresponding to the first keyboard component and a second window component corresponding to the first window component, wherein the second window component outputs data corresponding to the input signal without hiding the data,

provide the second keyboard component overlaying the first keyboard component, and

indicate a key included in the second keyboard component corresponding to the input signal currently received via the first keyboard component,

wherein the second keyboard component displayed through the display unit of the HMD has a second keyboard layout different from the first keyboard layout of the first keyboard component displayed through the external device, and the input signal is received in accordance with the second keyboard layout of the second keyboard component, and

wherein each of the first keyboard layout and the second keyboard layout includes a plurality of keys.

2. The HMD of claim 1, wherein the first window component is overlaid with the second window component.

3. The HMD of claim 1, wherein the protection layer is displayed over a screen of the external device.

4. The HMD of claim 3, wherein the protection layer includes a layer which makes contents displayed in the external device unseeable to a user.

5. The HMD of claim 3, wherein an interface for adjusting a display property of the protection layer is provided.

6. The HMD of claim 5, wherein the display property includes at least one of brightness, chroma, and transparency.

7. The HMD of claim 1, wherein the protection layer is displayed over the first window component of the external device.

8. The HMD of claim 1, wherein the protection layer is displayed over the first keyboard component of the external device.

9. The HMD of claim 8, wherein the protection layer includes an outline of the first keyboard component of the external device.

10. The HMD of claim 1, wherein a tactile feedback is provided via the first keyboard component, if a user'"'"'s touch input to the first keyboard component is detected.

11. The HMD of claim 1, wherein the processor recognizes the first keyboard component and the first window component from the external device when the external device is in the protection mode.

12. The HMD of claim 1, wherein the normal mode of the external device is switched to the protection mode of the external device, if the processor detects a user is wearing the HMD.

13. The HMD of claim 1, wherein indicating the key included in the second keyboard component corresponding to the input signal currently received via the first keyboard component comprises enlarging the key.

14. A method for controlling a Head Mount Display (HMD), the method comprising:

detecting an external device from an image captured by the HMD,

recognizing a first keyboard component and a first window component from the external device, wherein the first window component of the external device outputs hidden data corresponding to an input signal currently received at the external device via the first keyboard component having a first keyboard layout,

switching the external device from a normal mode to a protection mode if the processor recognizes the first keyboard component and the first window component from the external device,

displaying a protection layer over one or more components of the external device if the external device is in the protection mode,

generating a second keyboard component corresponding to the first keyboard component and a second window component corresponding to the first window component, wherein the second window component outputs data corresponding to the input signal without hiding data,

providing the second keyboard component overlaying the first keyboard component, and

indicating a key included in the second keyboard component corresponding to the input signal currently received via the first keyboard component,

wherein the second keyboard component displayed corresponding to the first keyboard component has a second keyboard layout different from the first keyboard layout of the first keyboard component and the input signal is received by using the second keyboard layout and in accordance with the second keyboard layout, and

wherein each of the first keyboard layout and the second keyboard layout includes a plurality of keys.

15. The method of claim 14, further comprising:

providing the second window component overlaying the first window component.

16. The method of claim 14, further comprising:

wherein the protection layer includes a layer which makes contents displayed in the external device unseeable to a user.

17. The method of claim 14, further comprising:

displaying the protection layer over the first window component of the external device.

18. The method of claim 17, wherein the protection layer includes an outline of the first keyboard component of the external device and a tactile feedback is provided via the first keyboard component, if a user'"'"'s touch input to the first keyboard component is detected.

19. The method of claim 14, further comprising:

switching the external device from the normal mode to the protection mode if a user is detected wearing the HMD.

20. A Head Mount Display (HMD) for use with an external device, the HIVID comprising:

an at least partially see-through display;

a sensor configured to capture an image;

a processor configured to;

detect the external device in the image;

recognize a first keyboard component including a plurality of keys in a first keyboard layout and a first window component of a first user interface of the external device in the image, wherein the first window component of the external device displays in a hidden format current input data that is input via an input signal to the first keyboard component;

when the first keyboard component and the first window component from the external device are recognized, send an instruction to the external device via a communication unit of the HIVID to cause the external device to switch from a normal mode to a protection mode, wherein the protection mode causes the display of a protection layer over one or more of the first keyboard component and the first window component;

generate a second user interface for display on the at least partially see- through display of the HMD, the second user interface including a second keyboard component including a plurality of keys of a second keyboard layout different than the first keyboard layout, and being displayed at a position corresponding to a position of the first keyboard component so as to overlay the first keyboard component as viewed through the at least partially see-through display, and a second window component displayed at a position corresponding to a position of the first window component so as to at least partially overlay the first window component as viewed through the at least partially see-through display, wherein the second window component displays the current input data in an unhidden format, and wherein the second keyboard component displays a graphical indication of selection of a selected key of the plurality of keys according to the input signal, a value of the selected key being determined according to the second keyboard layout.