Multiple stage shy user interface

• US 10,108,320 B2
• Filed: 10/08/2014
• Issued: 10/23/2018
• Est. Priority Date: 10/08/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Klein, Christian
• Primary Examiner(s): Augustine, Nicholas
• Application Number:
  US14/509,270
• Publication Number:
  US 20160103584A1
• Time in Patent Office: 1,476 Days
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Human device interactions are improved by revealing user interface (UI) elements or functionality on a device in multiple context-dependent stages based on discerning a user'"'"'s intent. Three dimensional sensors that detect the presence, position, orientation, direction, or rate of travel of a pointer with respect to a device may provide indicia of the user'"'"'s intent to interact with a UI element on the device. If the user was not actively navigating on the device, then context relevant UI elements or actions that were not previously displayed may be surfaced and displayed elements may be removed. If the user then maintains or narrows their focus on or near a particular UI element that was just revealed, a second or subsequent layer of UI elements or effects may be surfaced. Subsequent deeper layers in a multi-stage shy UI hierarchy may be presented as the user interacts with the surfaced elements.

Claims

1. A method for presenting multi-stage shy user interface (UI) elements on a display on a device, comprising:

receiving first electronic indicia of a user'"'"'s intent to interact with a current context associated with the device;

selectively displaying on the display a first element that is not currently displayed on the display, where the first element is associated with a first level of a multi-stage shy UI, and where the first element is selected based, at least in part, on the first electronic indicia and a user interaction context;

receiving second electronic indicia of the user'"'"'s intent to interact with a second, different level of the multi-stage shy UI;

selectively displaying on the display a second element that is not currently displayed on the display, where the second element is associated with a portion of the second level, where the second element is selected based, at least in part, on the second electronic indicia and the context, and where the portion of the second level depends, at least in part, on the first element;

receiving third electronic indicia of the user'"'"'s intent to apply an effect to the first element, the second element, or to an object displayed on the display; and

selectively applying the effect to the first element, to the second element, or to the object, where the effect is based, at least in part, on the third electronic indicia and the user interaction context, and wherein the effect is a data reveal effect that reveals a portion of a complex data set, wherein the amount of data from the complex data set that is displayed is controlled by a distance of a pointer, in three-dimensional hover-space above the display, from the display.

2. The method of claim 1, the first electronic indicia comprising a hover event generated in response to the pointer approaching the display or the pointer pointing to a pre-defined portion of the display.

3. The method of claim 2, the first electronic indicia comprising a touch event generated in response to the pointer touching the display, a gesture event generated in response to the pointer making a touch or hover gesture associated with the display, a tactile event generated in response to the user moving the device, or a speech event generated in response to the user verbally interacting with the device.

4. The method of claim 3, the second electronic indicia comprising a hover event generated in response to the pointer approaching the display, in response to the pointer maintaining a relationship with the first element, in response to the pointer pointing to a pre-defined location on the display, or in response to the pointer establishing a relationship with an object displayed on the display.

5. The method of claim 4, the second electronic indicia comprising a touch event generated in response to the pointer touching the display, a gesture event generated in response to the pointer making a touch or hover gesture associated with the display, a tactile event generated in response to the user moving the device, or a speech event generated in response to the user verbally interacting with the device.

6. The method of claim 3, wherein the third electronic indicia comprises a hover event generated in response to the pointer approaching the display, in response to the pointer maintaining a relationship with the first element, in response to the pointer pointing to a pre-defined location on the display, or in response to the pointer establishing a relationship with one or more objects displayed on the display.

7. The method of claim 6, the third electronic indicia comprising a touch event generated in response to the pointer touching the display, a gesture event generated in response to the pointer making a touch or hover gesture associated with the display, a tactile event generated in response to the user moving the device, or a speech event generated in response to the user verbally interacting with the device.

8. The method of claim 1, where the user interaction context is determined, at least in part, by a set of objects currently displayed on the display, and where the user interaction context concerns a task being performed on the device by the user.

9. The method of claim 1, where the user interaction context is determined, at least in part, by a relationship between the pointer and an object displayed on the display, and where the user interaction context concerns a task being performed on the device as controlled, at least in part, by the user.

10. The method of claim 6, where the effect is a light source position effect, a zoom effect, a lens effect, a scroll effect, an animation effect, a preview operation, or a reveal next level of multi-stage shy UI effect.

11. The method of claim 6, where the effect is determined, at least in part, by a position of the pointer, a three dimensional position of the pointer, an angle of the pointer, an orientation of the pointer, a movement of the pointer, or a velocity of the pointer.

12. The method of claim 1, where the portion is selected based on a position of the pointer with respect to an object that represents the complex data set, an angle of the pointer with respect to an object that represents the complex data set, an orientation of the pointer with respect to an object that represents the complex data set, a movement of the pointer with respect to an object that represents the complex data set, or a velocity of the pointer with respect to an object that represents the complex data set.

13. The method of claim 1, comprising:

upon receiving subsequent electronic indicia of the user'"'"'s intent to interact with a subsequent level of the multi-stage shy UI, selectively displaying on the display a subsequent element that is not currently displayed on the display, where the subsequent element is associated with a portion of a subsequent level of the multi-stage shy UI, where the portion depends on a previous element displayed on the display, and where the subsequent element is selected based on the subsequent electronic indicia, the previous element, and the user interaction context.

14. An apparatus, comprising:

a processor;

a touch or hover sensitive display;

a memory;

a set of logics that manage a multiple stage shy user interface (MSSUI), and

a physical interface to connect the processor, the memory, the display, and the set of logics,

the set of logics comprising;

a first logic that handles touch or hover events associated with the display;

a second logic that selectively displays on the display a subsequent element from the MSSUI, where the subsequent element is selected based on the touch or hover events and on a previous element already displayed on the display,

a third logic that manages a navigation point in the MSSUI based, at least in part, on the touch or hover events, the previous element, and the subsequent element, and

a fourth logic that selectively applies an effect to the subsequent element, wherein the effect is a data reveal effect that reveals a portion of a complex data set, wherein the amount of data from the complex data set that is displayed is controlled by a distance of a pointer, in three-dimensional hover-space above the display, from the display.

15. The apparatus of claim 14, where the first logic handles events including a single touch event, a multi-touch event, a single hover event, a multi-hover event, a touch gesture event, a hover gesture event, or a combined touch and hover event.

16. The apparatus of claim 15, where the second logic selects the subsequent element based, at least in part, on an angle of an item that produced the touch or hover event, a distance from the display of the item that produced the touch or hover event, a movement relative to the display of the item that produced the touch or hover event, an orientation with respect to the display of the item that produced the touch or hover event, a length of time during which the item that produced the touch or hover event remained in a position, or a speed at which the item that produced the touch or hover event moved with respect to the display.

17. The apparatus of claim 16, where the second logic selectively displays elements on the display and selectively removes elements from the display so that less than ten percent of the display is occupied by user interface elements and so that at least twenty five percent of the relevant functionality available through a current level of the multi-stage shy user interface is displayed.

18. The apparatus of claim 17, where the effect is determined by a position of the pointer relative to the subsequent element, an angle of the pointer with respect to the subsequent element, an orientation of the pointer with respect to the subsequent element, a movement of the pointer relative to the subsequent element, or a velocity of the pointer relative to the subsequent element.

19. The apparatus of claim 18, where the effect is a light source position effect, a zoom effect, a lens effect, a scroll effect, an animation effect, a preview operation, a reveal next level of multi-stage shy UI effect, or a data reveal effect that reveals a portion of a complex data set.

20. A system, comprising:

a display on which an element from a multi-stage shy user interface is displayed;

a proximity detector that detects actions in a space associated with the display, and

a user interface circuit that controls the presentation and appearance of user interface elements from the multi-stage shy user interface in response to the detected actions so that less than five percent of the display is occupied by user interface elements and so that at least ten percent of the functionality available through the multi-stage shy user interface is displayed, and that selectively applies an effect to a subsequent element, wherein the effect is a data reveal effect that reveals a portion of a complex data set, wherein the amount of data from the complex data set that is displayed is controlled by a distance of a pointer, in three-dimensional hover-space above the display, from the display.