User Interface Functionality for Facilitating Interaction between Users and their Environments

• US 20160124588A1
• Filed: 04/19/2015
• Published: 05/05/2016
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Cotier, Bradley Neville, Doroftei, Catalin Nicolae, Chudge, Jarnail Singh, Farmer, Matthew Robin, Parker, Michael James Belfield, Wilcox, Robert John
• Granted Patent: US 10,048,835 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3635 Guidance using 3D or perspective road maps
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G01S 5/0295 Proximity-based methods, e.g. position inferred from reception of particular signals
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/452 : Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06Q 30/0205 : Location or geographical consideration
• CPC Class Codes: G08G 1/00 : Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D
• CPC Class Codes: G08G 1/005 : including pedestrian guidance indicator
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/0968 : Systems involving transmission of navigation instructions to the vehicle
• CPC Class Codes: G09B 21/006 : using audible presentation of the information
• CPC Class Codes: H04R 1/1041 : Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 27/00 : Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 5/033 : Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04S 2400/11 : Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 : Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/304 : For headphones
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/021 : Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences

Abstract

Space interaction (SI) functionality is described herein for assisting a user in interacting with a space without unduly distracting the user. The SI functionality includes an application interface module that presents information and/or exposes functionality within a plurality of workspaces. Each workspace has a determined spatial relationship with respect to other workspaces. Further, the application interface module may detect and respond to various gestures, by which the user may move among workspaces and interact with menus and other information that are presented in those workspaces. The user'"'"'s interaction with these workspaces and menus may be supplemented by various sounds generated by a sound generation module, and/or various haptic cues (e.g., vibration cues) generated by a haptic cue generation module.

Claims

1. A computing device, implementing a space interaction module, for assisting a user in interacting with a space, comprising:

a touch-sensitive screen for presenting a user interface presentation and for receiving input from the user;

an application interface module that is configured to present information on the touch-sensitive screen within a plurality of workspaces,

each workspace having a predetermined spatial relationship with respect to other workspaces, and each workspace presenting a respective focus by which the user may interact with the space,

the application interface module being further configured to;

detect a gesture that involves contact by the user with a touch location on the touch-sensitive screen; and

display a menu on the touch-sensitive screen at the touch location, the menu having a plurality of menu items,

the touch location corresponding to one of a plurality possible touch locations at which the user can touch the surface to activate the menu.

2. The computing device of claim 1, wherein the application interface module is further configured to:

detect another gesture performed by the user, corresponding to an instruction to advance from a first workspace to a second workspace; and

advance from the first workspace to the second workspace in response to the gesture.

3. The computing device of claim 2,

wherein a path through the space, over which the user moves, is characterized by a series of contexts,

wherein each context pertains to one or more environment-related features which are relevant to the user at a particular time, at a particular juncture along the path, and

wherein a current context corresponds to one or more environment-related features which affect the user at a current time, at a current juncture along the path.

4. The computing device of claim 3,

wherein a home workspace serves as a principal zone that allows a user to interact with information and functionality associated with the current context, and

wherein the home workspace corresponds to either the first workspace or the second workspace.

5. The computing device of claim 4,

wherein another workspace provides a portal to notification information that is relevant to the user in traversing the path,

wherein another workspace provides a main portal to different functions that are accessible to the user,

wherein another workspace provides a portal to mechanisms by which the user may set parameter values that affect operation of the space interaction module, and

wherein another workspace provides information regarding items of interest that are associated with the current context.

6. The computing device of claim 4, wherein other workspaces, besides the home workspace, correspond to neighboring workspaces with respect to different respective sides of the home workspace.

7. The computing device of claim 3,

wherein the application interface module is further configured to receive another gesture, and

wherein the application interface module is further configured to deliver audio information that describes the current context, in response to detection of the other gesture.

8. The computing device of claim 1, wherein, for a given menu, a first subset of the menu items pertain to the current context, and a second subset of the menu items pertain to two or more contexts and/or two or more workspaces.

9. The computing device of claim 1,

wherein the application interface module is further configured to receive another gesture, and

wherein the application interface module is further configured to navigate over menu items in the menu, in response to detection of the other gesture.

10. The computing device of claim 9, wherein the space interaction module includes a sound generation module that is configured to present sounds of progressively increasing or decreasing tones as the user navigates over the menu items, starting from an initial menu item.

11. The computing device of claim 9, wherein the space interaction module includes a haptic cue generation module that is configured to present a haptic cue each time the user moves to a menu item in the menu.

12. A method, implemented by at least one computing device, for assisting a user in interacting with a space, comprising:

detecting a first gesture, performed by the user, corresponding to an instruction to activate a menu in an associated workspace,

the associated workspace corresponding to one of a plurality of workspaces, and the associated workspace having a predetermined spatial relationship with respect to other workspaces in the plurality of workspaces, and each workspace presenting a respective focus by which the user may interact with the space,

activating a menu in response to the first gesture;

detecting a second gesture, performed by a user, corresponding to an instruction to advance to a particular menu item, among a collection of menu items in the menu;

advancing to the particular menu item in response to the second gesture;

detecting a third gesture, performed by a user, corresponding to an instruction to select the particular menu item; and

performing an operation in response to the third gesture.

13. The method of claim 12, wherein the first gesture, the second gesture, and the third gesture correspond to gestures performed by the user with a single hand.

14. The method of claim 12,

wherein the first gesture involves contact by the user with a touch location on a touch-sensitive surface of a touch-sensitive screen, and

wherein said activating the menu involves displaying the menu on the touch-sensitive screen, centered at the touch location,

the touch location corresponding to one of a plurality possible touch locations at which the user can touch the surface to activate the menu.

15. The method of claim 12, wherein the third gesture involves removing contact by the user with a touch location on a touch-sensitive surface of the touch-sensitive screen.

16. The method of claim 12,

wherein a path through the space, over which the user moves, is characterized by a series of contexts,

wherein each context pertains to one or more environment-related features which are relevant to the user at a particular time, at a particular juncture along the path,

wherein a current context corresponds to one or more environment-related features which affect the user at a current time, at a current juncture along the path,

wherein a home workspace serves as a principal zone that allows the user to interact with information and functionality associated with the current context,

wherein another workspace provides a portal to notification information that is relevant to the user in traversing the path,

wherein another workspace provides a main portal to different functions that are accessible to the user,

wherein another workspace provides a portal to mechanisms by which the user may set parameter values that affect interaction with the space, and

wherein another workspace provides information regarding items of interest that are associated with the current context.

17. The method of claim 12, further comprising presenting one or more sounds in response to each of the first gesture, the second gesture, and the third gesture.

18. The method of claim 12, further comprising providing one or more haptic cues in response to each of the first gesture, the second gesture, and the third gesture.

19. A computer readable storage medium for storing computer readable instructions, the computer readable instructions implementing an application interface module when executed by one or more processing devices, the computer readable instructions comprising:

logic configured to provide a user interface presentation that presents a menu, having a plurality of menu items, within at least workspace;

logic configured to navigate over menu items in the menu, in response to detection of a corresponding gesture made by a user;

logic configured to generate sounds of progressively increasing or decreasing tones as the user advances through the menu items, starting from an initial menu item; and

logic configured to generate a haptic cue each time the user moves to a menu item in the menu.

20. The computer readable storage medium of claim 19, further including logic for presenting a spoken description associated with each menu item that is traversed.