User interface functionality for facilitating interaction between users and their environments

• US 10,048,835 B2
• Filed: 04/19/2015
• Issued: 08/14/2018
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Cotier, Bradley Neville, Doroftei, Catalin Nicolae, Chudge, Jarnail Singh, Farmer, Matthew Robin, Parker, Michael James Belfield, Wilcox, Robert John
• Primary Examiner(s): Choi, David
• Application Number:
  US14/690,420
• Publication Number:
  US 20160124588A1
• Time in Patent Office: 1,213 Days
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3635 Guidance using 3D or perspective road maps
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G01S 5/0295 Proximity-based methods, e.g. position inferred from reception of particular signals
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/452 : Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06Q 30/0205 : Location or geographical consideration
• CPC Class Codes: G08G 1/00 : Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D
• CPC Class Codes: G08G 1/005 : including pedestrian guidance indicator
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/0968 : Systems involving transmission of navigation instructions to the vehicle
• CPC Class Codes: G09B 21/006 : using audible presentation of the information
• CPC Class Codes: H04R 1/1041 : Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 27/00 : Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 5/033 : Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04S 2400/11 : Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 : Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/304 : For headphones
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/021 : Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences

Abstract

Space interaction (SI) functionality is described herein for assisting a user in interacting with a space without unduly distracting the user. The SI functionality includes an application interface module that presents information and/or exposes functionality within a plurality of workspaces. Each workspace has a determined spatial relationship with respect to other workspaces. Further, the application interface module may detect and respond to various gestures, by which the user may move among workspaces and interact with menus and other information that are presented in those workspaces. The user'"'"'s interaction with these workspaces and menus may be supplemented by various sounds generated by a sound generation module, and/or various haptic cues (e.g., vibration cues) generated by a haptic cue generation module.

Claims

1. A computing device comprising:

a touch-sensitive screen;

a processing device; and

a storage resource having computer-readable instructions which, when executed by the processing device, cause the processing device to;

obtain different location contexts for different locations of a user as the user moves through a physical space;

detect a touch gesture performed on the touch-sensitive screen;

responsive to the touch gesture performed on the touch-sensitive screen, navigate among a plurality of workspaces available for presentation on the touch-sensitive screen to a first workspace, the plurality of workspaces having predetermined spatial relationships with respect to other workspaces and respective functionality by which the user may interact with the physical space;

after the user has navigated to the first workspace, detect a tap on the touch-sensitive screen and, in response to the tap, output a spoken message conveying information regarding a current location context of the user;

detect another touch gesture performed on the touch-sensitive screen; and

responsive to the another touch gesture, display a menu associated with the first workspace on the touch-sensitive screen, the menu having a plurality of menu items relating to the current location context of the user in the physical space.

2. The computing device of claim 1, wherein the computer-readable instructions, when executed by the processing device, cause the processing device to:

advance from a previous workspace to the first workspace based at least in part on a specific direction of the touch gesture.

3. The computing device of claim 2, the touch gesture comprising a flick gesture in the specific direction.

4. The computing device of claim 3,

wherein the first workspace is a home workspace that serves as a principal zone that allows a user to interact with information and functionality associated with the current location context.

5. The computing device of claim 4, the plurality of workspaces including:

a second workspace that provides a portal to notification information that is relevant to the user in traversing a path in the physical space,

a third workspace that provides a main portal to different functions that are accessible to the user,

a fourth workspace that provides a portal to mechanisms by which the user may set parameter values that affect operation of the computing device, and

a fifth workspace that provides information regarding items of interest that are associated with the current location context.

6. The computing device of claim 5, wherein the second, third, fourth, and fifth workspaces correspond to neighboring workspaces with respect to different respective sides of the home workspace.

7. The computing device of claim 1, wherein the computer-readable instructions, when executed by the processing device, cause the processing device to:

detect that a touch contact providing the another touch gesture has moved to a particular menu item and then released from the touch-sensitive screen; and

responsive to detecting that the touch contact has released from the touch-sensitive screen, select the particular menu item from the menu.

8. The computing device of claim 1, wherein the menu comprises a subset of menu items that pertain to the current location context and at least one other menu item.

9. The computing device of claim 1, the tap consisting of a single tap.

10. The computing device of claim 9, the spoken message conveying a title of the menu.

11. The computing device of claim 1, the tap comprising a double tap, the spoken message conveying individual menu items of the menu.

12. A method performed by a computing device having a touch-sensitive screen, the method comprising:

obtaining different location contexts for different locations of a user as the user moves through a physical or virtual space using a mode of transportation;

detecting a touch gesture performed on the touch-sensitive screen;

responsive to the touch gesture, navigating among a plurality of workspaces to a particular workspace, the plurality of workspaces having respective predetermined spatial relationships and respective associated menus;

after the user has navigated to the particular workspace, detecting a tap on the touch-sensitive screen and, in response, outputting a spoken message conveying information regarding a current location context of the user in the physical or virtual space;

detecting another touch gesture performed on the touch-sensitive screen;

responsive to the another touch gesture, displaying an individual menu associated with the particular workspace on the touch-sensitive screen, the individual menu having a plurality of menu items relating to the current location context of the user in the physical or virtual space; and

performing a particular operation in response to a selection of a particular menu item from the individual menu.

13. The method of claim 12, the physical or virtual space being a virtual space.

14. The method of claim 12, further comprising:

upon receiving the touch gesture, moving from a previous workspace to the particular workspace based at least upon a specific direction of the touch gesture and the predetermined spatial relationships.

15. The method of claim 14, the touch gesture comprising a flick gesture in the specific direction.

16. The method of claim 12, further comprising:

when the current location context indicates the user is riding in a vehicle, designating the individual menu as a vehicle-specific menu having individual menu items pertaining to the experience of the user in the vehicle.

17. The method of claim 16, further comprising:

when the current location context has switched from riding in the vehicle to walking on a street, switching to a walking-specific menu that provides information regarding the experience of the user when walking on the street.

18. The method of claim 12, further comprising:

after navigating to the particular workspace, detecting a flick gesture in a particular direction;

responsive to the flick gesture, navigating to another workspace designated by a direction of the flick gesture; and

displaying another individual menu associated with the another workspace on the touch-sensitive screen.

19. The method of claim 18, further comprising:

performing another particular operation in response to activation of another individual menu item from the another individual menu.

20. A computer-readable storage medium storing instructions which, when executed by a processing device, cause the processing device to perform acts comprising:

obtaining different location contexts for different locations of a user as the user moves through a physical space using a mode of transportation;

detecting a gesture performed by the user;

based at least upon a direction of the gesture, selecting a particular workspace from a plurality of available workspaces having respective associated menus;

detecting a tap gesture performed by the user and, in response, outputting a spoken message conveying information regarding a current location context of the user;

detecting another gesture performed by the user;

displaying an individual menu associated with the particular workspace, the individual menu having a plurality of menu items relating to the current location context of the user; and

performing a particular operation in response to a user selection of a particular menu item from the individual menu.