Facilitating interaction between users and their environments using a headset having input mechanisms

• US 9,977,573 B2
• Filed: 04/19/2015
• Issued: 05/22/2018
• Est. Priority Date: 10/31/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ermilov, Andrei Stefan, Cotier, Bradley Neville, Doroftei, Catalin Nicolae, Chaaban, Hani Nabil, Chudge, Jarnail Singh, Farmer, Matthew Robin, Parker, Michael James Belfield, Salandin, Oscar James Francis, Wilcox, Robert John
• Primary Examiner(s): Pervan, Michael
• Application Number:
  US14/690,422
• Publication Number:
  US 20160124707A1
• Time in Patent Office: 1,129 Days
• Field of Search: 345 7- 9, 345156-178
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G01C 21/3635 Guidance using 3D or perspective road maps
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G01S 5/0295 Proximity-based methods, e.g. position inferred from reception of particular signals
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/452 : Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06Q 30/0205 : Location or geographical consideration
• CPC Class Codes: G08G 1/00 : Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D
• CPC Class Codes: G08G 1/005 : including pedestrian guidance indicator
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/0968 : Systems involving transmission of navigation instructions to the vehicle
• CPC Class Codes: G09B 21/006 : using audible presentation of the information
• CPC Class Codes: H04R 1/1041 : Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 27/00 : Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 5/033 : Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04S 2400/11 : Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 : Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/304 : For headphones
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/021 : Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences

Abstract

A headset is described herein for presenting audio information to the user as the user interacts with a space, e.g., as when a user navigates over a route within a space. The headset may include a set of input mechanisms for receiving commands from the user. The commands, in turn, invoke respective space-interaction-related functions to be performed by a space interaction (SI) module. The headset may operate with or without a separate user computing device.

Claims

1. A system comprising:

a headset configured to present audio information to a user as the user interacts with a space; and

one or more processing devices provided on the headset or on a separate computing device,

the headset comprising a set of input mechanisms configured to receive commands from the user invoking respective space-interaction-related functions to be performed by the one or more processing devices,

one or more processing being configured to;

when instructed by a specific command received from the user, enter an explore mode;

while in the explore mode;

identify an orientation of the headset;

based at least on the orientation of the headset, identify a subspace to which the user is currently directing attention;

identify specific items of interest within the subspace to which the user is currently directing attention; and

cause the headset to generate corresponding sounds for the specific items of interest;

when instructed by another specific command received from the user, enter an orientation mode; and

while in the orientation mode;

identify vertically-arranged items of interest outside of the subspace to which the user is currently directing attention;

based at least on vertical relationships among the vertically-arranged items of interest, determine a specific order in which to announce the vertically-arranged items of interest; and

cause the headset to generate, in the specific order, other corresponding sounds for the vertically-arranged items of interest.

2. The system of claim 1, wherein the headset uses a bone conducting mechanism to deliver the audio information to the user.

3. The system of claim 1,

wherein the one or more processing devices are configured to:

receive a further command instructing the system to start a listening mode;

enter the listening mode responsive to the further command; and

while in the listening mode, receive and follow instructions that are verbalized by the user.

4. The system of claim 1, wherein the one or more processing devices are configured to:

receive a further command instructing the system to stop delivering the audio information; and

stop delivering the audio information responsive to the further command.

5. The system of claim 1, wherein the one or more processing devices are configured to:

receive a further command instructing the system to repeat one or more prior-delivered items of audio information; and

repeat the one or more prior-delivered items of audio information responsive to the further command.

6. The system of claim 1,

wherein the sounds include a first three-dimensional sound identifying a first item of interest within the subspace and a second three-dimensional sound identifying a second item of interest within the subspace.

7. The system of claim 1, wherein the vertically-arranged items of interest are on different floors of a building.

8. The system of claim 7, wherein the one or more processing devices are configured to:

specify that the vertically-arranged items of interest are announced on a floor-by-floor basis.

9. The system of claim 1, wherein the one or more processing devices are configured to:

specify that the vertically-arranged items of interest are announced starting from a vertically lowest item of interest to progressively higher items of interest.

10. The system of claim 1, wherein the one or more processing devices are provided on the headset.

11. The system of claim 1, wherein the headset further includes:

a headset orientation determination mechanism configured to determine the orientation of the headset.

12. The system of claim 11, wherein the system further comprises at least one user computing device that includes the one or more processing devices.

13. The system of claim 12, wherein the user computing device and the headset are configured to communicate using short-range wireless radio transmission.

14. The system of claim 12, wherein the user computing device comprises a smartphone or a tablet-type computing device.

15. The system of claim 1, wherein the specific items of interest include a physical item of interest and a virtual item of interest.

16. A method comprising:

identifying an orientation of a headset worn by a user;

based at least on the orientation of the headset, identifying a search volume of interest to the user;

delivering audio information, via the headset, which identifies at least two different items of interest present in the search volume of interest to the user;

identifying vertically-arranged items of interest outside of the search volume;

based at least on vertical relationships among the vertically-arranged items of interest, determining a specific order in which to announce the vertically-arranged items of interest; and

delivering, via the headset and in the specific order, other audio information that identifies the vertically-arranged items of interest.

17. The method of claim 16, performed by the headset.

18. The method of claim 16, performed by a separate user computing device which is communicatively coupled to the headset.

19. A system comprising:

a wearable device having an audio output mechanism;

a processing device; and

logic which, when executed by the processing device, causes the processing device to;

determine an orientation of the wearable device;

based at least on the orientation of the wearable device, identify a search volume around the wearable device;

identify an item of interest that is located within the search volume;

cause the audio output mechanism of the wearable device to output a sound indicating that the item of interest is present in the search volume;

identify vertically-arranged items of interest outside of the search volume;

based at least on vertical relationships among the vertically-arranged items of interest, determine as specific order in which to announce the vertically-arranged items of interest; and

cause the audio output mechanism of the wearable device to output, in the specific order, other sounds identifying the vertically-arranged items of interest.

20. The system of claim 19, the search volume comprising an angular swath of less than 180 degrees.

21. The system of claim 19, wherein the logic, when executed by the processing device, causes the processing device to:

identify a cylindrical volume around the system, the cylindrical volume including an area not encompassed by the search volume; and

identify the vertically-arranged items of interest within the cylindrical volume.