Interaction with Devices Based on User State

• US 20140028542A1
• Filed: 07/30/2012
• Published: 01/30/2014
• Est. Priority Date: 07/30/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lovitt, Andrew W., Hall, Michael, Free, Gordon, Tomko, Stefanie Lyn, Farmer, Jason R., Rau, Nathan M.
• Granted Patent: US 9,678,573 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A device identifies users who is trying to interact with it and monitors their state. Using this user state information, the device dynamically adjusts a user interface, speech grammars, screen flow, input options, and the like to tailor interaction with the device to the user. The user interface and device interaction may also be influenced by user-specific settings or profiles. The device may prevent or allow user interaction with the device based on the user'"'"'s state, such as a position relative to the device. For example, the device may prevent or allow the user from using a set of speech or gesture commands or other interaction sets based on the user'"'"'s position or relative location.

Claims

1. A method, comprising:

detecting user interaction with a device;

identifying a user state; and

modifying how the device interacts with the user based upon the user state.

2. The method of claim 1, wherein the user state is a position relative to the device.

3. The method of claim 1, further comprising:

using one or more sensors to identify the user state.

4. The method of claim 1, further comprising:

the sensors are selected from one or more of cameras, microphones, acoustic sensors, sonar sensors, radar sensors, light sensors, electronic device detectors, skeletal tracking devices, and user physiological sensors.

5. The method of claim 1, wherein modifying how the device interacts with the user further comprises:

selecting a user interface context.

6. The method of claim 5, wherein the user interface context is selected from one or more of a user interface menu, a screen flow, an arrangement of user interface components, a menu order, and a group of inputs on a display.

7. The method of claim 1, wherein modifying how the device interacts with the user further comprises:

selecting an interaction set, the interaction set defining one or more allowed interactions between the user and the device.

8. The method of claim 7, wherein the allowed interactions are selected from one or more of grammars used to interpret user speech inputs, vocabularies for text-to-speech outputs, gestures, human-machine interface, display buttons, display menus, display icons, and screen ordering.

9. The method of claim 7, wherein the interaction set is a user physiological set that is used to monitor physiological factors of the user.

10. The method of claim 1, wherein modifying how the device interacts with the user further comprises:

disabling one or more device capabilities.

11. The method of claim 10, wherein the device capabilities are selected from one or more of functions, displays, buttons, menus, speakers, inputs, and outputs.

12. The method of claim 1, wherein modifying how the device interacts with the user further comprises:

selecting a subset of all available user interactions as allowed inputs and outputs for the device.

13. A method, comprising:

defining sets of interactions that can be used with a device, each set of interactions associated with a particular state of an operator;

detecting an attempt to access the device by an operator;

identifying the operator'"'"'s state; and

selecting one of the sets of interactions based upon the operator'"'"'s state.

14. The method of claim 13, further comprising:

the state of the operator is a position of the operator relative to the device.

15. The method of claim 13, further comprising:

identifying an operator input; and

comparing the operator input to the selected set of interactions to determine if the device will respond to the input.

16. The method of claim 13, wherein the sets of interactions comprises one or more of speech inputs, gesture inputs, action inputs, human interface device inputs, or user physiological inputs.

17. The method of claim 13, wherein the operator'"'"'s state is determined using one or more of skeletal tracking, weight sensors, video sensors, light sensors, audio sensors, acceleration sensors, electronic device sensors, sonar, and radar.

18. The method of claim 13, further comprising:

allowing the operator to interact with the device using the selected set of interactions and no other inputs for a predetermined period.

19. A device, comprising:

a user interface for receiving user inputs to the device;

one or more sensors for determining the user'"'"'s location relative to the device; and

a user-interface controller adapted to modify the user interface to use a set of selected inputs based upon the user'"'"'s relative location.

20. The device of claim 19, further comprising:

the one or more sensors further determining a second user'"'"'s location relative to the device; and

the user-interface controller adapted to modify the user interface to use a second set of selected inputs based upon the second user'"'"'s relative location.