Contextual user interface

• US 10,248,301 B2
• Filed: 09/16/2015
• Issued: 04/02/2019
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Stachniak, Szymon Piotr, Lobb, Kenneth Alan, Esposito, Mario, Woon, Clinton Chi-Wen
• Primary Examiner(s): Tran, Tuyetlien T
• Application Number:
  US14/856,281
• Publication Number:
  US 20160004301A1
• Time in Patent Office: 1,294 Days
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/493 Resuming a game, e.g. after pausing, malfunction or power failure
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 : Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04N 21/4122 : additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

Embodiments of the present invention analyze a context in which a user interacts with a computer interface and automatically optimizes the interface for the context. The controller or control mode the user selects for interaction may define the context, in part. Examples of control modes include gesturing, audio control, use of companion devices, and use of dedicated control devices, such as game controllers and remote controls. The different input devices are designed for different tasks. Nevertheless, a user will frequently attempt to perform a task using a control input that is not adapted for the task. Embodiments of the present invention change the characteristics of the user interface to make it easier for the user to complete an intended task using the input device of the user'"'"'s choice.

Claims

1. A method of presenting an interface that is optimized for a present context, the method comprising:

determining a gamepad previously used by a user when the user was playing a video game by analyzing image data showing the user'"'"'s past interaction with the gamepad;

determining a present context for a user interface by analyzing image data depicting an environment proximate to the user interface wherein the present context is the user picking up the gamepad; and

generating a present user interface that is optimized for the present context by automatically opening the video game last played by the user when using the gamepad.

2. The method of claim 1, further comprising:

determining a new context for the user interface by analyzing image data depicting the environment proximate to the user interface and signal data received from a companion device indicating the companion device is wirelessly connected to a computing system, the image data indicating that the user has picked up the companion device; and

automatically optimizing the user interface for the new context by changing the user interface to enable control by the companion device.

3. The method of claim 2, wherein the new context is disambiguated from multiple possible contexts using previous user interactions with the user interface.

4. The method of claim 2, wherein analyzing image data depicting the environment proximate to the user interface comprises determining a control mode utilized by a user to interact with the user interface.

5. The method of claim 1, wherein the image data is a three-dimensional image data.

6. The method of claim 1, wherein the method further comprises:

determining a new context for the user interface by analyzing image data depicting the environment proximate to the user interface, the image data indicating that the user is making a hand gesture; and

automatically optimizing the user interface for the new context by changing the user interface to increase a size of control inputs for improved gesture interaction.

7. The method of claim 1, wherein the method further comprises:

a object is a game object and an application is the video game related to the object.

8. A computing system comprising:

a processor; and

computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, implement a method of presenting an interface that is optimized for a present context, the method comprising;

determining a gamepad previously used by a user when the user was playing a video game by analyzing image data showing the user'"'"'s past interaction with the gamepad;

determining a present context for a user interface by analyzing image data depicting an environment proximate to the user interface wherein the present context is the user picking up the gamepad; and

generating a present user interface that is optimized for the present context by automatically opening the video game last played by the user when using the gamepad.

9. The system of claim 8, wherein the method further comprises:

determining a new context for the user interface by analyzing image data depicting the environment proximate to the user interface and signal data received from a companion device indicating the companion device is wirelessly connected to the computing system, the image data indicating that the user has picked up the companion device; and

automatically optimizing the user interface for the new context by changing the user interface to enable control by the companion device.

10. The system of claim 9, wherein the new context is disambiguated from multiple possible contexts using previous user interactions with the user interface.

11. The system of claim 9, wherein analyzing image data depicting the environment proximate to the user interface comprises determining a control mode utilized by a user to interact with the user interface.

12. he system of claim 8, wherein the image data is a three-dimensional image data.

13. The system of claim 12, wherein the method further comprises:

determining a new context for the user interface by analyzing image data depicting the environment proximate to the user interface, the image data indicating that the user is making a hand gesture; and

automatically optimizing the user interface for the new context by changing the user interface to increase a size of control inputs for improved gesture interaction.