Contextual user interface

• US 9,170,667 B2
• Filed: 12/21/2012
• Issued: 10/27/2015
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Esposito, Mario, Stachniak, Szymon Piotr, Lobb, Kenneth Alan, Woon, Clinton Chi-Wen
• Primary Examiner(s): TRAN, TUYETLIEN T
• Application Number:
  US13/725,925
• Publication Number:
  US 20130326376A1
• Time in Patent Office: 1,040 Days
• Field of Search: 715/762, 709/728
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/493 Resuming a game, e.g. after pausing, malfunction or power failure
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/403 : Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04N 21/4122 : additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

Embodiments of the present invention analyze a context in which a user interacts with a computer interface and automatically optimizes the interface for the context. The controller or control mode the user selects for interaction may define the context, in part. Examples of control modes include gesturing, audio control, use of companion devices, and use of dedicated control devices, such as game controllers and remote controls. The different input devices are designed for different tasks. Nevertheless, a user will frequently attempt to perform a task using a control input that is not adapted for the task. Embodiments of the present invention change the characteristics of the user interface to make it easier for the user to complete an intended task using the input device of the user'"'"'s choice.

Claims

1. A method of presenting an interface that is optimized for a present context, the method comprising:

determining a present context for a user interface by analyzing image data depicting an environment proximate to the user interface, wherein the present context is a user performing a gesture with a user'"'"'s left hand to interact with an object depicted on the user interface, the gesture detected by analyzing the image data;

determining that the user'"'"'s left hand is coming between a second user and the user interface while making the gesture by analyzing the image data; and

upon said determining that the user'"'"'s left hand is coming between a second user and the user interface, automatically generating an updated user interface with interface objects relocated to encourage the user to make right handed gestures instead of left handed gestures.

2. The method of claim 1, wherein the present context comprises the user facing the user interface while performing the gesture.

3. The method of claim 1, wherein the method further comprises automatically re-optimizing the user interface upon determining a new context for the interface.

4. The method of claim 1, wherein the user interface is optimized by moving the object toward the user'"'"'s right hand.

5. The method of claim 1, wherein the user interface is optimized for a gesture control mode in which the user does not use a controller by increasing a size of the object to an enlarged size that is larger than an original size of the object in a user interface optimized for interaction with a gamepad.

6. The method of claim 1, wherein the object is an overlay.

7. The method of claim 1, wherein the image data is three-dimensional image data.

8. One or more computer-storage hardware media having computer-executable instructions embodied thereon, that when executed by a computing device perform a method of presenting an interface that is optimized for a present context, the method comprising:

determining a present context for a user interface by analyzing image data depicting an environment proximate to the user interface, wherein the present context is a user performing a gesture with a user'"'"'s left hand to interact with an object depicted on the user interface, the gesture detected by analyzing the image data;

determining that the user'"'"'s left hand is coming between a second user and the user interface while making the gesture by analyzing the image data; and

upon said determining that the user'"'"'s left hand is coming between a second user and the user interface, automatically generating an updated user interface with interface objects relocated to encourage the user to make right handed gestures instead of left handed gestures.

9. The media of claim 8, wherein the present context comprises the user facing the user interface while performing the gesture.

10. The media of claim 8, wherein the method further comprises automatically re-optimizing the user interface upon determining a new context for the interface.

11. The media of claim 8, wherein the user interface is optimized by moving the object toward the user'"'"'s right hand.

12. The media of claim 8, wherein the user interface is optimized for a gesture control mode in which the user does not use a controller by increasing a size of the object to an enlarged size that is larger than an original size of the object in a user interface optimized for interaction with a gamepad.

13. The media of claim 8, wherein the object is an overlay.

14. The media of claim 8, wherein the image data is three-dimensional image data.

15. A computing system comprising:

a hardware processor; and

computer storage memory having computer-executable instructions stored thereon which, when executed by the processor, implement a method of presenting an interface that is optimized for a present context, the method comprising;

determining a present context for a user interface by analyzing image data depicting an environment proximate to the user interface, wherein the present context is a user performing a gesture with a user'"'"'s left hand to interact with an object depicted on the user interface, the gesture detected by analyzing the image data;

determining that the user'"'"'s left hand is coming between a second user and the user interface while making the gesture by analyzing the image data; and

upon said determining that the user'"'"'s left hand is coming between a second user and the user interface, automatically generating an updated user interface with interface objects relocated to encourage the user to make right handed gestures instead of left handed gestures.

16. The system of claim 15, wherein the present context comprises the user facing the user interface while performing the gesture.

17. The system of claim 15, wherein the method further comprises automatically re-optimizing the user interface upon determining a new context for the interface.

18. The system of claim 15, wherein the user interface is optimized by moving the object toward the user'"'"'s right hand.

19. The system of claim 15, wherein the user interface is optimized for a gesture control mode in which the user does not use a controller by increasing a size of the object to an enlarged size that is larger than an original size of the object in a user interface optimized for interaction with a gamepad.

20. The system of claim 15, wherein the object is an overlay.