Push personalization of interface controls

• US 9,067,136 B2
• Filed: 03/10/2011
• Issued: 06/30/2015
• Est. Priority Date: 03/10/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Langridge, Adam Jethro
• Primary Examiner(s): Lee, Jr., Kenneth B
• Application Number:
  US13/045,323
• Publication Number:
  US 20120229381A1
• Time in Patent Office: 1,573 Days
• Field of Search: 345156-184, 715/700, 715856-862, 63 6- 8, 63 30- 31, 463 1- 8, 463/30, 463/42, 463 50- 69, 710 1- 74
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A computing system is configured to receive one or more depth images, from the depth camera, of a world space scene including a human target. The computing system translates a world space position of a hand of the human target to a screen space cursor position of the user interface using a virtual desktop transformation. The computing system also dynamically adjusts the virtual desktop transformation based on a history of button press actions executed by the human target.

Claims

1. A computing system, comprising:

a peripheral input configured to receive depth images from a depth camera;

a display output configured to output a user interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a human target;

translate a world space position of a hand of the human target in a curved virtual interface space having a curved virtual shape to an uncurved screen space cursor position of the user interface using a virtual desktop transformation, the virtual desktop transformation and the curved virtual shape of the curved virtual interface space being dynamically adjusted based on a history of button press actions executed by the human target.

2. The computing system of claim 1, wherein each button press action in the history of button press actions is identified based on a change in the screen space cursor position responsive to a movement of the hand in world space.

3. The computing system of claim 1, wherein the history of button press actions includes a plurality of press vectors, each press vector corresponding to a button press action in the history of button press actions.

4. The computing system of claim 3, wherein the data holding subsystem holds instructions executable by the logic subsystem to derive a button press origin from the plurality of press vectors.

5. The computing system of claim 4, wherein the virtual desktop transformation translates the world space position of the hand in the virtual interface space.

6. The computing system of claim 5, wherein the virtual shape of the virtual interface space is dynamically adjusted based on the button press origin.

7. The computing system of claim 6, wherein the virtual interface space has a virtual position and orientation based on a position of the human target.

8. The computing system of claim 7, wherein the virtual position and orientation of the virtual interface space is dynamically adjusted based on a position of the button press origin in relation to a position of a head of the human target.

9. A computing system, comprising:

a peripheral input configured to receive depth images from a depth camera;

a display output configured to output a user interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a human target;

translate a world space position of a hand of the human target in a curved virtual interface space having a virtual shape to a screen space cursor position in a user interface of an uncurved screen space using a virtual desktop transformation, the virtual desktop transformation and the virtual shape of the curved virtual interface space being dynamically adjusted by a history of button press actions executed by the human target.

10. The computing system of claim 9, wherein a curvature of the curved virtual interface space has a virtual focal point derived from the history of button press actions executed by the human target.

11. The computing system of claim 10, wherein the history of button press actions includes one or more button press actions, each button press action identified based on a change in the screen space cursor position responsive to a movement of the hand in world space, each button press action further represented by a press vector.

12. The computing system of claim 11, wherein the history of button press actions further includes a button press origin derived from the press vector of each button press action.

13. The computing system of claim 12, wherein the virtual focal point is derived from the button press origin.

14. The computing system of claim 13, wherein the virtual interface space has a virtual position and orientation derived from a position of the button press origin in relation to a position of a head of the human target.

15. A computing system, comprising:

a peripheral input configured to receive depth images from a depth camera;

a display output configured to output a user interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a human target;

translate a world space position of a hand of the human target in a curved virtual interface space to a screen space cursor position of a user interface in an uncurved screen space using a virtual desktop transformation, the curved virtual interface including a matrix of grid segments;

compile press vectors generated for each button press action in a history of button press actions executed by the human target, each button press action identified based on a change in the screen space cursor position responsive to a movement of the hand in world space; and

dynamically adjust the curved virtual interface space based on a virtual focal point, the virtual focal point derived from the press vectors based on a position of the press vectors within the matrix of grid segments.

16. The computing system of claim 15, wherein dynamically adjusting the curved virtual interface space includes adjusting a curvature of the curved virtual interface space based on a position of the virtual focal point in relation to a position of the human target.

17. The computing system of claim 16, wherein the data holding subsystem holds instructions executable by the logic subsystem to:

group the press vectors into one or more clusters, each of the one or more clusters including one or more press vectors selected based on the position of each press vector in the matrix of grid segments;

calculate a mean press vector for each of the one or more clusters;

determine a button press origin based on the mean press vector of each of the one or more clusters; and

derive the virtual focal point of the curved virtual interface space based on the button press origin.

18. The computing system of claim 17, wherein the data holding subsystem holds instructions executable by the logic subsystem to:

derive a left hand virtual focal point based on press vectors generated for each left hand button press action in the history of button press actions executed by the human target, and derive a right hand virtual focal point based on press vectors generated for each right hand button press action in the history of button press actions executed by the human target.

19. The computing system of claim 18, wherein the data holding subsystem holds instructions executable by the logic subsystem to:

dynamically adjust the curved virtual interface space based on the left hand virtual focal point responsive to a movement of the left hand of the human target in world space; and

dynamically adjust the curved virtual interface space based on the right hand virtual focal point responsive to a movement of the right hand of the human target in world space.

20. The computing system of claim 15, wherein the history of button press actions includes implicit button press actions and/or explicit button press actions.