Gesture tool

• US 8,856,691 B2
• Filed: 05/29/2009
• Issued: 10/07/2014
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geisner, Kevin, Latta, Stephen, Snook, Gregory N., Markovic, Relja, Tomlin, Arthur Charles, Mihelich, Mark, Lee, Kyungsuk David, Horbach, David Jason Christopher, Puls, Matthew Jon
• Primary Examiner(s): Ng, Amy
• Assistant Examiner(s): Xiao, Di
• Application Number:
  US12/474,655
• Publication Number:
  US 20100306713A1
• Time in Patent Office: 1,957 Days
• Field of Search: 715/863, 715/707, 715/866
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

Systems, methods and computer readable media are disclosed for a gesture tool. A capture device captures user movement and provides corresponding data to a gesture recognizer engine and an application. From that, the data is parsed to determine whether it satisfies one or more gesture filters, each filter corresponding to user-performed gesture. The data and the information about the filters is also sent to a gesture tool, which displays aspects of the data and filters. In response to user input corresponding to a change in a filter, the gesture tool sends an indication of such to the gesture recognizer engine and application, where that change occurs.

Claims

1. A computer system, comprising:

a processor;

a tool operable to execute on the processor to facilitate the adjustment of parameters of a gesture filter of an application that is designed to recognize whether a user of the application has performed a particular gesture from data comprising information about position or movement of the user, the tool operable to;

receive data captured by a capture device, the data comprising information representing user position or movement;

display a visual representation of the position or movement represented by the received data, the visual representation comprising a list identifying a gesture filter;

in response to user selection of the gesture filter from the list, display at least one adjustable parameter of the gesture filter;

in response to the system processing the data with the gesture filter and the adjustable parameter, display an indication of whether the position or movement represented by the data would be recognized by the gesture filter as the performance of the gesture that the gesture filter is designed to recognize;

receive, via a user interface of the display, a change to a parameter of the gesture filter of the application;

alter the gesture filter according to the received change; and

display an indication of whether processing the data with the gesture filter and the changed parameter causes the system to determine that the position or movement represented by the data would be recognized by the gesture filter as the performance of the gesture that the gesture filter is designed to recognize.

2. The system of claim 1, the tool further comprising capability to operate to:

receive the change from the visual representation across a communications network.

3. The system of claim 1, wherein the tool is operable to display a visual representation, the tool further comprising capability to operate to:

display a skeletal map associated with the data; and

in response to user selection of a gesture filter from the list, displaying an indication of at least one part of the skeletal map associated with the gesture filter.

4. The system of claim 1, wherein the tool is operable to display a visual representation the tool further comprising capability to operate to:

in response to user selection of a gesture filter from the list, display at least one output of the selected gesture filter.

5. The system of claim 1, wherein the tool further comprises capability to operate to display at least one output as a graph of a value of the output over time.

6. The system of claim 1, wherein each gesture filter of the list is operable to be specified by the application.

7. The system of claim 1, wherein the tool further comprises capability to operate to record the user data is, the tool further comprising capability to operate to:

adjust the rate or direction of display of the visual representation.

8. A method for altering visualized parameters for a gesture filter of an application, the gesture filter being designed to recognize whether to recognize a user of the application has performed a particular gesture from data comprising information about position or movement of the user, the method comprising:

receiving data captured by a capture device, the data comprising information about user position or movement;

displaying a visual output of the application;

displaying a visual representation of at least one aspect of the user data, the visual representation comprising a list comprising the gesture filter;

displaying a skeletal map associated with the user data;

in response to user selection of a gesture filter from the list, displaying an indication of at least one body part involved in the performance of the gesture that the gesture filter is designed to recognize;

in response to the system processing the data with the gesture filter and the adjustable parameter, displaying a representation indicating whether the position or movement represented by the data would be recognized by the gesture filter as the performance of the gesture that the gesture filter is designed to recognize;

receiving, via a user interface of the display, a change to a parameter of the gesture filter of the application;

altering the gesture filter corresponding to the received change; and

displaying a representation of whether processing the data with the gesture filter and the changed parameter causes the system to determine that the position or movement represented by the data would be recognized by the gesture filter as the performance of the gesture that the gesture filter is designed to recognize.

9. The method of claim 8, wherein the data is either received from the capture device as it is captured, or the data is received from a storage location where the data is stored as it is captured.

10. The method of claim 8, wherein the visual representation and the visual output are either displayed on a single display device or displayed on separate display devices.

11. The method of claim 10, wherein the visual representation and the visual output are displayed on a single display device through either a split-screen or picture-in-picture.

12. The method of claim 8, wherein the data comprises at least one from a set, the set comprising:

skeletal data, visible color data, infra-red color data, and depth data.

13. The method of claim 8, wherein the captured data comprises color image data and depth data, and wherein the visual representation comprises an overlay of the color image data and skeletal mapping data derived at least in part from the depth data.

14. The method of claim 8, wherein the visual output comprises a list of a plurality of gesture filters of the application, further comprising:

in response to user selection of a selected gesture filter of the plurality of gesture filters, displaying the visual representation of at least one aspect of the user data, the visual representation corresponding to the selected gesture filter.

15. The method of claim 8, wherein the visual representation comprises a skeletal map of at least part of the user corresponding to the data, further comprising:

displaying an indication in the visual representation of at least one part of the skeletal map that corresponds to a selected gesture filter.

16. The method of claim 8, further comprising:

storing the data captured by the capture device.

17. The method of claim 8, wherein the visual representation comprises a value, a range of values, or a bound of the parameter.

18. A volatile or nonvolatile computer storage medium for altering visualized parameters for a gesture filter of an application bearing computer-readable instructions that, when executed on a processor, cause the processor to perform operations comprising:

receiving data captured by a capture device, the data comprising information representing user position or movement;

displaying a visual representation of the position or movement represented by the received data, the visual representation comprising a list comprising the gesture filter;

in response to user selection of the gesture filter from the list, displaying at least one adjustable parameter of the gesture filter and an indication of at least one body part involved in the performance of the gesture that the gesture filter is designed to recognize;

displaying a representation of whether the data invokes a computer-recognized input command that corresponds to the gesture filter in response to the system processing the data with the gesture filter and the adjustable parameter, based on the determination of the system;

receiving, via a user interface of the display, a change to a parameter of the gesture filter of the application;

altering the gesture filter according to the received change; and

displaying a representation of whether processing the data with the gesture filter and the changed parameter causes the system to determine that the data invokes the gesture filter.

19. The computer-readable storage medium of claim 18, wherein displaying a visual representation of the position or movement represented by the received data comprises:

in response to user selection of the gesture filter from the list, displaying at least one adjustable parameter of the gesture filter.

20. The computer-readable storage medium of claim 18, further bearing computer-readable instructions that, when executed on the processor, cause the processor to perform operations comprising:

displaying a skeletal map associated with the data; and

in response to user selection of a gesture filter from the list, displaying an indication of at least one part of the skeletal map associated with the gesture filter.