MAPPING A NATURAL INPUT DEVICE TO A LEGACY SYSTEM

• US 20100199229A1
• Filed: 03/25/2009
• Published: 08/05/2010
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Snook, Gregory G., Polzin, Stephen, Bennett, Darren, Tsunoda, Kudo, Kipman, Alex, Markovic, Relja, Finocchio, Mark, Latta, Stephen
• Granted Patent: US 8,448,094 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Systems and methods for mapping natural input devices to legacy system inputs are disclosed. One example system may include a computing device having an algorithmic preprocessing module configured to receive input data containing a natural user input and to identify the natural user input in the input data. The computing device may further include a gesture module coupled to the algorithmic preprocessing module, the gesture module being configured to associate the natural user input to a gesture in a gesture library. The computing device may also include a mapping module to map the gesture to a legacy controller input, and to send the legacy controller input to a legacy system in response to the natural user input.

Claims

1. A method of mapping a natural input device to a legacy system, the method comprising:

selecting a gesture library to interpret a natural user input;

associating the natural user input to a gesture in the gesture library;

mapping the gesture to a legacy controller input; and

sending the legacy controller input to a legacy system in response to the natural user input.

2. The method of claim 1, wherein mapping the gesture to a legacy controller input includes:

mapping the selected gesture to a legacy controller state using a look up table, and

generating the legacy controller input via a legacy controller emulator.

3. The method of claim 1, further comprising:

receiving a depth image from an image capture device, the depth image having depth values for objects in a field of view of the depth camera; and

processing the depth image to identify the natural user input.

4. The method of claim 3, wherein the legacy system is a legacy gaming system, wherein processing the depth image to identify a natural user input further comprises:

removing background elements from the depth image;

segmenting players of the gaming system within the depth image;

mapping a skeleton to each segmented player;

associating a change of at least one skeleton with a gesture in the gesture library; and

identifying the natural user input based on the gesture.

5. The method of claim 4, wherein the gesture library is selected from a plurality of gesture libraries, each of the plurality of gesture libraries corresponding to a respective game genre.

6. The method of claim 1, wherein the legacy controller input is a first legacy controller input, the method further comprising:

adding a configurable gesture to the gesture library;

mapping the configurable gesture to a second legacy controller input; and

sending the second legacy controller input to a legacy system in response to the natural user input.

7. The method of claim 3, further comprising:

operating a legacy computer program on the legacy system; and

providing the legacy controller input to the legacy game in response to the natural user input.

16. The computer-readable medium of claim 1, wherein mapping the gesture to a legacy controller input includes:

mapping the selected gesture to a legacy controller state using a look up table, and

generating the legacy controller input via a legacy controller emulator.

8. A computing device for mapping a natural input device to a legacy system, the device comprising:

an algorithmic preprocessing module configured to receive input data containing a natural user input and to identify the natural user input in the input data;

a gesture module coupled to the algorithmic preprocessing module, the gesture module to associate the natural user input to a gesture in a gesture library;

a mapping module to map the gesture to a legacy controller input, and to send the legacy controller input to a legacy system in response to the natural user input.

9. The method of claim 8, wherein the mapping module includes:

a mapper configured to map the selected gesture to a legacy controller state using a look up table, and

a legacy controller emulator configured to receive the legacy controller state from the mapper and generate the legacy controller input based on the legacy controller state.

10. The computing device of claim 9, wherein the legacy system is a legacy gaming system.

11. The computing device of claim 8, wherein the input data containing a natural user input includes a depth image from an image capture device, the depth image having depth values for objects in a field of view of the image capture device.

12. The computing device of claim 9, wherein the algorithmic preprocessing module further comprises:

a background removal service to remove background elements from the depth image;

a player segmenting service in communication with the background removal service, the player segmenting service to segment players of the legacy gaming system within the depth image;

a player mapping service to map a skeleton to each segmented player, wherein in response to a change of at least one skeleton the algorithmic preprocessing module associates the change with a gesture in the gesture library and identifies the natural user input based on the gesture.

13. The computing device of claim 8, wherein the gesture module is further configured to select a gesture library from a plurality of gesture libraries to interpret the natural user input, wherein each of the plurality of gesture libraries corresponds to a respective computer program genre.

14. The computing device of claim 13, further comprising a genre update module configured to securely update a gesture library in the gesture module.

15. A computer-readable medium comprising instructions executable by a computing device to map a natural input device to a legacy gaming system, the instructions being executable to perform a method comprising:

receiving a depth image from a depth camera, the depth image having depth values for objects in a field of view of the depth camera;

processing the depth image to identify a natural user input;

selecting a gesture library to interpret the natural user input;

associating the natural user input to a gesture in the gesture library;

mapping the gesture to a legacy controller input; and

sending the legacy controller input to a gaming system in response to the natural user input.

17. The computer-readable medium of claim 15, further comprising instructions executable for:

removing background elements from the depth image;

segmenting players of the gaming system within the depth image;

mapping a skeleton to each segmented player;

associating a change of at least one skeleton with a gesture in the gesture library; and

identifying the natural user input based on the gesture.

18. The computer-readable medium of claim 15, further comprising instructions executable for:

adding a configurable gesture to the gesture library;

mapping the configurable gesture to a second legacy controller input; and

sending the second legacy controller input to a gaming system in response to the natural user input.

19. The computer-readable medium of claim 15, wherein the gesture library is selected from a plurality of gesture libraries, each of the plurality of gesture libraries corresponding to a game genre.

20. The computer-readable medium of claim 15, further comprising instructions executable for:

operating a legacy game on the gaming system; and

providing the legacy controller input to the legacy game in response to the natural user input.