Driving simulator control with virtual skeleton

• US 9,821,224 B2
• Filed: 12/21/2010
• Issued: 11/21/2017
• Est. Priority Date: 12/21/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Latta, Stephen, Bennett, Darren, Geisner, Kevin, Markovic, Relja, Tsunoda, Kudo, Mathis, Rhett, Monson, Matthew, Gierok, David, Giese, William Paul, Brown, Darrin, McRae, Cam, Seymour, David, Olsen, William Axel, Searcy, Matthew
• Primary Examiner(s): Lewis, David L
• Assistant Examiner(s): LEICHLITER, CHASE E
• Application Number:
  US12/975,011
• Publication Number:
  US 20120157198A1
• Time in Patent Office: 2,527 Days
• Field of Search: 463 1, 463 6, 463 7, 463 36, 463 37, 463 30- 33, 345156, 345419, 345473, 345474, 382128, 382154, 382103, 703 2, 703 6, 702 41, 348169, 715757
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09B 9/04 for teaching control of land vehicles

Abstract

Depth-image analysis is performed with a device that analyzes a human target within an observed scene by capturing depth-images that include depth information from the observed scene. The human target is modeled with a virtual skeleton including a plurality of joints. The virtual skeleton is used as an input for controlling a driving simulation.

Claims

1. A data holding device holding instructions executable by a logic sub system to:

render a driving game for display on a display device;

receive a virtual skeleton including a plurality of joints, the plurality of joints including a left hand joint and a right hand joint, the virtual skeleton providing a machine readable representation of a human target observed with a three-dimensional depth camera;

translate an angle between a reference line and a line connecting the left hand joint and the right hand joint into a gestured steering angle control; and

steer a virtual vehicle of the driving game in proportion to the gestured steering angle control.

2. The data holding device of claim 1, further holding instructions executable by the logic subsystem to also steer the virtual vehicle in proportion to an artificial intelligence recommended control such that the virtual vehicle turns in proportion to a blend of both the gestured steering angle control and the artificial intelligence recommended control.

3. The data holding device of claim 2, wherein a relative percentage of the gestured steering angle control and the artificial intelligence recommended control is dynamically set according to one or more changeable game parameters.

4. The data holding device of claim 3, wherein the one or more changeable game parameters includes a speed of the virtual vehicle, and wherein the relative percentage of the gestured steering angle control decreases as the speed of the virtual vehicle increases.

5. The data holding device of claim 3, wherein the one or more changeable game parameters includes a turning radius of a virtual track, and wherein the relative percentage of the gestured steering angle control decreases as the turning radius of the virtual track increases.

6. The data holding device of claim 3, wherein the one or more changeable game parameters includes a deviation from a recommended path, and wherein the relative percentage of the gestured steering angle control decreases as the deviation from the recommended path increases.

7. The data holding device of claim 1, further holding instructions executable by the logic subsystem to:

translate a position of a left foot joint of the virtual skeleton relative to a position of a right foot joint of the virtual skeleton into a gestured acceleration control; and

accelerate the virtual vehicle in proportion to the gestured acceleration control.

8. The data holding device of claim 7, wherein a right foot joint in front of the left foot joint is translated as a positive acceleration, and a right foot joint behind the left foot joint is translated as a negative acceleration.

9. The data holding device of claim 1, further holding instructions executable by the logic subsystem to:

translate a distance of the left hand joint and the right hand joint away from one or more torso joints into a gestured acceleration control; and

accelerate the virtual vehicle in proportion to the gestured acceleration control.

10. The data holding device of claim 9, wherein positive acceleration increases in proportion to the distance of the left hand joint and the right hand joint away from the one or more torso joints.

11. The data holding device of claim 1, further holding instructions executable by the logic subsystem to:

translate a sudden movement of a hand joint from near one or more torso joints to a position away from the one or more torso joints into a gestured turbo boost control; and

rapidly accelerating the virtual vehicle in response to the gestured turbo boost control.

12. The data holding device of claim 1, further holding instructions executable by the logic subsystem to:

translate a movement of a hand joint near a hip joint into a gestured gear shift control; and

changing a gear of the virtual vehicle in response to the gestured gear shift control.

13. The data holding device of claim 12, wherein the gear is increased responsive to a forward movement of the hand joint and the gear is decreased responsive to a backwards movement of the hand joint.

14. The data holding device of claim 1, further holding instructions executable by the logic subsystem to:

translate a posture of one or more hands modeled by corresponding hand joints of the virtual skeleton into a gestured gear shift control; and

changing a gear of the virtual vehicle in response to the gestured gear shift control.

15. The data holding device of claim 14, wherein the gear is increased responsive to an opening and closing of a right hand and the gear is decreased responsive to an opening and closing of a left hand.

16. The data holding device of claim 1, wherein the angle between a reference line and the line connecting the left hand joint and the right hand joint is translated into a gestured steering angle control only if the right hand joint and the left hand joint are outstretched in front of one or more torso joints.

17. An entertainment system, comprising:

a peripheral input to receive depth images from a depth camera;

a display output to output a driving game interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a human target;

model the human target with a virtual skeleton including a plurality of joints, the plurality of joints including a left hand joint, a right hand joint, one or more torso joints, a left foot joint, and a right foot joint;

if the right hand joint and the left hand joint are outstretched in front of the one or more torso joints;

translate an angle between a horizon and a line connecting the left hand joint and the right hand joint into a gestured steering angle control, and

steer a virtual vehicle in proportion to the gestured steering angle control;

translate a position of the left foot joint relative to a position of the right foot joint into a gestured acceleration control; and

accelerate the virtual vehicle in proportion to the gestured acceleration control.

18. The entertainment system of claim 17, wherein the data holding subsystem further holds instructions executable by the logic subsystem to also steer the virtual vehicle in proportion to an artificial intelligence recommended control such that the virtual vehicle turns in proportion to a blend of both the gestured steering angle control and the artificial intelligence recommended control.

19. The entertainment system of claim 18, wherein a relative percentage of the gestured steering angle control and the artificial intelligence recommended control is dynamically set according to one or more changeable game parameters.