System and method for executing a game process

• US 20100138798A1
• Filed: 06/17/2009
• Published: 06/03/2010
• Est. Priority Date: 03/25/2003
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Oliver, Nuria M., Wilson, Andrew D.
• Application Number:
  US12/457,656
• Publication Number:
  US 20100138798A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals

Abstract

A 3-D imaging system for recognition and interpretation of gestures to control a computer. The system includes a 3-D imaging system that performs gesture recognition and interpretation based on a previous mapping of a plurality of hand poses and orientations to user commands for a given user. When the user is identified to the system, the imaging system images gestures presented by the user, performs a lookup for the user command associated with the captured image(s), and executes the user command(s) to effect control of the computer, programs, and connected devices.

Claims

1. A game apparatus for executing a game process using acceleration data which is sequentially outputted by an acceleration sensor of an input device, the game apparatus comprising:

acquisition means for sequentially acquiring the acceleration data;

smoothing means for sequentially calculating smoothed data representing a value which is obtained by smoothing an acceleration represented by the acceleration data; and

game process means for executing the game process using the acceleration data and the smoothed data.

2. The game apparatus according to claim 1, wherein the game process means includes

first action control means for determining, using the smoothed data, a state of a player object in a game space for determining a first action to be performed by the player object, and

second action control means for determining, using the acceleration data, a state of the player object for determining a second action to be performed by the player object.

3. The game apparatus according to claim 2, wherein

the first action control means determines an attitude of the player object in the game space using the smoothed data, and

the second action control means determines a position of the player object in the game space using the acceleration data.

4. The game apparatus according to claim 3, wherein the second action control means determines the position of the player object in the game space by determining a movement distance using the acceleration data, and by determining a moving direction in accordance with the attitude of the player object determined by the first action control means.

5. The game apparatus according to claim 1, wherein the game process means includes

difference calculation means for calculating a difference between a value of the acceleration represented by the acceleration data and the value represented by the smoothed data, and

process execution means for executing the game process using the smoothed data and the difference.

6. The game apparatus according to claim 5, wherein the game process means includes

third action control means for determining, using the smoothed data, a state of a player object in a game space for determining a third action to be performed by the player object, and

fourth action control means for determining, using the difference, a state of the player object for determining a fourth action to be performed by the player object.

7. The game apparatus according to claim 6, wherein

the third action control means determines an attitude of the player object in the game space using the smoothed data, and

the fourth action control means determines a position of the player object in the game space using the difference.

8. The game apparatus according to claim 7, wherein the fourth action control means determines the position of the player object in the game space by determining a movement distance using the difference, and by determining a moving direction in accordance with the attitude of the player object determined by the third action control means.

9. The game apparatus according to claim 6, wherein the game process means includes

fifth action control means for controlling, using the smoothed data, a state of a first player object appearing in the game space for determining an action to be performed by the first player object, and

sixth action control means for controlling, using the difference, a state of a second player object appearing in the game space for determining an action to be performed by the second player object.

10. The game apparatus according to claim 1, wherein the game process means includes

seventh action control means for controlling, using the smoothed data, a state of a first player object appearing in a game space for determining an action to be performed by the first player object, and

eighth action control means for controlling, using the acceleration data, a state of a second player object appearing in the game space for determining an action to be performed by the second player object.

11. The game apparatus according to claim 1, wherein the smoothing means calculates the smoothed data by calculating the smoothed data each time the acquisition means acquires the acceleration data, and performing a calculation such that the value represented by the smoothed data having been most recently calculated approaches, at a predetermined rate, a value represented by the acceleration data having been acquired by the acquisition means.

12. The game apparatus according to claim 1, wherein the game process means includes

tilt calculation means for calculating, using the smoothed data, first data corresponding to a tilt of the input device including the acceleration sensor,

movement calculation means for calculating, using the acceleration data, second data corresponding to a movement of the input device, and

process execution means for executing the game process using the first data and the second data.

13. A computer-readable storage medium having stored thereon a game program executed by a computer of a game apparatus for executing a game process using acceleration data which is sequentially outputted by an acceleration sensor of an input device, the game program causing the computer to execute:

an acquisition step of sequentially acquiring the acceleration data;

a smoothing step of sequentially calculating smoothed data representing a value which is obtained by smoothing an acceleration represented by the acceleration data; and

a game process step of executing the game process using the acceleration data and the smoothed data.

14. The storage medium according to claim 13, wherein the game process step includes

a first action control step of determining, using the smoothed data, a state of a player object in a game space for determining a first action to be performed by the player object, and

a second action control step of determining, using the acceleration data, a state of the player object for determining a second action to be performed by the player object.

15. The storage medium according to claim 14, wherein

the computer determines, in the first action control step, an attitude of the player object in the game space using the smoothed data, and

the computer determines, in the second action control step, a position of the player object in the game space using the acceleration data.

16. The storage medium according to claim 15, wherein the computer determines, in the second action control step, the position of the player object in the game space by determining a movement distance using the acceleration data, and by determining a moving direction in accordance with the attitude of the player object determined in the first action control step.

17. The storage medium according to claim 13, wherein the game process step includes

a difference calculation step of calculating a difference between a value of the acceleration represented by the acceleration data and the value represented by the smoothed data, and

a process execution step of executing the game process using the smoothed data and the difference.

18. The storage medium according to claim 17, wherein the game process step includes

a third action control step of determining, using the smoothed data, a state of a player object in a game space for determining a third action to be performed by the player object, and

a fourth action control step of determining, using the difference, a state of the player object for determining a fourth action to be performed by the player object.

19. The storage medium according to claim 18, wherein

the computer determines, in the third action control step, an attitude of the player object in the game space using the smoothed data, and

the computer determines, in the fourth action control step, a position of the player object in the game space using the difference.

20. The storage medium according to claim 19, wherein the computer determines, in the fourth action control step, the position of the player object in the game space by determining a movement distance using the difference, and by determining a moving direction in accordance with the attitude of the player object determined in the third action control step.

21. The storage medium according to claim 17, wherein the game process step includes

a fifth action control step of controlling, using the smoothed data, a state of a first player object appearing in a game space for determining an action to be performed by the first player object, and

a sixth action control step of controlling, using the difference, a state of a second player object appearing in the game space for determining an action to be performed by the second player object.

22. The storage medium according to claim 13, wherein the game process step includes

a seventh action control step of controlling, using the smoothed data, a state of a first player object appearing in a game space for determining an action to be performed by the first player object, and

an eighth action control step of controlling, using the acceleration data, a state of a second player object appearing in the game space for determining an action to be performed by the second player object.

23. The storage medium according to claim 13, wherein

the smoothing step is performed each time the acceleration data is acquired, and

the computer performs, in the smoothing step, a smoothing process of calculating the smoothed data by performing a calculation by which the value represented by the smoothed data having been most recently calculated approaches, at a predetermined rate, a value represented by the acceleration data having been acquired in the acquisition step.

24. The storage medium according to claim 13, wherein the game process means includes

a tilt calculation step of calculating, using the smoothed data, first data corresponding to a tilt of the input device including the acceleration sensor,

a movement calculation step of calculating, using the acceleration data, second data corresponding to a movement of the input device, and

a process execution step of executing the game process using the first data and the second data.

25. A game computing device for executing a game process using accelerometer values which are sequentially output by an accelerometer of a pointing device, the game computing device comprising:

acquisition processing unit for sequentially acquiring the accelerometer values;

normalizing processing unit for sequentially computing normalized accelerometer values; and

game processor for executing the game process using the accelerometer values and the normalized accelerometer values.

26. The game computing device according to claim 25, wherein the game processor includes

first action control processing unit for determining, using the normalized accelerometer values, a characteristic of an object in a game space for determining a first action to be performed by the object, and

second action control processing unit for determining, using the accelerometer values, a characteristic of the object for determining a second action to be performed by the object.

27. The game computing device according to claim 26, wherein

the first action control processing unit determines an angle of the object in the game space using the normalized accelerometer values, and

the second action control processing unit determines a position of the object in the game space using the accelerometer values.

28. The game computing device according to claim 27, wherein the second action control processing unit determines the position of the object in the game space by determining a movement distance using the accelerometer values, and by determining a moving direction in accordance with the angle of the object determined by the first action control processing unit.

29. The game computing device according to claim 25, wherein the game processor includes

difference calculation processor unit for calculating a difference between a value of the acceleration represented by the accelerometer values and the value represented by the normalized accelerometer values, and

a processor for executing the game process using the normalized accelerometer values and the difference.

30. The game computing device according to claim 29, wherein the game processor includes

third action control processing unit for determining, using the normalized accelerometer values, a characteristic of an object in a game space for determining a third action to be performed by the object, and

fourth action control processing unit for determining, using the difference, a characteristic of the object for determining a fourth action to be performed by the object.

31. The game computing device according to claim 30, wherein

the third action control processing unit determines an angle of the object in the game space using the normalized accelerometer values, and

the fourth action control processing unit determines a position of the object in the game space using the difference.

32. The game computing device according to claim 31, wherein the fourth action control processing unit determines the position of the object in the game space by determining a movement distance using the difference, and by determining a moving direction in accordance with the angle of the object determined by the third action control processing unit.

33. The game computing device according to claim 30, wherein the game processor includes

fifth action control processing unit for controlling, using the normalized accelerometer values, a characteristic of a first player object appearing in the game space for determining an action to be performed by the first player object, and

sixth action control processing unit for controlling, using the difference, a characteristic of a second player object appearing in the game space for determining an action to be performed by the second player object.

34. The game computing device according to claim 25, wherein the game processor includes

seventh action control processing unit for controlling, using the normalized accelerometer values, a characteristic of a first player object appearing in a game space for determining an action to be performed by the first player object, and

eighth action control processing unit for controlling, using the accelerometer values, a characteristic of a second player object appearing in the game space for determining an action to be performed by the second player object.

35. The game computing device according to claim 25, wherein the normalizing processing unit calculates the normalized accelerometer values by calculating the normalized accelerometer values each time the acquisition processing unit acquires the accelerometer values, and performing a calculation such that the value represented by the normalized accelerometer values having been most recently calculated approaches, at a predetermined rate, a value represented by the accelerometer values having been acquired by the acquisition processing unit.

36. The game computing device according to claim 25, wherein the game processor includes

tilt calculation processing unit for calculating, using the normalized accelerometer values, first data corresponding to a tilt of the pointing device including the acceleration sensor,

movement calculation means for calculating, using the accelerometer values, second data corresponding to a movement of the pointing device, and

a processor for executing the game process using the first data and the second data.

37. A computer-readable storage medium having stored thereon a game program executed by a computer of a game computing device for executing a game process using accelerometer values which are sequentially output by an accelerometer of a pointing device, the game program causing the computer to execute:

an acquisition step of sequentially acquiring the accelerometer values;

a normalizing step of sequentially calculating normalized accelerometer values; and

a game process step of executing the game process using the accelerometer values and the normalized accelerometer values.

38. The storage medium according to claim 37, wherein the game process step includes

a first action control step of determining, using the normalized accelerometer values, a characteristic of an object in a game space for determining a first action to be performed by the object, and

a second action control step of determining, using the accelerometer values, a characteristic of the object for determining a second action to be performed by the object.

39. The storage medium according to claim 38, wherein

the computer determines, in the first action control step, an angle of the object in the game space using the normalized accelerometer values, and

the computer determines, in the second action control step, a position of the object in the game space using the accelerometer values.

40. The storage medium according to claim 39, wherein the computer determines, in the second action control step, the position of the object in the game space by determining a movement distance using the accelerometer values, and by determining a moving direction in accordance with the angle of the object determined in the first action control step.

41. The storage medium according to claim 37, wherein the game process step includes

a difference calculation step of calculating a difference between a value of the acceleration represented by the accelerometer values and the value represented by the normalized accelerometer values, and

a process execution step of executing the game process using the normalized accelerometer values and the difference.

42. The storage medium according to claim 41, wherein the game process step includes

a third action control step of determining, using the normalized accelerometer values, a characteristic of an object in a game space for determining a third action to be performed by the object, and

a fourth action control step of determining, using the difference, a characteristic of the object for determining a fourth action to be performed by the object.

43. The storage medium according to claim 42, wherein

the computer determines, in the third action control step, an angle of the object in the game space using the normalized accelerometer values, and

the computer determines, in the fourth action control step, a position of the object in the game space using the difference.

44. The storage medium according to claim 43, wherein the computer determines, in the fourth action control step, the position of the object in the game space by determining a movement distance using the difference, and by determining a moving direction in accordance with the angle of the object determined in the third action control step.

45. The storage medium according to claim 41, wherein the game process step includes

a fifth action control step of controlling, using the normalized accelerometer values, a characteristic of a first player object appearing in a game space for determining an action to be performed by the first player object, and

a sixth action control step of controlling, using the difference, a characteristic of a second player object appearing in the game space for determining an action to be performed by the second player object.

46. The storage medium according to claim 37, wherein the game process step includes

a seventh action control step of controlling, using the normalized accelerometer values, a characteristic of a first player object appearing in a game space for determining an action to be performed by the first player object, and

an eighth action control step of controlling, using the accelerometer values, a characteristic of a second player object appearing in the game space for determining an action to be performed by the second player object.

47. The storage medium according to claim 37, wherein

the normalizing step is performed each time the accelerometer values are acquired, and

the computer performs, in the normalizing step, a normalizing process of calculating the normalized accelerometer values by performing a calculation by which the value represented by the normalized accelerometer values having been most recently calculated approaches, at a predetermined rate, a value represented by the accelerometer values having been acquired in the acquisition step.

48. The storage medium according to claim 37, wherein the game processor includes

a tilt calculation step of calculating, using the normalized accelerometer values, first data corresponding to a tilt of the pointing device including the acceleration sensor,

a movement calculation step of calculating, using the accelerometer values, second data corresponding to a movement of the pointing device, and

a process execution step of executing the game process using the first data and the second data.