Manual and Camera-based Game Control

• US 20120277001A1
• Filed: 08/29/2011
• Published: 11/01/2012
• Est. Priority Date: 04/28/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lansdale, Thomas William, Griffiths, Charles Robert
• Application Number:
  US13/220,609
• Publication Number:
  US 20120277001A1
• Time in Patent Office: Days
• US Class Current: 463/39
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/638 according to the timing of operation or a time limit

Abstract

Manual and camera-based game control is described. In one embodiment, a game system receives an image stream depicting a player of a game and also receives manual input from a hand operated controller used by the player. In an example, the manual input may be used to control an avatar and the image stream is used to recognize gestures of the player and control background objects in the game using the recognized gestures. For example, by using gestures such as head rotations the player'"'"'s hands are free to continue operating a hand-held controller and the player has increased control and an immersive game play experience. In various embodiments, methods for detecting head gestures from image streams in real time are described such as tracking velocity of player'"'"'s heads and motion through quadrants of an image stream.

Claims

1. A method of controlling a computer game system comprising:

receiving a stream of images from an image capture device depicting at least one player of a game;

recognizing a gesture of the player by analyzing the stream of images;

receiving player manual input from a hand operated controller operated by the player;

displaying a computer game comprising at least two objects at a display being viewed by the at least one player;

controlling the display of one of the objects of the computer game on the basis of the recognized gesture and controlling the display of the other object of the computer game on the basis of the manual input from the controller.

2. A method as claimed in claim 1 wherein displaying the computer game comprises displaying an avatar, as one of the at least two objects, in an environment comprising at least one background object, being the other object, and wherein controlling the display of the computer game comprises using the recognized gesture to control the background object.

3. A method as claimed in claim 1 wherein the controlling the display of the background object is achieved without an associated physics model.

4. A method as claimed in claim 1 comprising storing details of specified gestures for use when recognizing the gesture and wherein the specified gestures may be performed by a player at the same time as making manual input at a hand operated controller.

5. A method as claimed in claim 2 wherein controlling the display of the background object using the recognized gesture is only possible at scripted moments of the game or on manual input from the player.

6. A method as claimed in claim 1 wherein displaying the computer game comprises displaying an avatar as one of the objects in an environment and wherein controlling the display of the computer game comprises controlling the avatar using the manual input from the controller and controlling the other object in the environment using the recognized gesture.

7. A method as claimed in claim 1 wherein recognizing the gesture comprises detecting a head of the player and tracking motion of the head in the image stream.

8. A method as claimed in claim 7 comprising dividing each image of the image stream into four quadrants and recognizing a head rotation when the head is detected in each of the four quadrants in turn.

9. A method as claimed in claim 8 wherein recognizing the head rotation comprises using a time limit.

10. A method as claimed in claim 7 which comprises recognizing a head rotation when motion of the head exceeds a specified velocity.

11. A method as claimed in claim 7 wherein recognizing a head rotation comprises tracking only horizontal motion of the head.

12. A computer game system comprising:

an image capture device arranged to receive a stream of images depicting at least one player of a game;

a gesture recognition engine arranged to recognizing a gesture of the player by analyzing the stream of images;

an input arranged to receive player manual input from a hand operated controller;

an output arranged to display a computer game comprising at least two objects at a display being viewed by the at least one player;

a processor arranged to control the display of one of the objects of the computer game on the basis of the recognized gesture and controlling the display of the other object of the computer game on the basis of the manual input from the controller.

13. A system as claimed in claim 12 wherein the processor is arranged to display an avatar, as one of the at least two objects, in an environment comprising at least one background object being the other object, and wherein controlling the display of the computer game comprises using the recognized gesture to control the background object.

14. A system as claimed in claim 12 wherein the gesture recognition engine and the input arranged to receive player manual input are arranged to operate substantially simultaneously.

15. A system as claimed in claim 12 comprises a memory storing details of specified gestures for use when recognizing the gesture and wherein the specified gestures may be performed by a player at the same time as making manual input at a hand operated controller.

16. A system as claimed in claim 13 wherein the processor is arranged to control the display of the background object using the recognized gesture only at scripted moments of the game or on specified manual input from the player.

17. A system as claimed in claim 12 wherein the gesture recognition engine is arranged to detecting a head of the player and track motion of the head in the image stream.

18. A method of controlling a computer game system comprising:

receiving a stream of images from an image capture device depicting at least one player of a game;

recognizing a gesture of the player by analyzing the stream of images, the gesture not requiring use of at least one of the player'"'"'s hands;

receiving player manual input from a hand operated controller operated by the player at the same time as recognizing the gesture;

displaying a computer game, comprising an avatar in an environment having at least one background object;

controlling the display of the avatar using the manual input from the controller and controlling the display of the background object in the environment using the recognized gesture.

19. A method as claimed in claim 18 wherein the gesture comprises a head movement and wherein recognizing the gesture comprises monitoring a velocity of the player'"'"'s head in the stream of images.

20. A method as claimed in claim 18 wherein the gesture comprises a head movement and wherein recognizing the gesture comprises monitoring motion of the player'"'"'s head in a single direction.